Reviewerly/exHarmony huggingface dataset:
An Extended Agglomerative Hierarchical Clustering Techniques

Clustering is a significant method of data analytics in real world environments since human labelling of the data is often costly. Clustering was developed as an alternative to manual tagging. In the field of data analytics, hierarchical clustering is of critical significance, particularly in light of the exponential rise of data derived from the normal world. You may derive a variety of hierarchical agglomerative clustering algorithms from this architecture by providing an inter-cluster semantic similarity, an expression patterns of the -similarity graph, and a cover procedure. These three pieces of information are required. According to the findings of our experiments, our approaches are not only more efficient than conventional hierarchical algorithms, but they also produce smaller agglomerative hierarchical clustering while maintaining the same level of clustering effectiveness. It is generally agreed that topology management is an effective strategy for addressing these challenges. This method groups nodes together for the purpose of managing them and/or carrying out a variety of duties in a dispersed way, such as resource management. There are many quality-driven goals that may be accomplished by clustering, despite the fact that approaches for clustering are mostly renowned for their ability to reduce energy usage. The purpose of this study is to provide a comprehensive explanation on various enhanced agglomerative hierarchical clustering techniques. In addition to this, the authors have provided certain criteria, on the basis of which one may also assess which of these previously described algorithms is the most effective.
Implementation of Novel Test Rank Algorithm for Effective Text Summarization

Despite their busy schedule, people often wish to know the recent updates of what’s going on around with-in a short span of time. We are often amazed to know what could be the technology behind this technological innovation. Here comes into light Text Encapsulation. To accomplish the task of encapsulating the text into "Manually Tag Articles" or "Design an Automated System for phrase extraction". As the amount of unstructured text data in the digital world grows, automatic summarizing systems are needed that will be possible for people to quickly understand the source material. We require an extractive NLP model that is capable of doing the task on a wide scale. We use text ranking algorithm for text extraction where the number of common words measure's the similarity between two sentences. It gives the most informative document or summary which is also used to find the most relevant sentences in the text.
DimonGen: Diversified Generative Commonsense Reasoning for Explaining Concept Relationships

In this paper, we propose DimonGen, which aims to generate diverse sentences describing concept relationships in various everyday scenarios.To support this, we first create a benchmark dataset for this task by adapting the existing CommonGen dataset.We then propose a two-stage model called MoREE to generate the target sentences.MoREE consists of a mixture of retrievers model that retrieves diverse context sentences related to the given concepts, and a mixture of generators model that generates diverse sentences based on the retrieved contexts.We conduct experiments on the DimonGen task and show that MoREE outperforms strong baselines in terms of both the quality and diversity of the generated sentences.Our results demonstrate that MoREE is able to generate diverse sentences that reflect different relationships between concepts, leading to a comprehensive understanding of concept relationships. 1
Why Aren’t We NER Yet? Artifacts of ASR Errors in Named Entity Recognition in Spontaneous Speech Transcripts

Transcripts of spontaneous human speech present a significant obstacle for traditional NER models.The lack of grammatical structure of spoken utterances and word errors introduced by the ASR make downstream NLP tasks challenging.In this paper, we examine in detail the complex relationship between ASR and NER errors which limit the ability of NER models to recover entity mentions from spontaneous speech transcripts.Using publicly available benchmark datasets (SWNE, Earnings-21, OntoNotes), we present the full taxonomy of ASR-NER errors and measure their true impact on entity recognition.We find that NER models fail to recognize entity spans even if no word errors are introduced by the ASR.We also show why the F 1 score is inadequate to evaluate NER models on conversational transcripts 1 .
miCSE: Mutual Information Contrastive Learning for Low-shot Sentence Embeddings

This paper presents miCSE, a mutual information-based contrastive learning framework that significantly advances the stateof-the-art in few-shot sentence embedding.The proposed approach imposes alignment between the attention pattern of different views during contrastive learning.Learning sentence embeddings with miCSE entails enforcing the structural consistency across augmented views for every sentence, making contrastive self-supervised learning more sample efficient.As a result, the proposed approach shows strong performance in the few-shot learning domain.While it achieves superior results compared to state-of-the-art methods on multiple benchmarks in few-shot learning, it is comparable in the full-shot scenario.This study opens up avenues for efficient self-supervised learning methods that are more robust than current contrastive methods for sentence embedding.
Contextual Distortion Reveals Constituency: Masked Language Models are Implicit Parsers

Recent advancements in pre-trained language models (PLMs) have demonstrated that these models possess some degree of syntactic awareness.To leverage this knowledge, we propose a novel chart-based method for extracting parse trees from masked language models (LMs) without the need to train separate parsers.Our method computes a score for each span based on the distortion of contextual representations resulting from linguistic perturbations.We design a set of perturbations motivated by the linguistic concept of constituency tests, and use these to score each span by aggregating the distortion scores.To produce a parse tree, we use chart parsing to find the tree with the minimum score.Our method consistently outperforms previous state-of-the-art methods on English with masked LMs, and also demonstrates superior performance in a multilingual setting, outperforming the state of the art in 6 out of 8 languages.Notably, although our method does not involve parameter updates or extensive hyperparameter search, its performance can even surpass some unsupervised parsing methods that require fine-tuning.Our analysis highlights that the distortion of contextual representation resulting from syntactic perturbation can serve as an effective indicator of constituency across languages. 1 Further details of performance comparison by layer on the SPMRL dataset can be found in Appendix A.2.12 Results on other languages can be found in Appendix A.3 13 We conduct our analysis on the BERT-base model.The full results can be seen in Appendix A.3.
Won’t Get Fooled Again: Answering Questions with False Premises

Pre-trained language models (PLMs) have shown unprecedented potential in various fields, especially as the backbones for questionanswering (QA) systems.However, they tend to be easily deceived by tricky questions such as "How many eyes does the sun have?".Such frailties of PLMs often allude to the lack of knowledge within them.In this paper, we find that the PLMs already possess the knowledge required to rebut such questions, and the key is how to activate the knowledge.To systematize this observation, we investigate the PLMs' responses to one kind of tricky questions, i.e., the false premises questions (FPQs).We annotate a FalseQA dataset containing 2365 human-written FPQs, with the corresponding explanations for the false premises and the revised true premise questions.Using FalseQA, we discover that PLMs are capable of discriminating FPQs by fine-tuning on moderate numbers (e.g., 256) of examples.PLMs also generate reasonable explanations for the false premise, which serve as rebuttals.Further replaying a few general questions during training allows PLMs to excel on FPQs and general questions simultaneously.Our work suggests that once the rebuttal ability is stimulated, knowledge inside the PLMs can be effectively utilized to handle FPQs, which incentivizes the research on PLM-based QA systems.The FalseQA dataset and code are available at https://github.com/thunlp/FalseQA.
Mixture-of-Domain-Adapters: Decoupling and Injecting Domain Knowledge to Pre-trained Language Models’ Memories

Pre-trained language models (PLMs) demonstrate excellent abilities to understand texts in the generic domain while struggling in a specific domain.Although continued pre-training on a large domain-specific corpus is effective, it is costly to tune all the parameters on the domain.In this paper, we investigate whether we can adapt PLMs both effectively and efficiently by only tuning a few parameters.Specifically, we decouple the feed-forward networks (FFNs) of the Transformer architecture into two parts: the original pre-trained FFNs to maintain the old-domain knowledge and our novel domain-specific adapters to inject domainspecific knowledge in parallel.Then we adopt a mixture-of-adapters gate to fuse the knowledge from different domain adapters dynamically.Our proposed Mixture-of-Domain-Adapters (MixDA) employs a two-stage adapter-tuning strategy that leverages both unlabeled data and labeled data to help the domain adaptation: i) domain-specific adapter on unlabeled data; followed by ii) the task-specific adapter on labeled data.MixDA can be seamlessly plugged into the pretraining-finetuning paradigm and our experiments demonstrate that MixDA achieves superior performance on in-domain tasks (GLUE), out-of-domain tasks (ChemProt, RCT, IMDB, Amazon), and knowledge-intensive tasks (KILT).Further analyses demonstrate the reliability, scalability, and efficiency of our method.
DeTexD: A Benchmark Dataset for Delicate Text Detection

Over the past few years, much research has been conducted to identify and regulate toxic language. However, few studies have addressed a broader range of sensitive texts that are not necessarily overtly toxic. In this paper, we introduce and define a new category of sensitive text called “delicate text.” We provide the taxonomy of delicate text and present a detailed annotation scheme. We annotate DeTexD, the first benchmark dataset for delicate text detection. The significance of the difference in the definitions is highlighted by the relative performance deltas between models trained each definitions and corpora and evaluated on the other. We make publicly available the DeTexD Benchmark dataset, annotation guidelines, and baseline model for delicate text detection.
Randomized Smoothing with Masked Inference for Adversarially Robust Text Classifications

Large-scale pre-trained language models have shown outstanding performance in a variety of NLP tasks.However, they are also known to be significantly brittle against specifically crafted adversarial examples, leading to increasing interest in probing the adversarial robustness of NLP systems.We introduce RSMI, a novel two-stage framework that combines randomized smoothing (RS) with masked inference (MI) to improve the adversarial robustness of NLP systems.RS transforms a classifier into a smoothed classifier to obtain robust representations, whereas MI forces a model to exploit the surrounding context of a masked token in an input sequence.RSMI improves adversarial robustness by 2 to 3 times over existing state-of-the-art methods on benchmark datasets.We also perform in-depth qualitative analysis to validate the effectiveness of the different stages of RSMI and probe the impact of its components through extensive ablations.By empirically proving the stability of RSMI, we put it forward as a practical method to robustly train large-scale NLP models.Our code and datasets are available at https://github.com/Han8931/rsmi_nlp.
Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment

Despite recent progress towards scaling up multimodal vision-language models, these models are still known to struggle on compositional generalization benchmarks such as Winoground.We find that a critical component lacking from current vision-language models is relation-level alignment: the ability to match directional semantic relations in text (e.g., 'mug in grass') with spatial relationships in the image (e.g., the position of the mug relative to the grass).To tackle this problem, we show that relation alignment can be enforced by encouraging the language attention from 'mug' to 'grass' (capturing the semantic relation 'in') to match the visual attention from the mug to the grass (capturing the corresponding physical relation).Tokens and their corresponding objects are softly identified using a weighted mean of cross-modal attention.We prove that this notion of soft cross-modal equivalence is equivalent to enforcing congruence between vision and language attention matrices under a 'change of basis' provided by the cross-modal attention matrix.Intuitively, our approach projects visual attention into the language attention space to calculate its divergence from the actual language attention, and vice versa.We apply our Cross-modal Attention Congruence Regularization (CACR) loss to fine-tune UNITER and improve its Winoground Group score by 5.75 points.
Unsupervised Discontinuous Constituency Parsing with Mildly Context-Sensitive Grammars

We study grammar induction with mildly context-sensitive grammars for unsupervised discontinuous parsing.Using the probabilistic linear context-free rewriting system (LCFRS) formalism, our approach fixes the rule structure in advance and focuses on parameter learning with maximum likelihood.To reduce the computational complexity of both parsing and parameter estimation, we restrict the grammar formalism to binary LCFRS with fan-out two and further discard rules that require O(ℓ 6 ) time to parse, reducing inference to O(ℓ 5 ).We find that using a large number of nonterminals is beneficial and thus make use of tensor decomposition-based rank-space dynamic programming with an embedding-based parameterization of rule probabilities to scale up the number of nonterminals.Experiments on German and Dutch show that our approach is able to induce linguistically meaningful trees with continuous and discontinuous structures.
Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation

A challenge in the Dialogue State Tracking (DST) field is adapting models to new domains without using any supervised data -zero-shot domain adaptation.Parameter-Efficient Transfer Learning (PETL) has the potential to address this problem due to its robustness.However, it has yet to be applied to the zero-shot scenarios, as it is not clear how to apply it unsupervisedly.Our method, Prompter, uses descriptions of target domain slots to generate dynamic prefixes that are concatenated to the key and values at each layer's self-attention mechanism.This allows for the use of prefix-tuning in zeroshot.Prompter outperforms previous methods on both the MultiWOZ and SGD benchmarks.In generating prefixes, our analyses find that Prompter not only utilizes the semantics of slot descriptions but also how often the slots appear together in conversation.Moreover, Prompter's gains are due to its improved ability to distinguish "none"-valued dialogue slots, compared against baselines.
Identity Construction in a Misogynist Incels Forum

Online communities of involuntary celibates (incels) are a prominent source of misogynist hate speech.In this paper, we use quantitative text and network analysis approaches to examine how identity groups are discussed on incels.is, the largest black-pilled incels forum.We find that this community produces a wide range of novel identity terms and, while terms for women are most common, mentions of other minoritized identities are increasing.An analysis of the associations made with identity groups suggests an essentialist ideology where physical appearance, as well as gender and racial hierarchies, determine human value.We discuss implications for research into automated misogynist hate speech detection.
Improving Language Model Integration for Neural Machine Translation

The integration of language models for neural machine translation has been extensively studied in the past.It has been shown that an external language model, trained on additional target-side monolingual data, can help improve translation quality.However, there has always been the assumption that the translation model also learns an implicit target-side language model during training, which interferes with the external language model at decoding time.Recently, some works on automatic speech recognition have demonstrated that, if the implicit language model is neutralized in decoding, further improvements can be gained when integrating an external language model.In this work, we transfer this concept to the task of machine translation and compare with the most prominent way of including additional monolingual data -namely backtranslation.We find that accounting for the implicit language model significantly boosts the performance of language model fusion, although this approach is still outperformed by back-translation.
Toward Interactive Dictation

Voice dictation is an increasingly important text input modality.Existing systems that allow both dictation and editing-by-voice restrict their command language to flat templates invoked by trigger words.In this work, we study the feasibility of allowing users to interrupt their dictation with spoken editing commands in open-ended natural language.We introduce a new task and dataset, TERTiUS, to experiment with such systems.To support this flexibility in real-time, a system must incrementally segment and classify spans of speech as either dictation or command, and interpret the spans that are commands.We experiment with using large pre-trained language models to predict the edited text, or alternatively, to predict a small text-editing program.Experiments show a natural trade-off between model accuracy and latency: a smaller model achieves 28% singlecommand interpretation accuracy with 1.3 seconds of latency, while a larger model achieves 55% with 7 seconds of latency.* Work performed during a research internship at Microsoft Semantic Machines.Just wanted to ask about the event on Friday the 23rd.Is the event still on?Just wanted to ask about the event on the 23rd, on Friday the 23rd.Is the event still on?Change"the event" to "it" in the last sentence.Just wanted to ask about the event on
On Evaluating and Mitigating Gender Biases in Multilingual Settings

While understanding and removing gender biases in language models has been a longstanding problem in Natural Language Processing, prior research work has primarily been limited to English.In this work, we investigate some of the challenges with evaluating and mitigating biases in multilingual settings which stem from a lack of existing benchmarks and resources for bias evaluation beyond English especially for non-western context.In this paper, we first create a benchmark for evaluating gender biases in pre-trained masked language models by extending DisCo to different Indian languages using human annotations.We extend various debiasing methods to work beyond English and evaluate their effectiveness for SOTA massively multilingual models on our proposed metric.Overall, our work highlights the challenges that arise while studying social biases in multilingual settings and provides resources as well as mitigation techniques to take a step toward scaling to more languages.
SAB at SemEval-2023 Task 2: Does Linguistic Information Aid in Named Entity Recognition?

This paper describes the submission to SemEval-2023 Task 2: Multilingual Complex Named Entity Recognition (MultiCoNER II) by team SAB.This task aims to encourage growth in the field of Named Entity Recognition (NER) by focusing on complex and difficult categories of entities, in 12 different language tracks.The task of NER has historically shown the best results when a model incorporates an external knowledge base or gazetteer, however, less research has been applied to examining the effects of incorporating linguistic information into the model.In this task, we explored combining NER, part-of-speech (POS), and dependency relation labels into a multi-task model and report on the findings.We determine that the addition of POS and dependency relation information in this manner does not improve results.
SORTIE: Dependency-Aware Symbolic Reasoning for Logical Data-to-text Generation

Logical data-to-text generation is a representative task in measuring the capabilities of both language generation and complex reasoning.Despite the introduction of reasoning skills in generation, existing works still rely on neural language models to output the final table description.However, due to the inefficacy of neural language models in complex reasoning, these methods inevitably have difficulty working out key entities in the description and might produce unfaithful descriptions.To alleviate these issues, we propose a dependencyaware symbolic reasoning framework that reasons out each entity in the table description with our designed table-compatible programming language.To figure out the dependency relationship among entities, we devise an entity scheduling mechanism to determine the order of programme synthesis such that the reasoning of an entity only relies on other "resolved" entities.Experiments on three datasets and three backbones show that ours outperforms previous methods not only in surfacelevel fidelity but also in logical fidelity.Notably, the proposed framework enhances GPT-2, BART and T5 with an absolute improvement of 5.7% ∼ 11.5% on SP-Acc.
Learning from Children: Improving Image-Caption Pretraining via Curriculum

Image-caption pretraining has been quite successfully used for downstream vision tasks like zero-shot image classification and object detection.However, image-caption pretraining is still a hard problem -it requires multiple concepts (nouns) from captions to be aligned to several objects in images.To tackle this problem, we go to the roots -the best learner, children.We take inspiration from cognitive science studies dealing with children's language learning to propose a curriculum learning framework.The learning begins with easy-to-align image caption pairs containing one concept per caption.The difficulty is progressively increased with each new phase by adding one more concept per caption.Correspondingly, the knowledge acquired in each learning phase is utilized in subsequent phases to effectively constrain the learning problem to aligning one new concept-object pair in each phase.We show that this learning strategy improves over vanilla image-caption training in various settings -pretraining from scratch, using a pretrained image or/and pretrained text encoder, low data regime etc.
UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization

Yulong Chen, Yang Liu, Ruochen Xu, Ziyi Yang, Chenguang Zhu, Michael Zeng, Yue Zhang. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.