Reviewerly/exHarmony huggingface dataset:
Generation of Synthetic Error Data of Verb Order Errors for Swedish

We report on our work-in-progress to generate a synthetic error dataset for Swedish by replicating errors observed in the authentic errorannotated dataset.We analyze a small subset of authentic errors, capture regular patterns based on parts of speech, and design a set of rules to corrupt new data.We explore the approach and identify its capabilities, advantages and limitations as a way to enrich the existing collection of error-annotated data.This work focuses on word order errors, specifically those involving the placement of finite verbs in a sentence.
Providing Agile Faculty Development in Times of Uncertainty: Case of the American University in Cairo

Abstract The move toward emergency remote teaching meant educational development centers suddenly had to train and support all who teach at their institutions simultaneously. This article will focus on the agile, responsive and value-centric faculty development done locally via the American University in Cairo’s Center for Learning &amp; Teaching, the “glocal” opportunities offered via DigPINS (see Bali and Caines, Int J Educ Technol High Educ 15(46):1–24, 2018), and the curation of openly available community-building resources available to educators worldwide. Central to all of these initiatives is that centering equity and care in how we support faculty (Czerniewicz, Agherdien, Badenhorst, Postdigit Sci Educ 2:946–967, 2020) will trickle down to the ways faculty treat their students during the trauma of the pandemic. We conclude that the pandemic has taught us the importance of centering values of equity and care while supporting faculty during a time of uncertainty and trauma (Imad, TIA 39, 2021), that fostering agency and imagination is more valuable than offering one-size-fits all standard solutions, and that faculty developers need to model adaptability and good pedagogy. Moreover, it is important to nurture and leverage learning communities, take advantage of “glocal” and “open” learning opportunities, and to build capability long-term via developing digital literacies and creativity.
Lightweight Transformers for Conversational AI

Daniel Pressel, Wenshuo Liu, Michael Johnston, Minhua Chen. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track. 2022.
Unsupervised Paraphrasability Prediction for Compound Nominalizations

Commonly found in academic and formal texts, a nominalization uses a deverbal noun to describe an event associated with its corresponding verb.Nominalizations can be difficult to interpret because of ambiguous semantic relations between the deverbal noun and its arguments.Automatic generation of clausal paraphrases for nominalizations can help disambiguate their meaning.However, previous work has not identified cases where it is awkward or impossible to paraphrase a compound nominalization.This paper investigates unsupervised prediction of paraphrasability, which determines whether the prenominal modifier of a nominalization can be re-written as a noun or adverb in a clausal paraphrase.We adopt the approach of overgenerating candidate paraphrases followed by candidate ranking with a neural language model.In experiments on an English dataset, we show that features from an Abstract Meaning Representation graph lead to statistically significant improvement in both paraphrasability prediction and paraphrase generation.
Contrastive Representation Learning for Cross-Document Coreference Resolution of Events and Entities

Identifying related entities and events within and across documents is fundamental to natural language understanding.We present an approach to entity and event coreference resolution utilizing contrastive representation learning.Earlier state-of-the-art methods have formulated this problem as a binary classification problem and leveraged large transformers in a cross-encoder architecture to achieve their results.For large collections of documents and corresponding set of n mentions, the necessity of performing n 2 transformer computations in these earlier approaches can be computationally intensive.We show that it is possible to reduce this burden by applying contrastive learning techniques that only require n transformer computations at inference time.Our method achieves state-of-the-art results on a number of key metrics on the ECB+ corpus and is competitive on others.
”Diversity and Uncertainty in Moderation” are the Key to Data Selection for Multilingual Few-shot Transfer

Few-shot transfer often shows substantial gain over zero-shot transfer (Lauscher et al., 2020), which is a practically useful trade-off between fully supervised and unsupervised learning approaches for multilingual pretained modelbased systems.This paper explores various strategies for selecting data for annotation that can result in a better few-shot transfer.The proposed approaches rely on multiple measures such as data entropy using n-gram language model, predictive entropy, and gradient embedding.We propose a loss embedding method for sequence labeling tasks, which induces diversity and uncertainty sampling similar to gradient embedding.The proposed data selection strategies are evaluated and compared for POS tagging, NER, and NLI tasks for up to 20 languages.Our experiments show that the gradient and loss embedding-based strategies consistently outperform random data selection baselines, with gains varying with the initial performance of the zero-shot transfer.Furthermore, the proposed method shows similar trends in improvement even when the model is fine-tuned using a lower proportion of the original task-specific labeled training data for zero-shot transfer.
Narrative Datasets through the Lenses of NLP and HCI

In this short paper, we compare existing value systems and approaches in NLP and HCI for collecting narrative data.Building on these parallel discussions, we shed light on the challenges facing some popular NLP dataset types, which we discuss these in relation to widelyused narrative-based HCI research methods; and we highlight points where NLP methods can broaden qualitative narrative studies.In particular, we point towards contextuality, positionality, dataset size, and open research design as central points of difference and windows for collaboration when studying narratives.Through the use case of narratives, this work contributes to a larger conversation regarding the possibilities for bridging NLP and HCI through speculative mixed-methods.
Morfessor-enriched features and multilingual training for canonical morphological segmentation

In our submission to the SIGMORPHON 2022 Shared Task on Morpheme Segmentation, we study whether an unsupervised morphological segmentation method, Morfessor, can help in a supervised setting.Previous research has shown the effectiveness of the approach in semisupervised settings with small amounts of labeled data.The current tasks vary in data size: the amount of word-level annotated training data is much larger, but the amount of sentencelevel annotated training data remains small.Our approach is to pre-segment the input data for a neural sequence-to-sequence model with the unsupervised method.As the unsupervised method can be trained with raw text data, we use Wikipedia to increase the amount of training data.In addition, we train multilingual models for the sentence-level task.The results for the Morfessor-enriched features are mixed, showing benefit for all three sentencelevel tasks but only some of the word-level tasks.The multilingual training yields considerable improvements over the monolingual sentence-level models, but it negates the effect of the enriched features.
TechSSN at SemEval-2022 Task 6: Intended Sarcasm Detection using Transformer Models

Irony detection in the social media is an upcoming research which places a main role in sentiment analysis and offensive languague identification.Sarcasm is one form of irony that is used to provide intended comments against realism.This paper describes a method to detect intended sarcasm in text (SemEval-2022 Task 6).The TECHSSN team used Bidirectional Encoder Representations from Transformers (BERT) models and its variants to classify the text as sarcastic or non-sarcastic in English and Arabic languages.The data is preprocessed and fed to the model for training.The transformer models learn the weights during the training phase from the given dataset and predicts the output class labels for the unseen test data.
Empathic Machines: Using Intermediate Features as Levers to Emulate Emotions in Text-To-Speech Systems

We present a method to control the emotional prosody of Text to Speech (TTS) systems by using phoneme-level intermediate features (pitch, energy, and duration) as levers.As a key idea, we propose Differential Scaling (DS) to disentangle features relating to affective prosody from those arising due to acoustics conditions and speaker identity.With thorough experimental studies, we show that the proposed method improves over the prior art in accurately emulating the desired emotions while retaining the naturalness of speech.We extend the traditional evaluation of using individual sentences for a more complete evaluation of HCI systems.We present a novel experimental setup by replacing an actor with a TTS system in offline and live conversations.The emotion to be rendered is either predicted or manually assigned.The results show that the proposed method is strongly preferred over the state-of-the-art TTS system and adds the much-coveted "human touch" in machine dialogue.Audio samples for our experiments and the code are available at:
Understand before Answer: Improve Temporal Reading Comprehension via Precise Question Understanding

This work studies temporal reading comprehension (TRC), which reads a free-text passage and answers temporal ordering questions.Precise question understanding is critical for temporal reading comprehension.For example, the question "What happened before the victory" and "What happened after the victory" share almost all words except one, while their answers are totally different.Moreover, even if two questions query about similar temporal relations, different varieties might also lead to various answers.For example, although both the question "What usually happened during the press release?" and "What might happen during the press release" query events which happen after the press release, they convey divergent semantics.To this end, we propose a novel reading comprehension approach with precise question understanding.Specifically, a temporal ordering question is embedded into two vectors to capture the referred event and the temporal relation.Then we evaluate the temporal relation between candidate events and the referred event based on that.Such fine-grained representations offer two benefits.First, it enables a better understanding of the question by focusing on different elements of a question.Second, it provides good interpretability when evaluating temporal relations.Furthermore, we also harness an auxiliary contrastive loss for representation learning of temporal relations, which aims to distinguish relations with subtle but critical changes.The proposed approach outperforms strong baselines and achieves state-of-the-art performance on the TORQUE dataset.It also increases the accuracy of four pre-trained language models (BERT base, BERT large, RoBERTa base, and RoBETRa large), demonstrating its generic effectiveness on divergent models.
Understanding the Sarcastic Nature of Emojis with SarcOji

Identifying sarcasm is a challenging research problem owing to its highly contextual nature.Several researchers have attempted numerous mechanisms to incorporate context, linguistic aspects, and supervised and semi-supervised techniques to determine sarcasm.It has also been noted that emojis in a text may also hold key indicators of sarcasm.However, the availability of sarcasm datasets with emojis is scarce.This makes it challenging to effectively study the sarcastic nature of emojis.In this work, we present SarcOji which has been compiled from five publicly available sarcasm datasets.SarcOji contains labeled English texts which all have emojis.We also analyze SarcOji to determine if there is an incongruence in the polarity of text and emojis used therein.Further, emojis' usage, occurrences, and positions in the context of sarcasm are also studied in this compiled dataset.With SarcOji we have been able to demonstrate that frequency of occurrence of an emoji and its position are strong indicators of sarcasm.SarcOji dataset is now publicly available with several derived features like sentiment scores of text and emojis, most frequent emoji, and its position in the text.Compilation of the SarcOji dataset is an initial step to enable the study of the role of emojis in communicating sarcasm.SarcOji dataset can also serve as a go-to dataset for various emoji-based sarcasm detection techniques.
A Human-machine Interface for Few-shot Rule Synthesis for Information Extraction

We propose a system that assists a user in constructing transparent information extraction models, consisting of patterns (or rules) written in a declarative language, through program synthesis.Users of our system can specify their requirements through the use of examples, which are collected with a search interface.The rule-synthesis system proposes rule candidates and the results of applying them on a textual corpus; the user has the option to accept the candidate, request another option, or adjust the examples provided to the system.Through an interactive evaluation, we show that our approach generates high-precision rules even in a 1-shot setting.On a second evaluation on a widely-used relation extraction dataset (TACRED), our method generates rules that outperform considerably manually written patterns.Our code, demo, and documentation is available at https: //clulab.github.io/odinsynth/.
NamedEntityRangers at SemEval-2022 Task 11: Transformer-based Approaches for Multilingual Complex Named Entity Recognition

This paper presents the two submissions of NamedEntityRangers Team to the Multi-CoNER Shared Task, hosted at SemEval-2022.We evaluate two state-of-the-art approaches, of which both utilize pre-trained multi-lingual language models differently.The first approach follows the token classification schema, in which each token is assigned with a tag.The second approach follows a recent templatefree paradigm (Ma et al., 2021), in which an encoder-decoder model translates the input sequence of words to a special output, encoding named entities with predefined labels.We utilize RemBERT and mT5 as backbone models for these two approaches, respectively.Our results show that the oldie but goodie token classification outperforms the template-free method by a wide margin.Our code is available at: https://github.com/Abiks/MultiCoNER.
Intent Discovery for Enterprise Virtual Assistants: Applications of Utterance Embedding and Clustering to Intent Mining

A key challenge in the creation and refinement of virtual assistants is the ability to mine unlabeled utterance data to discover common intents.We develop an approach to this problem that combines large-scale pre-training and multi-task learning to derive a semantic embedding that can be leveraged to identify clusters of utterances that correspond to unhandled intents.An utterance encoder is first trained with a language modeling objective and subsequently adapted to predict intent labels from a large collection of cross-domain enterprise virtual assistant data using a multi-task cosine softmax loss.Experimental evaluation shows significant advantages for this multi-step pretraining approach, with large gains in downstream clustering accuracy on new applications compared to standard sentence embedding approaches.The approach has been incorporated into an interactive discovery tool that enables visualization and exploration of intents by system analysts and builders.
ParaNames: A Massively Multilingual Entity Name Corpus

We present ParaNames, a Wikidata-derived multilingual parallel name resource consisting of over 118 million names for 13.7 million entities, spanning over 400 languages.ParaNames is useful for multilingual language processing, both for defining name translation tasks and as supplementary data for other tasks.We demonstrate an application of ParaNames by training a multilingual model for canonical name translation to and from English.
Q2R: A Query-to-Resolution System for Natural-Language Queries

We present a system for document retrieval that combines direct classification with standard content-based retrieval approaches to significantly improve the relevance of the retrieved documents.Our system exploits the availability of an imperfect but sizable amount of labeled data from past queries.For domains such as technical support, the proposed approach enhances the system's ability to retrieve documents that are otherwise ranked very low based on content alone.The system is easy to implement and can make use of existing text ranking methods, augmenting them through the novel Q2R orchestration framework.Q2R has been extensively tested and is in use at IBM.
FAtNet: Cost-Effective Approach Towards Mitigating the Linguistic Bias in Speaker Verification Systems

Linguistic bias in Deep Neural Network (DNN) based Natural Language Processing (NLP) systems is a critical problem that needs attention.The problem further intensifies in the case of security systems, such as speaker verification, where fairness is essential.Speaker verification systems are intelligent systems that determine if two speech recordings belong to the same speaker.Such human-oriented security systems should be usable by diverse people speaking varied languages.Thus, a speaker verification system trained on speech in one language should generalize when tested for other languages.However, DNN-based models are often language-dependent.Previous works explore domain adaptation to fine-tune the pre-trained model for out-of-domain languages.Fine-tuning the model individually for each existing language is expensive.Hence, it limits the usability of the system.This paper proposes the cost-effective idea of integrating a lightweight embedding with existing speaker verification systems to mitigate linguistic bias without adaptation.This work is motivated by the theoretical hypothesis that attentive-frames could help generate language-agnostic embeddings.For scientific validation of this hypothesis, we propose two frame-attentive networks and investigate the effect of their integration with baselines for twelve languages.Empirical results suggest that frame-attentive embedding can cost-effectively reduce linguistic bias and enhance the usability of baselines.
niksss at SemEval-2022 Task7:Transformers for Grading the Clarifications on Instructional Texts

This paper describes the 9th place system description for SemEval-2022 Task 7. The goal of this shared task was to develop computational models to predict how plausible a clarification made on an instructional text is.This shared task was divided into two Subtasks A and B. We attempted to solve these using various transformers-based architecture under different regime.We initially treated this as a text2text generation problem but comparing it with our recent approach we dropped it and treated this as a text-sequence classification and regression depending on the Subtask.
UU-Tax at SemEval-2022 Task 3: Improving the generalizability of language models for taxonomy classification through data augmentation

This paper presents our strategy to address the SemEval-2022 Task 3 PreTENS: Presupposed Taxonomies Evaluating Neural Network Semantics. The goal of the task is to identify if a sentence is deemed acceptable or not, depending on the taxonomic relationship that holds between a noun pair contained in the sentence. For sub-task 1 -- binary classification -- we propose an effective way to enhance the robustness and the generalizability of language models for better classification on this downstream task. We design a two-stage fine-tuning procedure on the ELECTRA language model using data augmentation techniques. Rigorous experiments are carried out using multi-task learning and data-enriched fine-tuning. Experimental results demonstrate that our proposed model, UU-Tax, is indeed able to generalize well for our downstream task. For sub-task 2 -- regression -- we propose a simple classifier that trains on features obtained from Universal Sentence Encoder (USE). In addition to describing the submitted systems, we discuss other experiments that employ pre-trained language models and data augmentation techniques. For both sub-tasks, we perform error analysis to further understand the behaviour of the proposed models. We achieved a global F1_Binary score of 91.25% in sub-task 1 and a rho score of 0.221 in sub-task 2.
Aligning Generative Language Models with Human Values

Although current large-scale generative language models (LMs) can show impressive insights about factual knowledge, they do not exhibit similar success with respect to human values judgements (e.g., whether or not the generations of an LM are moral).Existing methods learn human values either by directly mimicking the behavior of human data, or rigidly constraining the generation space to human-chosen tokens.These methods are inherently limited in that they do not consider the contextual and abstract nature of human values and as a result often fail when dealing with out-of-domain context or sophisticated and abstract human values.This paper proposes SENSEI, a new reinforcement learning based method that can embed human values judgements into each step of language generation.SENSEI deploys an Actor-Critic framework, where the Critic is a reward distributor that simulates the reward assignment procedure of humans, while the Actor guides the generation towards the maximum reward direction.Compared with five existing methods in three human values alignment datasets, SENSEI not only achieves higher alignment performance in terms of both automatic and human evaluations, but also shows improvements on robustness and transfer learning on unseen human values.
INF-UFRGS at SemEval-2022 Task 5: analyzing the performance of multimodal models

This paper describes INF-UFRGS submission for SemEval-2022 Task 5 Multimodal Automatic Misogyny Identification (MAMI).Unprecedented levels of harassment came with the ever-growing internet usage as a means of worldwide communication.The goal of MAMI is to improve the quality of existing methods for misogyny identification, many of which require dedicated personnel, hence the need for automation.We experimented with five existing models, including ViLBERT and VisualBERT -both uni and multimodally pretrained -and MMBT.The datasets consist of memes with captions in English.The results show that all models achieved Macro-F1 scores above 0.64.ViLBERT was the best performer with a score of 0.698.
Assessing the Limits of the Distributional Hypothesis in Semantic Spaces: Trait-based Relational Knowledge and the Impact of Co-occurrences

The increase in performance in NLP due to the prevalence of distributional models and deep learning has brought with it a reciprocal decrease in interpretability.This has spurred a focus on what neural networks learn about natural language with less of a focus on how.Some work has focused on the data used to develop data-driven models, but typically this line of work aims to highlight issues with the data, e.g.highlighting and offsetting harmful biases.This work contributes to the relatively untrodden path of what is required in data for models to capture meaningful representations of natural language.This entails evaluating how well English and Spanish semantic spaces capture a particular type of relational knowledge, namely the traits associated with concepts (e.g.bananas-yellow), and exploring the role of cooccurrences in this context.