Reviewerly/exHarmony huggingface dataset:
Exploiting Document Structures and Cluster Consistencies for Event Coreference Resolution

We study the problem of event coreference resolution (ECR) that seeks to group coreferent event mentions into the same clusters.Deep learning methods have recently been applied for this task to deliver state-of-the-art performance.However, existing deep learning models for ECR are limited in that they cannot exploit important interactions between relevant objects for ECR, e.g., context words and entity mentions, to support the encoding of document-level context.In addition, consistency constraints between golden and predicted clusters of event mentions have not been considered to improve representation learning in prior deep learning models for ECR.This work addresses such limitations by introducing a novel deep learning model for ECR.At the core of our model are document structures to explicitly capture relevant objects for ECR.Our document structures introduce diverse knowledge sources (discourse, syntax, semantics) to compute edges/interactions between structure nodes for document-level representation learning.We also present novel regularization techniques based on consistencies of golden and predicted clusters for event mentions in documents.Extensive experiments show that our model achieve state-of-the-art performance on two benchmark datasets.
Moroccan High School EFL Teachers’ Attitudes and Anxiety on Using Microsoft Teams Platform

The world has witnessed major changes in delivery modes in education due to the COVID-19 pandemic. Being no exception, Morocco declared a full lockdown in March 2020, and this resulted in the use of online platforms to cope with the new situation. The platform that was adopted by the Moroccan Ministry of Education was Microsoft Teams (MT). Hence, the current paper investigated Moroccan high school EFL teachers’ use of the platform accounting for their age, gender, anxiety, and attitudes towards the platform. For this aim, a questionnaire adapted from Computer Anxiety Rating Scale (CARS) by Heinssen et al. (1987) was distributed to 171 EFL teachers working in different public high schools belonging to the academy of Casablanca/Settat. Results revealed that the participants’ attitudes towards the usability of the platform were negatively associated with their years of experience, and their general anxiety and power and control of the platform negatively affected the time they spent on the platform. The findings imply that anxiety should be taken into consideration as an integral component in further implementation of technology in education.
Self-supervised Pre-training and Contrastive Representation Learning for Multiple-choice Video QA

Video Question Answering (VideoQA) requires fine-grained understanding of both video and language modalities to answer the given questions. In this paper, we propose novel training schemes for multiple-choice video question answering with a self-supervised pre-training stage and a supervised contrastive learning in the main stage as an auxiliary learning. In the self-supervised pre-training stage, we transform the original problem format of predicting the correct answer into the one that predicts the relevant question to provide a model with broader contextual inputs without any further dataset or annotation. For contrastive learning in the main stage, we add a masking noise to the input corresponding to the ground-truth answer, and consider the original input of the ground-truth answer as a positive sample, while treating the rest as negative samples. By mapping the positive sample closer to the masked input, we show that the model performance is improved. We further employ locally aligned attention to focus more effectively on the video frames that are particularly relevant to the given corresponding subtitle sentences. We evaluate our proposed model on highly competitive benchmark datasets related to multiple-choice video QA: TVQA, TVQA+, and DramaQA. Experimental results show that our model achieves state-of-the-art performance on all datasets. We also validate our approaches through further analyses.
Embodied Visual Active Learning for Semantic Segmentation

We study the task of embodied visual active learning, where an agent is set to explore a 3d environment with the goal to acquire visual scene understanding by actively selecting views for which to request annotation. While accurate on some benchmarks, today's deep visual recognition pipelines tend to not generalize well in certain real-world scenarios, or for unusual viewpoints. Robotic perception, in turn, requires the capability to refine the recognition capabilities for the conditions where the mobile system operates, including cluttered indoor environments or poor illumination. This motivates the proposed task, where an agent is placed in a novel environment with the objective of improving its visual recognition capability. To study embodied visual active learning, we develop a battery of agents - both learnt and pre-specified - and with different levels of knowledge of the environment. The agents are equipped with a semantic segmentation network and seek to acquire informative views, move and explore in order to propagate annotations in the neighbourhood of those views, then refine the underlying segmentation network by online retraining. The trainable method uses deep reinforcement learning with a reward function that balances two competing objectives: task performance, represented as visual recognition accuracy, which requires exploring the environment, and the necessary amount of annotated data requested during active exploration. We extensively evaluate the proposed models using the photorealistic Matterport3D simulator and show that a fully learnt method outperforms comparable pre-specified counterparts, even when requesting fewer annotations.
Towards Argument Mining for Social Good: A Survey

Eva Maria Vecchi, Neele Falk, Iman Jundi, Gabriella Lapesa. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Argument Pair Extraction via Attention-guided Multi-Layer Multi-Cross Encoding

Argument pair extraction (APE) is a research task for extracting arguments from two passages and identifying potential argument pairs.Prior research work treats this task as a sequence labeling problem and a binary classification problem on two passages that are directly concatenated together, which has a limitation of not fully utilizing the unique characteristics and inherent relations of two different passages.This paper proposes a novel attention-guided multi-layer multi-cross encoding scheme to address the challenges.The new model processes two passages with two individual sequence encoders and updates their representations using each other's representations through attention.In addition, the pair prediction part is formulated as a tablefilling problem by updating the representations of two sequences' Cartesian product.Furthermore, an auxiliary attention loss is introduced to guide each argument to align to its paired argument.An extensive set of experiments show that the new model significantly improves the APE performance over several alternatives 1 .
A Novel Visual Interpretability for Deep Neural Networks by Optimizing Activation Maps with Perturbation

Interpretability has been regarded as an essential component for deploying deep neural networks, in which the saliency-based method is one of the most prevailing interpretable approaches since it can generate individually intuitive heatmaps that highlight parts of the input image that are most important to the decision of the deep networks on a particular classification target. However, heatmaps generated by existing methods either contain little information to represent objects (perturbation-based methods) or cannot effectively locate multi-class objects (activation-based approaches). To address this issue, a two-stage framework for visualizing the interpretability of deep neural networks, called Activation Optimized with Perturbation (AOP), is designed to optimize activation maps generated by general activation-based methods with the help of perturbation-based methods. Finally, in order to obtain better explanations for different types of images, we further present an instance of the AOP framework, Smooth Integrated Gradient-based Class Activation Map (SIGCAM), which proposes a weighted GradCAM by applying the feature map as weight coefficients and employs I-GOS to optimize the base-mask generated by weighted GradCAM. Experimental results on common-used benchmarks, including deletion and insertion tests on ImageNet-1k, and pointing game tests on COCO2017, show that the proposed AOP and SIGCAM outperform the current state-of-the-art methods significantly by generating higher quality image-based saliency maps.
Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation

Early fusion models with cross-attention have shown better-than-human performance on some question answer benchmarks, while it is a poor fit for retrieval since it prevents pre-computation of the answer representations.We present a supervised data mining method using an accurate early fusion model to improve the training of an efficient late fusion retrieval model.We first train an accurate classification model with cross-attention between questions and answers.The cross-attention model is then used to annotate additional passages in order to generate weighted training examples for a neural retrieval model.The resulting retrieval model with additional data significantly outperforms retrieval models directly trained with gold annotations on Precision at N (P@N) and Mean Reciprocal Rank (MRR).
Learning Proofs of Motion Planning Infeasibility

We present a learning-based approach to prove infeasibility of kinematic motion planning problems.Samplingbased motion planners are effective in high-dimensional spaces but are only probabilistically complete.Consequently, these planners cannot provide a definite answer if no plan exists, which is important for high-level scenarios, such as task-motion planning.We propose a combination of bidirectional samplingbased planning (such as RRT-connect) and machine learning to construct an infeasibility proof alongside the two search trees.An infeasibility proof is a closed manifold in the obstacle region of the configuration space that separates the start and goal into disconnected components of the free configuration space.We train the manifold using common machine learning techniques and then triangulate the manifold into a polytope to prove containment in the obstacle region.Under assumptions about learning hyper-parameters and robustness of configuration space optimization, the output is either an infeasibility proof or a motion plan.We demonstrate proof construction for 3-DOF and 4-DOF manipulators and show improvement over a previous algorithm.
Comparing Test Sets with Item Response Theory

Recent years have seen numerous NLP datasets introduced to evaluate the performance of fine-tuned models on natural language understanding tasks.Recent results from large pretrained models, though, show that many of these datasets are largely saturated and unlikely to be able to detect further progress.What kind of datasets are still effective at discriminating among strong models, and what kind of datasets should we expect to be able to detect future improvements?To measure this uniformly across datasets, we draw on Item Response Theory and evaluate 29 datasets using predictions from 18 pretrained Transformer models on individual test examples.We find that Quoref, Hel-laSwag, and MC-TACO are best suited for distinguishing among state-of-the-art models, while SNLI, MNLI, and CommitmentBank seem to be saturated for current strong models.We also observe span selection task format, which is used for QA datasets like QAMR or SQuAD2.0, is effective in differentiating between strong and weak models.
Evaluating Countermeasures for Verifying the Integrity of Ethereum Smart Contract Applications

Blockchain technology, which provides digital security in a distributed manner, has evolved into a key technology that can build efficient and reliable decentralized applications (called DApps) beyond the function of cryptocurrency. The characteristics of blockchain such as immutability and openness, however, have made DApps more vulnerable to various security risks, and thus it has become of great significance to validate the integrity of DApps before they actually operate upon blockchain. Recently, research on vulnerability in smart contracts (a building block of DApps) has been actively conducted, and various vulnerabilities and their countermeasures were reported. However, the effectiveness of such countermeasures has not been studied well, and no appropriate methods have been proposed to evaluate them. In this paper, we propose a software tool that can easily perform comparative studies by adding existing/new countermeasures and labeled smart contract codes. The proposed tool demonstrates verification performance using various statistical indicators, which helps to identify the most effective countermeasures for each type of vulnerability. Using the proposed tool, we evaluated state-of-the-art countermeasures with 237 labeled benchmark codes. The results indicate that for certain types of vulnerabilities, some countermeasures show evenly good performance scores on various metrics. However, it is also observed that countermeasures that detect the largest number of vulnerable codes typically generate much more false positives, resulting in very low precision and accuracy. Consequently, under given constraints, different countermeasures may be recommended for detecting vulnerabilities of interest. We believe that the proposed tool could effectively be utilized for a future verification study of smart contract applications and contribute to the development of practical and secure smart contract applications.
LexFit: Lexical Fine-Tuning of Pretrained Language Models

Transformer-based language models (LMs) pretrained on large text collections implicitly store a wealth of lexical semantic knowledge, but it is non-trivial to extract that knowledge effectively from their parameters.Inspired by prior work on semantic specialization of static word embedding (WE) models, we show that it is possible to expose and enrich lexical knowledge from the LMs, that is, to specialize them to serve as effective and universal "decontextualized" word encoders even when fed input words "in isolation" (i.e., without any context).Their transformation into such word encoders is achieved through a simple and efficient lexical fine-tuning procedure (termed LEXFIT) based on dual-encoder network structures.Further, we show that LEXFIT can yield effective word encoders even with limited lexical supervision and, via cross-lingual transfer, in different languages without any readily available external knowledge.Our evaluation over four established, structurally different lexical-level tasks in 8 languages indicates the superiority of LEXFIT-based WEs over standard static WEs (e.g., fastText) and WEs from vanilla LMs.Other extensive experiments and ablation studies further profile the LEXFIT framework, and indicate best practices and performance variations across LEXFIT variants, languages, and lexical tasks, also directly questioning the usefulness of traditional WE models in the era of large neural models.
Dodrio: Exploring Transformer Models with Interactive Visualization

Zijie J. Wang, Robert Turko, Duen Horng Chau. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations. 2021.
PROST: Physical Reasoning about Objects through Space and Time

We present a new probing dataset named PROST: Physical Reasoning about Objects Through Space and Time.This dataset contains 18,736 multiple-choice questions made from 14 manually curated templates, covering 10 physical reasoning concepts.All questions are designed to probe both causal and masked language models in a zero-shot setting.We conduct an extensive analysis which demonstrates that state-of-the-art pretrained models are inadequate at physical reasoning: they are influenced by the order in which answer options are presented to them, they struggle when the superlative in a question is inverted (e.g., most ↔ least), and increasing the amount of pretraining data and parameters only yields minimal improvements.These results provide support for the hypothesis that current pretrained models' ability to reason about physical interactions is inherently limited by a lack of real world experience.By highlighting these limitations, we hope to motivate the development of models with a human-like understanding of the physical world.
A Neural Transition-based Joint Model for Disease Named Entity Recognition and Normalization

Disease is one of the fundamental entities in biomedical research.Recognizing such entities from biomedical text and then normalizing them to a standardized disease vocabulary offer a tremendous opportunity for many downstream applications.Previous studies have demonstrated that joint modeling of the two sub-tasks has superior performance than the pipelined counterpart.Although the neural joint model based on multi-task learning framework has achieved state-of-the-art performance, it suffers from the boundary inconsistency problem due to the separate decoding procedures.Moreover, it ignores the rich information (e.g., the text surface form) of each candidate concept in the vocabulary, which is quite essential for entity normalization.In this work, we propose a neural transition-based joint model to alleviate these two issues.We transform the end-to-end disease recognition and normalization task as an action sequence prediction task, which not only jointly learns the model with shared representations of the input, but also jointly searches the output by state transitions in one search space.Moreover, we introduce attention mechanisms to take advantage of the text surface form of each candidate concept for better normalization performance.Experimental results conducted on two publicly available datasets show the effectiveness of the proposed method.
Best of Both Worlds: Making High Accuracy Non-incremental Transformer-based Disfluency Detection Incremental

While Transformer-based text classifiers pretrained on large volumes of text have yielded significant improvements on a wide range of computational linguistics tasks, their implementations have been unsuitable for live incremental processing thus far, operating only on the level of complete sentence inputs.We address the challenge of introducing methods for word-by-word left-to-right incremental processing to Transformers such as BERT, models without an intrinsic sense of linear order.We modify the training method and live decoding of non-incremental models to detect speech disfluencies with minimum latency and without pre-segmentation of dialogue acts.We experiment with several decoding methods to predict the rightward context of the word currently being processed using a GPT-2 language model and apply a BERT-based disfluency detector to sequences, including predicted words.We show our method of incrementalising Transformers maintains most of their high non-incremental performance while operating strictly incrementally.We also evaluate our models' incremental performance to establish the trade-off between incremental performance and final performance, using different prediction strategies.We apply our system to incremental speech recognition results as they arrive into a live system and achieve state-of-the-art results in this setting.
Event Extraction from Historical Texts: A New Dataset for Black Rebellions

Understanding historical events is necessary for the study of contemporary society, culture, and politics.In this work, we focus on the event extraction task (EE) to detect event trigger words and their arguments in a novel domain of historical texts.In particular, we introduce a new EE dataset for a corpus of nineteenth-century African American newspapers.Our goal is to study the discourse of slave and non-slave African diaspora rebellions published in the periodical press in this period.Our dataset features 5 entity types, 12 event types, and 6 argument roles that concern slavery and black movements between the eighteenth and nineteenth centuries.Historical newspapers present many challenges for existing EE systems, including the evolution of meanings of words and the extensive use of religious discourse in newspapers from this era.Our experiments with current state-ofthe-art EE systems and BERT models demonstrate their poor performance over historical texts and call for more robust research efforts in this area.
PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception

The ability to perceive and reason about social interactions in the context of physical environments is core to human social intelligence and human-machine cooperation. However, no prior dataset or benchmark has systematically evaluated physically grounded perception of complex social interactions that go beyond short actions, such as high-fiving, or simple group activities, such as gathering. In this work, we create a dataset of physically-grounded abstract social events, PHASE, that resemble a wide range of real-life social interactions by including social concepts such as helping another agent. PHASE consists of 2D animations of pairs of agents moving in a continuous space generated procedurally using a physics engine and a hierarchical planner. Agents have a limited field of view, and can interact with multiple objects, in an environment that has multiple landmarks and obstacles. Using PHASE, we design a social recognition task and a social prediction task. PHASE is validated with human experiments demonstrating that humans perceive rich interactions in the social events, and that the simulated agents behave similarly to humans. As a baseline model, we introduce a Bayesian inverse planning approach, SIMPLE (SIMulation, Planning and Local Estimation), which outperforms state-of-the-art feed-forward neural networks. We hope that PHASE can serve as a difficult new challenge for developing new models that can recognize complex social interactions.
A Closer Look at How Fine-tuning Changes BERT

Given the prevalence of pre-trained contextualized representations in today's NLP, there have been many efforts to understand what information they contain, and why they seem to be universally successful.The most common approach to use these representations involves fine-tuning them for an end task.Yet, how fine-tuning changes the underlying embedding space is less studied.In this work, we study the English BERT family and use two probing techniques to analyze how fine-tuning changes the space.We hypothesize that fine-tuning affects classification performance by increasing the distances between examples associated with different labels.We confirm this hypothesis with carefully designed experiments on five different NLP tasks.Via these experiments, we also discover an exception to the prevailing wisdom that "fine-tuning always improves performance".Finally, by comparing the representations before and after fine-tuning, we discover that fine-tuning does not introduce arbitrary changes to representations; instead, it adjusts the representations to downstream tasks while largely preserving the original spatial structure of the data points.