Reviewerly/exHarmony huggingface dataset:
Geometry-Aware Network for Unsupervised Learning of Monocular Camera’s Ego-Motion

Deep neural networks have been shown to be effective for unsupervised monocular visual odometry that can predict the camera’s ego-motion based on an input of monocular video sequence. However, most existing unsupervised monocular methods haven’t fully exploited the extracted information from both local geometric structure and visual appearance of the scenes, resulting in degraded performance. In this paper, a novel geometry-aware network is proposed to predict the camera’s ego-motion by learning representations in both 2D and 3D space. First, to extract geometry-aware features, we design an RGB-PointCloud feature fusion module to capture information from both geometric structure and the visual appearance of the scenes by fusing local geometric features from depth-map-derived point clouds and visual features from RGB images. Furthermore, the fusion module can adaptively allocate different weights to the two types of features to emphasize important regions. Then, we devise a relevant feature filtering module to build consistency between the two views and preserve informative features with high relevance. It can capture the correlation of frame pairs in the feature-embedding space by attention mechanisms. Finally, the obtained features are fed into the pose estimator to recover the 6-DoF poses of the camera. Extensive experiments show that our method achieves promising results among the unsupervised monocular deep learning methods on the KITTI odometry and TUM-RGBD datasets.
An Imperceptible Data Augmentation Based Blackbox Clean-Label Backdoor Attack on Deep Neural Networks

Deep neural networks (DNNs) have permeated into many diverse application domains, making them attractive targets of malicious attacks. DNNs are particularly susceptible to data poisoning attacks. Such attacks can be made more venomous and harder to detect by poisoning the training samples without changing their ground-truth labels. Despite its pragmatism, the clean-label requirement imposes a stiff restriction and strong conflict in simultaneous optimization of attack stealth, success rate, and utility of the poisoned model. Attempts to circumvent the pitfalls often lead to a high injection rate, ineffective embedded backdoors, unnatural triggers, low transferability, and/or poor robustness. In this paper, we overcome these constraints by amalgamating different data augmentation techniques for the backdoor trigger. The spatial intensities of the augmentation methods are iteratively adjusted by interpolating the clean sample and its augmented version according to their tolerance to perceptual loss and augmented feature saliency to target class activation. Our proposed attack is comprehensively evaluated on different network models and datasets. Compared with state-of-the-art clean-label backdoor attacks, it has lower injection rate, stealthier poisoned samples, higher attack success rate, and greater backdoor mitigation resistance while preserving high benign accuracy. Similar attack success rates are also demonstrated on the Intel Neural Compute Stick 2 edge AI device implementation of the poisoned model after weight-pruning and quantization.
Morphological Inflection: A Reality Check

Morphological inflection is a popular task in sub-word NLP with both practical and cognitive applications.For years now, state-of-theart systems have reported high, but also highly variable, performance across data sets and languages.We investigate the causes of this high performance and high variability; we find several aspects of data set creation and evaluation which systematically inflate performance and obfuscate differences between languages.To improve generalizability and reliability of results, we propose new data sampling and evaluation strategies that better reflect likely usecases.Using these new strategies, we make new observations on the generalization abilities of current inflection systems.
Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization

For text summarization, the role of discourse structure is pivotal in discerning the core content of a text.Regrettably, prior studies on incorporating Rhetorical Structure Theory (RST) into transformer-based summarization models only consider the nuclearity annotation, thereby overlooking the variety of discourse relation types.This paper introduces the 'RSTformer', a novel summarization model that comprehensively incorporates both the types and uncertainty of rhetorical relations.Our RST-attention mechanism, rooted in document-level rhetorical structure, is an extension of the recently devised Longformer framework.Through rigorous evaluation, the model proposed herein exhibits significant superiority over state-of-theart models, as evidenced by its notable performance on several automatic metrics and human evaluation. 1
Concept-Based Explanations to Test for False Causal Relationships Learned by Abusive Language Classifiers

Classifiers tend to learn a false causal relationship between an over-represented concept and a label, which can result in over-reliance on the concept and compromised classification accuracy.It is imperative to have methods in place that can compare different models and identify over-reliances on specific concepts.We consider three well-known abusive language classifiers trained on large English datasets and focus on the concept of negative emotions, which is an important signal but should not be learned as a sufficient feature for the label of abuse.Motivated by the definition of global sufficiency, we first examine the unwanted dependencies learned by the classifiers by assessing their accuracy on a challenge set across all decision thresholds.Further, recognizing that a challenge set might not always be available, we introduce concept-based explanation metrics to assess the influence of the concept on the labels.These explanations allow us to compare classifiers regarding the degree of false global sufficiency they have learned between a concept and a label.
Cross-Platform and Cross-Domain Abusive Language Detection with Supervised Contrastive Learning

The prevalence of abusive language on different online platforms has been a major concern that raises the need for automated crossplatform abusive language detection.However, prior works focus on concatenating data from multiple platforms, inherently adopting Empirical Risk Minimization (ERM) method.In this work, we address this challenge from the perspective of domain generalization objective.We design SCL-Fish, a supervised contrastive learning integrated meta-learning algorithm to detect abusive language on unseen platforms.Our experimental analysis shows that SCL-Fish achieves better performance over ERM and the existing state-of-the-art models.We also show that SCL-Fish is data-efficient and achieves comparable performance with the large-scale pre-trained models upon finetuning for the abusive language detection task. 1
Problematic Webpage Identification: A Trilogy of Hatespeech, Search Engines and GPT

In this paper, we introduce a fine-tuned transformer-based model focused on problematic webpage classification to identify webpages promoting hate and violence of various forms.Due to the unavailability of labelled problematic webpage data, first we propose a novel webpage data collection strategy which leverages well-studied short-text hate speech datasets.We have introduced a custom GPT-4 few-shot prompt annotation scheme taking various webpage features to label the prohibitively expensive webpage annotation task.The resulting annotated data is used to build our problematic webpage classification model.We report the accuracy (87.6% F1-score) of our webpage classification model and conduct a detailed comparison of it against other state-of-the-art hate speech classification model on problematic webpage identification task.Finally, we have showcased the importance of various webpage features in identifying a problematic webpage.
Exploring the Capacity of Pretrained Language Models for Reasoning about Actions and Change

Reasoning about actions and change (RAC) is essential to understand and interact with the ever-changing environment.Previous AI research has shown the importance of fundamental and indispensable knowledge of actions, i.e., preconditions and effects.However, traditional methods rely on logical formalization which hinders practical applications.With recent transformer-based language models (LMs), reasoning over text is desirable and seemingly feasible, leading to the question of whether LMs can effectively and efficiently learn to solve RAC problems.We propose four essential RAC tasks as a comprehensive textual benchmark and generate problems in a way that minimizes the influence of other linguistic requirements (e.g., grounding) to focus on RAC.The resulting benchmark, TRAC, encompassing problems of various complexities, facilitates a more granular evaluation of LMs, precisely targeting the structural generalization ability much needed for RAC.Experiments with three high-performing transformers indicate that additional efforts are needed to tackle challenges raised by TRAC.
“Female Astronaut: Because sandwiches won’t make themselves up there”: Towards Multimodal misogyny detection in memes

A rise in the circulation of memes has led to the spread of a new form of multimodal hateful content. Unfortunately, the degree of hate women receive on the internet is disproportionately skewed against them. This, combined with the fact that multimodal misogyny is more challenging to detect as opposed to traditional text-based misogyny, signifies that the task of identifying misogynistic memes online is one of utmost importance. To this end, the MAMI dataset was released, consisting of 12000 memes annotated for misogyny and four sub-classes of misogyny - shame, objectification, violence and stereotype. While this balanced dataset is widely cited, we find that the task itself remains largely unsolved. Thus, in our work, we investigate the performance of multiple models in an effort to analyse whether domain specific pretraining helps model performance. We also investigate why even state of the art models find this task so challenging, and whether domain-specific pretraining can help. Our results show that pretraining BERT on hateful memes and leveraging an attention based approach with ViT outperforms state of the art models by more than 10%. Further, we provide insight into why these models may be struggling with this task with an extensive qualitative analysis of random samples from the test set.
Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training

In this paper, we introduce Cross-View Language Modeling, a simple and effective pretraining framework that unifies cross-lingual and cross-modal pre-training with shared architectures and objectives.Our approach is motivated by a key observation that cross-lingual and cross-modal pre-training share the same goal of aligning two different views of the same object into a common semantic space.To this end, the cross-view language modeling framework considers both multi-modal data (i.e., image-caption pairs) and multi-lingual data (i.e., parallel sentence pairs) as two different views of the same object, and trains the model to align the two views by maximizing the mutual information between them with conditional masked language modeling and contrastive learning.We pre-train CCLM, a Crosslingual Cross-modal Language Model, with the cross-view language modeling framework.Empirical results on IGLUE, a multi-lingual multi-modal benchmark, and two multi-lingual image-text retrieval datasets show that while conceptually simpler, CCLM significantly outperforms the prior state-of-the-art with an average absolute improvement of over 10%.Moreover, CCLM is the first multi-lingual multimodal pre-trained model that surpasses the translate-test performance of representative English vision-language models by zero-shot cross-lingual transfer.
Connective Prediction for Implicit Discourse Relation Recognition via Knowledge Distillation

Implicit discourse relation recognition (IDRR) remains a challenging task in discourse analysis due to the absence of connectives.Most existing methods utilize one-hot labels as the sole optimization target, ignoring the internal association among connectives.Besides, these approaches spend lots of effort on template construction, negatively affecting the generalization capability.To address these problems, we propose a novel Connective Prediction via Knowledge Distillation (CP-KD) approach to instruct large-scale pre-trained language models (PLMs) mining the latent correlations between connectives and discourse relations, which is meaningful for IDRR.Experimental results on the PDTB 2.0/3.0 and CoNLL 2016 datasets show that our method significantly outperforms the state-of-the-art models on coarse-grained and fine-grained discourse relations.Moreover, our approach can be transferred to explicit discourse relation recognition (EDRR) and achieve acceptable performance.Our code is released in https://github.com/cubenlp/CP_KD-for-IDRR.
Hybrid Knowledge Transfer for Improved Cross-Lingual Event Detection via Hierarchical Sample Selection

In this paper, we address the Event Detection task under a zero-shot cross-lingual setting where a model is trained on a source language but evaluated on a distinct target language for which there is no labeled data available.Most recent efforts in this field follow a direct transfer approach in which the model is trained using language-invariant features and then directly applied to the target language.However, we argue that these methods fail to take advantage of the benefits of the data transfer approach where a cross-lingual model is trained on targetlanguage data and is able to learn task-specific information from syntactical features or wordlabel relations in the target language.As such, we propose a hybrid knowledge-transfer approach that leverages a teacher-student framework where the teacher and student networks are trained following the direct and data transfer approaches, respectively.Our method is complemented by a hierarchical training-sample selection scheme designed to address the issue of noisy labels being generated by the teacher model.Our model achieves state-of-the-art results on 9 morphologically-diverse target languages across 3 distinct datasets, highlighting the importance of exploiting the benefits of hybrid transfer.
Towards Safer Communities: Detecting Aggression and Offensive Language in Code-Mixed Tweets to Combat Cyberbullying

Cyberbullying is a serious societal issue widespread on various channels and platforms, particularly social networking sites.Such platforms have proven to be exceptionally fertile grounds for such behavior.The dearth of highquality training data for multilingual and lowresource scenarios, data that can accurately capture the nuances of social media conversations, often poses a roadblock to this task.This paper attempts to tackle cyberbullying, specifically its two most common manifestationsaggression and offensiveness.We present a novel, manually annotated dataset of a total of 10, 000 English and Hindi-English codemixed tweets, manually annotated for aggression detection and offensive language detection tasks 1 .Our annotations are supported by interannotator agreement scores of 0.67 and 0.74 for the two tasks, indicating substantial agreement.We perform comprehensive fine-tuning of pre-trained language models (PTLMs) using this dataset to check its efficacy.Our challenging test sets show that the best models achieve macro F1-scores of 67.87 and 65.45 on the two tasks, respectively.Further, we perform cross-dataset transfer learning to benchmark our dataset against existing aggression and offensive language datasets.We also present a detailed quantitative and qualitative analysis of errors in prediction, and with this paper, we publicly release the novel dataset, code, and models.On the other hand, offensiveness has been described as any word or string of words which has or can have a negative impact on the sense of self or well-being of those who encounter it (Molek-Kozakowska, 2022) -that is, it makes or can make them feel mildly or extremely discomfited, insulted, hurt or frightened.Motivation The dearth of manually-annotated datasets for the tasks of aggression detection and offensive language detection, especially in the Hindi-English code-mixed setting, necessitated us to work in this area.This paper investigates the tasks of aggression detection and offensive language detection on Twitter data.We curate politically-themed tweets and perform manual annotation to create a dataset for the tasks.Our annotation schema is in line with the existing aggressive and offensive language detection datasets.With the help of pre-trained language models, we fine-tune pre-trained language models for both tasks and discuss the obtained results regarding precision, recall, and macro F1-scores.
What is the best recipe for character-level encoder-only modelling?

This paper aims to benchmark recent progress in language understanding models that output contextualised representations at the character level.Many such modelling architectures and methods to train those architectures have been proposed, but it is currently unclear what the relative contributions of the architecture vs. the pretraining objective are to final model performance.We explore the design space of such models, comparing architectural innovations (Clark et al., 2022;Jaegle et al., 2022;Tay et al., 2021), and a variety of different pretraining objectives on a suite of evaluation tasks in order to find the optimal way to build and train character-level BERT-like models.We find that the best recipe combines the Charformer and CANINE model architectures, and follows the CANINE training procedure.This model exceeds the performance of a tokenbased model trained with the same settings on the same data, suggesting that character-level models are ready for more widespread adoption.Unfortunately, the best method to train character-level models still relies on a learnt tokeniser during pretraining, and final model performance is highly dependent on tokeniser quality.We believe our results demonstrate the readiness of character-level models for multilingual language representation, and encourage NLP practitioners to try them for their needs.
Aporophobia: An Overlooked Type of Toxic Language Targeting the Poor

Content Warning: This paper presents textual examples that may be offensive or upsetting.While many types of hate speech and online toxicity have been the focus of extensive research in NLP, toxic language stigmatizing poor people has been mostly disregarded.Yet, aporophobia, a social bias against the poor, is a common phenomenon online, which can be psychologically damaging as well as hindering poverty reduction policy measures.We demonstrate that aporophobic attitudes are indeed present in social media and argue that the existing NLP datasets and models are inadequate to effectively address this problem.Efforts toward designing specialized resources and novel socio-technical mechanisms for confronting aporophobia are needed.
BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Training

Automatic metrics play a crucial role in machine translation.Despite the widespread use of n-gram-based metrics, there has been a recent surge in the development of pre-trained model-based metrics that focus on measuring sentence semantics.However, these neural metrics, while achieving higher correlations with human evaluations, are often considered to be black boxes with potential biases that are difficult to detect.In this study, we systematically analyze and compare various mainstream and cutting-edge automatic metrics from the perspective of their guidance for training machine translation systems.Through Minimum Risk Training (MRT), we find that certain metrics exhibit robustness defects, such as the presence of universal adversarial translations in BLEURT and BARTScore.In-depth analysis suggests two main causes of these robustness deficits: distribution biases in the training datasets, and the tendency of the metric paradigm.By incorporating token-level constraints, we enhance the robustness of evaluation metrics, which in turn leads to an improvement in the performance of machine translation systems.
What is Wrong with Language Models that Can Not Tell a Story?

In this position paper, we contend that advancing our understanding of narrative and the effective generation of longer, subjectively engaging texts is crucial for progress in modern Natural Language Processing (NLP) and potentially the broader field of Artificial Intelligence.We highlight the current lack of appropriate datasets, evaluation methods, and operational concepts necessary for initiating work on narrative processing.
Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Review

In recent years, pre-trained Multilingual Language Models (MLLMs) have shown a strong ability to transfer knowledge across different languages.However, given that the aspiration for such an ability has not been explicitly incorporated in the design of the majority of MLLMs, it is challenging to obtain a unique and straightforward explanation for its emergence.In this review paper, we survey literature that investigates different factors contributing to the capacity of MLLMs to perform zero-shot cross-lingual transfer and subsequently outline and discuss these factors in detail.To enhance the structure of this review and to facilitate consolidation with future studies, we identify five categories of such factors.In addition to providing a summary of empirical evidence from past studies, we identify consensuses among studies with consistent findings and resolve conflicts among contradictory ones.Our work contextualizes and unifies existing research streams which aim at explaining the cross-lingual potential of MLLMs.This review provides, first, an aligned reference point for future research and, second, guidance for a better-informed and more efficient way of leveraging the crosslingual capacity of MLLMs.
An Ordinal Latent Variable Model of Conflict Intensity

Measuring the intensity of events is crucial for monitoring and tracking armed conflict.Advances in automated event extraction have yielded massive data sets of "who did what to whom" micro-records that enable datadriven approaches to monitoring conflict.The Goldstein scale is a widely-used expert-based measure that scores events on a conflictualcooperative scale.It is based only on the action category ("what") and disregards the subject ("who") and object ("to whom") of an event, as well as contextual information, like associated casualty count, that should contribute to the perception of an event's "intensity".To address these shortcomings, we take a latent variablebased approach to measuring conflict intensity.We introduce a probabilistic generative model that assumes each observed event is associated with a latent intensity class.A novel aspect of this model is that it imposes an ordering on the classes, such that higher-valued classes denote higher levels of intensity.The ordinal nature of the latent variable is induced from naturally ordered aspects of the data (e.g., casualty counts) where higher values naturally indicate higher intensity.We evaluate the proposed model both intrinsically and extrinsically, showing that it obtains good held-out predictive performance.https://github.com/niklasstoehr/ordinal-conflict-intensity