Reviewerly/exHarmony huggingface dataset:
Language Model Augmented Monotonic Attention for Simultaneous Translation

The state-of-the-art adaptive policies for simultaneous neural machine translation (SNMT) use monotonic attention to perform read/write decisions based on the partial source and target sequences.The lack of sufficient information might cause the monotonic attention to take poor read/write decisions, which in turn negatively affects the performance of the SNMT model.On the other hand, human translators make better read/write decisions since they can anticipate the immediate future words using linguistic information and domain knowledge.In this work, we propose a framework to aid monotonic attention with an external language model to improve its decisions.Experiments on MuST-C English-German and English-French speech-to-text translation tasks show the future information from language model improves the state-of-the-art monotonic multi-head attention model further.
Codec at SemEval-2022 Task 5: Multi-Modal Multi-Transformer Misogynous Meme Classification Framework

In this paper we describe our work towards building a generic framework for both multimodal embedding and multi-label binary classification tasks, while participating in task 5 (Multimedia Automatic Misogyny Identification) of SemEval 2022 competition.Since pretraining deep models from scratch is a resource and data hungry task, our approach is based on three main strategies.We combine different state-of-the-art architectures to capture a wide spectrum of semantic signals from the multi-modal input.We employ a multitask learning scheme to be able to use multiple datasets from the same knowledge domain to help increase the model's performance.We also use multiple objectives to regularize and fine tune different system components.
On the Dynamics of Gender Learning in Speech Translation

Due to the complexity of bias and the opaque nature of current neural approaches, there is a rising interest in auditing language technologies.In this work, we contribute to such a line of inquiry by exploring the emergence of gender bias in Speech Translation (ST).As a new perspective, rather than focusing on the final systems only, we examine their evolution over the course of training.In this way, we are able to account for different variables related to the learning dynamics of gender translation, and investigate when and how gender divides emerge in ST.Accordingly, for three language pairs (en -> es, fr, it) we compare how ST systems behave for masculine and feminine translation at several levels of granularity.We find that masculine and feminine curves are dissimilar, with the feminine one being characterized by a more erratic behaviour and late improvements over the course of training.Also, depending on the considered phenomena, their learning trends can be either antiphase or parallel.Overall, we show how such a progressive analysis can inform on the reliability and time-wise acquisition of gender, which is concealed by static evaluations and standard metrics.
Pragmatic and Logical Inferences in NLI Systems: The Case of Conjunction Buttressing

An intelligent system is expected to perform reasonable inferences, accounting for both the literal meaning of a word and the meanings a word can acquire in different contexts.A specific kind of inference concerns the connective and, which in some cases gives rise to a temporal succession or causal interpretation in contrast with the logic, commutative one (Levinson, 2000).In this work, we investigate the phenomenon by creating a new dataset for evaluating the interpretation of and by NLI systems, which we use to test three Transformer-based models.Our results show that all systems generalize patterns that are consistent with both the logical and the pragmatic interpretation, perform inferences that are inconsistent with each other, and show clear divergences with both theoretical accounts and humans' behavior.
Generating unlabelled data for a tri-training approach in a low resourced NER task

Training a tagger for Named Entity Recognition (NER) requires a substantial amount of labeled data in the task domain.Manual labeling is a tedious and complicated task.Semisupervised learning methods can reduce the quantity of labeled data necessary to train a model.However, these methods require large quantities of unlabeled data, which remains an issue in many cases.We address this problem by generating unlabeled data.Large language models have proven to be powerful tools for text generation.We use their generative capacity to produce new sentences and variations of the sentences of our available data.This generation method, combined with a semi-supervised method, is evaluated on CoNLL and I2B2.We prepare both of these corpora to simulate a low resource setting.We obtain significant improvements for semisupervised learning with synthetic data against supervised learning on natural data.
WueDevils at SemEval-2022 Task 8: Multilingual News Article Similarity via Pair-Wise Sentence Similarity Matrices

We present a system that creates pair-wise cosine and arccosine sentence similarity matrices using multilingual sentence embeddings obtained from pre-trained SBERT and Universal Sentence Encoder models respectively.For each news article sentence, it searches the most similar sentence from the other article and computes an average score.Further, a convolutional neural network calculates a total similarity score for the article pairs on these matrices.Finally, a random forest regressor merges the previous results to a final score that can optionally be extended with a publishing date score.
UM6P-CS at SemEval-2022 Task 11: Enhancing Multilingual and Code-Mixed Complex Named Entity Recognition via Pseudo Labels using Multilingual Transformer

Building real-world complex Named Entity Recognition (NER) systems is a challenging task.This is due to the complexity and ambiguity of named entities that appear in various contexts such as short input sentences, emerging entities, and complex entities.Besides, real-world queries are mostly malformed, as they can be code-mixed or multilingual, among other scenarios.In this paper, we introduce our submitted system to the Multilingual Complex Named Entity Recognition (MultiCoNER) shared task.We approach the complex NER for multilingual and code-mixed queries, by relying on the contextualized representation provided by the multilingual Transformer XLM-RoBERTa.In addition to the CRF-based token classification layer, we incorporate a span classification loss to recognize named entities spans.Furthermore, we use a self-training mechanism to generate weakly-annotated data from a large unlabeled dataset.Our proposed system is ranked 6th and 8th in the multilingual and code-mixed MultiCoNER's tracks respectively.
Don’t sweat the small stuff, classify the rest: Sample Shielding to protect text classifiers against adversarial attacks

Deep learning (DL) is being used extensively for text classification.However, researchers have demonstrated the vulnerability of such classifiers to adversarial attacks.Attackers modify the text in a way which misleads the classifier while keeping the original meaning close to intact.State-of-the-art (SOTA) attack algorithms follow the general principle of making minimal changes to the text so as to not jeopardize semantics.Taking advantage of this we propose a novel and intuitive defense strategy called Sample Shielding.It is attacker and classifier agnostic, does not require any reconfiguration of the classifier or external resources and is simple to implement.Essentially, we sample subsets of the input text, classify them and summarize these into a final decision.We shield three popular DL text classifiers with Sample Shielding, test their resilience against four SOTA attackers across three datasets in a realistic threat setting.Even when given the advantage of knowing about our shielding strategy the adversary's attack success rate is <= 10% with only one exception and often < 5%.Additionally, Sample Shielding maintains near original accuracy when applied to original texts.Crucially, we show that the 'make minimal changes' approach of SOTA attackers leads to critical vulnerabilities that can be defended against with an intuitive sampling strategy. 1
ITNLP2022 at SemEval-2022 Task 8: Pre-trained Model with Data Augmentation and Voting for Multilingual News Similarity

This article introduces a system to solve the SemEval 2022 Task 8: Multilingual News Article Similarity.The task focuses on the consistency of events reported in two news articles.The system consists of a pre-trained model(e.g., INFOXLM and XLM-RoBERTa) to extract multilingual news features, following fully-connected networks to measure the similarity.In addition, data augmentation and Ten Folds Voting are used to enhance the model.Our final submitted model is an ensemble of three base models, with a Pearson value of 0.784 on the test dataset.
FII UAIC at SemEval-2022 Task 6: iSarcasmEval - Intended Sarcasm Detection in English and Arabic

The "iSarcasmEval -Intended Sarcasm Detection in English and Arabic" task at the Se-mEval 2022 competition focuses on detecting and rating the distinction between intended and perceived sarcasm in the context of textual sarcasm detection, as well as the level of irony contained in these texts.In the context of SemEval, we present a binary classification method which classifies the text as sarcastic or non-sarcastic (task A, for English) based on five classical machine learning approaches by trying to train the models based on this dataset solely (i.e., no other datasets have been used).This process indicates low performance compared to previously studied datasets, which indicates that the previous ones might be biased.
I2C at SemEval-2022 Task 6: Intended Sarcasm in English using Deep Learning Techniques

Sarcasm is often expressed through several verbal and non-verbal cues, e.g., a change of tone, overemphasis in a word, a drawnout syllable, or a straight looking face.Most of the recent work in sarcasm detection has been carried out on textual data.This paper describes how the problem proposed in Task 6: Intended Sarcasm Detection in English (Abu Arfa et al. 2022) has been solved.Specifically, we participated in Subtask B: a binary multi-label classification task, where it is necessary to determine whether a tweet belongs to an ironic speech category, if any.Several approaches (classic machine learning and deep learning algorithms) were developed.The final submission consisted of a BERT based model and a macro-F1 score of 0.0699 was obtained.
Masking Morphosyntactic Categories to Evaluate Salience for Schizophrenia Diagnosis

Natural language processing tools have been shown to be effective for detecting symptoms of schizophrenia in transcribed speech.We analyze and assess the contribution of the various syntactic and morphological categories towards successful machine classification of texts produced by subjects with schizophrenia and by others.Specifically, we fine-tune a language model for the classification task, and mask all words that are attributed with each category of interest.The speech samples were generated in a controlled way by interviewing inpatients who were officially diagnosed with schizophrenia, and a corresponding group of healthy controls.All participants are native Hebrew speakers.Our results show that nouns are the most significant category for classification performance.
Poirot at SemEval-2022 Task 5: Leveraging Graph Network for Misogynistic Meme Detection

In recent years, there has been an upsurge in a new form of entertainment medium called memes.These memes although seemingly innocuous have transcended the boundary of online harassment against women and created an unwanted bias against them.To help alleviate this problem, we propose an early fusion model for the prediction and identification of misogynistic memes and their type in this paper for which we participated in SemEval-2022 Task 5.The model receives as input meme image with its text transcription with a target vector.Given that a key challenge with this task is the combination of different modalities to predict misogyny, our model relies on pre-trained contextual representations from different stateof-the-art transformer-based language models and pre-trained image pre-trained models to get an effective image representation.Our model achieved competitive results on both SubTask-A and SubTask-B with the other competing teams and significantly outperforms the baselines.
MMG at SemEval-2022 Task 1: A Reverse Dictionary approach based on a review of the dataset from a lexicographic perspective

This paper presents a novel and linguistic-driven system for the Spanish Reverse Dictionary task of SemEval-2022 Task 1.The aim of this task is the automatic generation of a word using its gloss.The conclusion is that this task results could improve if the quality of the dataset did as well by incorporating high-quality lexicographic data.Therefore, in this paper we analyze the main gaps in the proposed dataset and describe how these limitations could be tackled.
A Predictive Model for the Detection of Clients Suspicious Behavior

The purpose of this work is to identify the suspicious behavior of the clients of a financial institution. Financial institutions use rule-based systems to detect unusual transactions. These systems focus on individual transactions or simple transaction patterns. Due to this, the need arises to detect suspicious behavior using machine learning since many of the machine learning algorithms are designed to capture complex patterns. Descriptive analysis and predictive analysis are used to detect suspicious behavior. Within the descriptive analysis, outliers are sought in transactional movements. For the predictive analysis, we start from a set of alerts with the label of whether or not they were reported by the Compliance Unit. As a result of the descriptive model, a set of customers that have behaved in an unusual way is obtained and as a result of the predictive model, the alerts that should be reported are predicted. It is concluded that the techniques: descriptive analysis, descriptive analysis and the rule-based system can complement each other, since they focus on different aspects of the identification of unusual transactions and should not be considered as exclusive alternatives.
Massively Multilingual Document Alignment with Cross-lingual Sentence-Mover's Distance

Document alignment aims to identify pairs of documents in two distinct languages that are of comparable content or translations of each other. Such aligned data can be used for a variety of NLP tasks from training cross-lingual representations to mining parallel data for machine translation. In this paper we develop an unsupervised scoring function that leverages cross-lingual sentence embeddings to compute the semantic distance between documents in different languages. These semantic distances are then used to guide a document alignment algorithm to properly pair cross-lingual web documents across a variety of low, mid, and high-resource language pairs. Recognizing that our proposed scoring function and other state of the art methods are computationally intractable for long web documents, we utilize a more tractable greedy algorithm that performs comparably. We experimentally demonstrate that our distance metric performs better alignment than current baselines outperforming them by 7% on high-resource language pairs, 15% on mid-resource language pairs, and 22% on low-resource language pairs.
Causal Structure Discovery from Distributions Arising from Mixtures of DAGs

We consider distributions arising from a mixture of causal models, where each model is represented by a directed acyclic graph (DAG). We provide a graphical representation of such mixture distributions and prove that this representation encodes the conditional independence relations of the mixture distribution. We then consider the problem of structure learning based on samples from such distributions. Since the mixing variable is latent, we consider causal structure discovery algorithms such as FCI that can deal with latent variables. We show that such algorithms recover a "union" of the component DAGs and can identify variables whose conditional distribution across the component DAGs vary. We demonstrate our results on synthetic and real data showing that the inferred graph identifies nodes that vary between the different mixture components. As an immediate application, we demonstrate how retrieval of this causal information can be used to cluster samples according to each mixture component.
Unsupervised Gaze Prediction in Egocentric Videos by Energy-based Surprise Modeling

Egocentric perception has grown rapidly with the advent of immersive computing devices. Human gaze prediction is an important problem in analyzing egocentric videos and has primarily been tackled through either saliency-based modeling or highly supervised learning. We quantitatively analyze the generalization capabilities of supervised, deep learning models on the egocentric gaze prediction task on unseen, out-of-domain data. We find that their performance is highly dependent on the training data and is restricted to the domains specified in the training annotations. In this work, we tackle the problem of jointly predicting human gaze points and temporal segmentation of egocentric videos without using any training data. We introduce an unsupervised computational model that draws inspiration from cognitive psychology models of event perception. We use Grenander's pattern theory formalism to represent spatial-temporal features and model surprise as a mechanism to predict gaze fixation points. Extensive evaluation on two publicly available datasets - GTEA and GTEA+ datasets-shows that the proposed model can significantly outperform all unsupervised baselines and some supervised gaze prediction baselines. Finally, we show that the model can also temporally segment egocentric videos with a performance comparable to more complex, fully supervised deep learning baselines.
NCVis: Noise Contrastive Approach for Scalable Visualization

Modern methods for data visualization via dimensionality reduction, such as t-SNE, usually have performance issues that prohibit their application to large amounts of high-dimensional data. In this work, we propose NCVis -- a high-performance dimensionality reduction method built on a sound statistical basis of noise contrastive estimation. We show that NCVis outperforms state-of-the-art techniques in terms of speed while preserving the representation quality of other methods. In particular, the proposed approach successfully proceeds a large dataset of more than 1 million news headlines in several minutes and presents the underlying structure in a human-readable way. Moreover, it provides results consistent with classical methods like t-SNE on more straightforward datasets like images of hand-written digits. We believe that the broader usage of such software can significantly simplify the large-scale data analysis and lower the entry barrier to this area.
A Survey on String Constraint Solving

String constraint solving refers to solving combinatorial problems involving constraints over string variables. String solving approaches have become popular over the last years given the massive use of strings in different application domains like formal analysis, automated testing, database query processing, and cybersecurity. This paper reports a comprehensive survey on string constraint solving by exploring the large number of approaches that have been proposed over the last decades to solve string constraints.
Optimally selecting the top $k$ values from $X+Y$ with layer-ordered heaps

Selection and sorting the Cartesian sum, $X+Y$, are classic and important problems. Here, a new algorithm is presented, which generates the top $k$ values of the form $X_i+Y_j$. The algorithm relies only on median-of-medians and is simple to implement. Furthermore, it uses data structures contiguous in memory, and is fast in practice. The presented algorithm is demonstrated to be theoretically optimal.
Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages

Most combinations of NLP tasks and language varieties lack in-domain examples for supervised training because of the paucity of annotated data. How can neural models make sample-efficient generalizations from task-language combinations with available data to low-resource ones? In this work, we propose a Bayesian generative model for the space of neural parameters. We assume that this space can be factorized into latent variables for each language and each task. We infer the posteriors over such latent variables based on data from seen task-language combinations through variational inference. This enables zero-shot classification on unseen combinations at prediction time. For instance, given training data for named entity recognition (NER) in Vietnamese and for part-of-speech (POS) tagging in Wolof, our model can perform accurate predictions for NER in Wolof. In particular, we experiment with a typologically diverse sample of 33 languages from 4 continents and 11 families, and show that our model yields comparable or better results than state-of-the-art, zero-shot cross-lingual transfer methods. Moreover, we demonstrate that approximate Bayesian model averaging results in smoother predictive distributions, whose entropy inversely correlates with accuracy. Hence, the proposed framework also offers robust estimates of prediction uncertainty. Our code is located at github.com/cambridgeltl/parameter-factorization
Provably Secure Isolation for Interruptible Enclaved Execution on Small Microprocessors: Extended Version

Computer systems often provide hardware support for isolation mechanisms like privilege levels, virtual memory, or enclaved execution. Over the past years, several successful software-based side-channel attacks have been developed that break, or at least significantly weaken the isolation that these mechanisms offer. Extending a processor with new architectural or micro-architectural features, brings a risk of introducing new such side-channel attacks. This paper studies the problem of extending a processor with new features without weakening the security of the isolation mechanisms that the processor offers. We propose to use full abstraction as a formal criterion for the security of a processor extension, and we instantiate that criterion to the concrete case of extending a microprocessor that supports enclaved execution with secure interruptibility of these enclaves. This is a very relevant instantiation as several recent papers have shown that interruptibility of enclaves leads to a variety of software-based side-channel attacks. We propose a design for interruptible enclaves, and prove that it satisfies our security criterion. We also implement the design on an open-source enclave-enabled microprocessor, and evaluate the cost of our design in terms of performance and hardware size.