OpenRaiser/Intern-Atlas huggingface dataset:
Gender Bias in Remote Pair Programming among Software Engineering Students: The twincode Exploratory Study
Venue: arXiv cs.SE 2021
Year: 2021

Context. Pair programming (PP) has been found to increase student interest in Computer Science, particularly so for women, and would therefore appear to be a way to help remedy their under-representation, which could be partially motivated by gender stereotypes applied to software engineers, assuming that men perform better than their women peers. If this same bias is present in pair programming, it could work against the goal of improving gender balance. Objective. In a remote setting in which students cannot directly observe their peers, we aim to explore whether they behave differently when the perceived gender of their remote PP partners changes, searching for differences in (i) the perceived productivity compared to solo programming; (ii) the partner's perceived technical competency compared to their own; (iii) the partner's perceived skill level; (iv) the interaction behavior, such as the frequency of source code additions, deletions, etc.; and (v) the type and relative frequencies of dialog messages in a chat window. Method. Using the twincode platform, several behaviors are automatically measured during the remote PP process, together with two questionnaires and a semantic tagging of the pairs' chats. A series of experiments to identify the effect, if any, of possible gender bias shall be performed. The control group will have no information about their partner's gender, whereas the treatment group will receive such information but will be selectively deceived about their partner's gender. For each response variable we will (i) compare control and experimental groups for the score distance between two in-pair tasks; then, using the data from the experimental group only, we will (ii) compare scores using the partner's perceived gender as a within-subjects variable; and (iii) analyze the interaction between the partner's perceived gender and the subject's gender.
Multimodal datasets: misogyny, pornography, and malignant stereotypes
Venue: arXiv cs.CY 2021
Year: 2021

We have now entered the era of trillion parameter machine learning models trained on billion-sized datasets scraped from the internet. The rise of these gargantuan datasets has given rise to formidable bodies of critical work that has called for caution while generating these large datasets. These address concerns surrounding the dubious curation practices used to generate these datasets, the sordid quality of alt-text data available on the world wide web, the problematic content of the CommonCrawl dataset often used as a source for training large language models, and the entrenched biases in large-scale visio-linguistic models (such as OpenAI's CLIP model) trained on opaque datasets (WebImageText). In the backdrop of these specific calls of caution, we examine the recently released LAION-400M dataset, which is a CLIP-filtered dataset of Image-Alt-text pairs parsed from the Common-Crawl dataset. We found that the dataset contains, troublesome and explicit images and text pairs of rape, pornography, malign stereotypes, racist and ethnic slurs, and other extremely problematic content. We outline numerous implications, concerns and downstream harms regarding the current state of large scale datasets while raising open questions for various stakeholders including the AI community, regulators, policy makers and data subjects.
Mastering the Explicit Opinion-role Interaction: Syntax-aided Neural Transition System for Unified Opinion Role Labeling
Venue: arXiv cs.CL 2021
Year: 2021

Unified opinion role labeling (ORL) aims to detect all possible opinion structures of 'opinion-holder-target' in one shot, given a text. The existing transition-based unified method, unfortunately, is subject to longer opinion terms and fails to solve the term overlap issue. Current top performance has been achieved by employing the span-based graph model, which however still suffers from both high model complexity and insufficient interaction among opinions and roles. In this work, we investigate a novel solution by revisiting the transition architecture, and augmenting it with a pointer network (PointNet). The framework parses out all opinion structures in linear-time complexity, meanwhile breaks through the limitation of any length of terms with PointNet. To achieve the explicit opinion-role interactions, we further propose a unified dependency-opinion graph (UDOG), co-modeling the syntactic dependency structure and the partial opinion-role structure. We then devise a relation-centered graph aggregator (RCGA) to encode the multi-relational UDOG, where the resulting high-order representations are used to promote the predictions in the vanilla transition system. Our model achieves new state-of-the-art results on the MPQA benchmark. Analyses further demonstrate the superiority of our methods on both efficacy and efficiency.
Sound Event Detection Transformer: An Event-based End-to-End Model for Sound Event Detection
Venue: arXiv cs.SD 2021
Year: 2021

Sound event detection (SED) has gained increasing attention with its wide application in surveillance, video indexing, etc. Existing models in SED mainly generate frame-level prediction, converting it into a sequence multi-label classification problem. A critical issue with the frame-based model is that it pursues the best frame-level prediction rather than the best event-level prediction. Besides, it needs post-processing and cannot be trained in an end-to-end way. This paper firstly presents the one-dimensional Detection Transformer (1D-DETR), inspired by Detection Transformer for image object detection. Furthermore, given the characteristics of SED, the audio query branch and a one-to-many matching strategy for fine-tuning the model are added to 1D-DETR to form Sound Event Detection Transformer (SEDT). To our knowledge, SEDT is the first event-based and end-to-end SED model. Experiments are conducted on the URBAN-SED dataset and the DCASE2019 Task4 dataset, and both show that SEDT can achieve competitive performance.
Multi-Object Tracking with Deep Learning Ensemble for Unmanned Aerial System Applications
Venue: arXiv cs.CV 2021
Year: 2021

Multi-object tracking (MOT) is a crucial component of situational awareness in military defense applications. With the growing use of unmanned aerial systems (UASs), MOT methods for aerial surveillance is in high demand. Application of MOT in UAS presents specific challenges such as moving sensor, changing zoom levels, dynamic background, illumination changes, obscurations and small objects. In this work, we present a robust object tracking architecture aimed to accommodate for the noise in real-time situations. We propose a kinematic prediction model, called Deep Extended Kalman Filter (DeepEKF), in which a sequence-to-sequence architecture is used to predict entity trajectories in latent space. DeepEKF utilizes a learned image embedding along with an attention mechanism trained to weight the importance of areas in an image to predict future states. For the visual scoring, we experiment with different similarity measures to calculate distance based on entity appearances, including a convolutional neural network (CNN) encoder, pre-trained using Siamese networks. In initial evaluation experiments, we show that our method, combining scoring structure of the kinematic and visual models within a MHT framework, has improved performance especially in edge cases where entity motion is unpredictable, or the data presents frames with significant gaps.
Graph Coloring: Comparing Cluster Graphs to Factor Graphs
Venue: arXiv cs.LG 2021
Year: 2021

We present a means of formulating and solving graph coloring problems with probabilistic graphical models. In contrast to the prevalent literature that uses factor graphs for this purpose, we instead approach it from a cluster graph perspective. Since there seems to be a lack of algorithms to automatically construct valid cluster graphs, we provide such an algorithm (termed LTRIP). Our experiments indicate a significant advantage for preferring cluster graphs over factor graphs, both in terms of accuracy as well as computational efficiency.
Are Training Resources Insufficient? Predict First Then Explain!
Venue: arXiv cs.CL 2021
Year: 2021

Natural language free-text explanation generation is an efficient approach to train explainable language processing models for commonsense-knowledge-requiring tasks. The most predominant form of these models is the explain-then-predict (EtP) structure, which first generates explanations and uses them for making decisions. The performance of EtP models is highly dependent on that of the explainer by the nature of their structure. Therefore, large-sized explanation data are required to train a good explainer model. However, annotating explanations is expensive. Also, recent works reveal that free-text explanations might not convey sufficient information for decision making. These facts cast doubts on the effectiveness of EtP models. In this paper, we argue that the predict-then-explain (PtE) architecture is a more efficient approach in terms of the modelling perspective. Our main contribution is twofold. First, we show that the PtE structure is the most data-efficient approach when explanation data are lacking. Second, we reveal that the PtE structure is always more training-efficient than the EtP structure. We also provide experimental results that confirm the theoretical advantages.
Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images
Venue: arXiv cs.CV 2021
Year: 2021

Detecting changes on the ground in multitemporal Earth observation data is one of the key problems in remote sensing. In this paper, we introduce Sibling Regression for Optical Change detection (SiROC), an unsupervised method for change detection in optical satellite images with medium and high resolution. SiROC is a spatial context-based method that models a pixel as a linear combination of its distant neighbors. It uses this model to analyze differences in the pixel and its spatial context-based predictions in subsequent time periods for change detection. We combine this spatial context-based change detection with ensembling over mutually exclusive neighborhoods and transitioning from pixel to object-level changes with morphological operations. SiROC achieves competitive performance for change detection with medium-resolution Sentinel-2 and high-resolution Planetscope imagery on four datasets. Besides accurate predictions without the need for training, SiROC also provides a well-calibrated uncertainty of its predictions. This makes the method especially useful in conjunction with deep-learning based methods for applications such as pseudo-labeling.
Models for Narrative Information: A Study
Venue: arXiv cs.DL 2021
Year: 2021

The major objective of this work is to study and report the existing ontology-driven models for narrative information. The paper aims to analyze these models across various domains. The goal of this work is to bring the relevant literature, and ontology models under one umbrella, and perform a parametric comparative study. A systematic literature review methodology was adopted for an extensive literature selection. A random stratified sampling technique was used to select the models from the literature. The findings explicate a comparative view of the narrative models across domains. The differences and similarities of knowledge representation across domains, in case of narrative information models based on ontology was identified. There are significantly fewer studies that reviewed the ontology-based narrative models. This work goes a step further by evaluating the ontologies using the parameters from narrative components. This paper will explore the basic concepts and top-level concepts in the models. Besides, this study provides a comprehensive study of the narrative theories in the context of ongoing research. The findings of this work demonstrate the similarities and differences among the elements of the ontology across domains. It also identifies the state of the art literature for ontology-based narrative information.
The generalized Floquet-Bloch spectrum for periodic thermodiffusive layered materials
Venue: arXiv cs.CE 2021
Year: 2021

The dynamic behaviour of periodic thermodiffusive multi-layered media excited by harmonic oscillations is studied. In the framework of linear thermodiffusive elasticity, periodic laminates, whose elementary cell is composed by an arbitrary number of layers, are considered. The generalized Floquet-Bloch conditions are imposed, and the universal dispersion relation of the composite is obtained by means of an approach based on the formal solution for a single layer together with the transfer matrix method. The eigenvalue problem associated with the dispersion equation is solved by means of an analytical procedure based on the symplecticity properties of the transfer matrix to which corresponds a palindromic characteristic polynomial, and the frequency band structure associated to wave propagating inside the medium are finally derived. The proposed approach is tested through illustrative examples where thermodiffusive multilayered structures of interest for renewable energy devices fabrication are analyzed. The effects of thermodiffusion coupling on both the propagation and attenuation of Bloch waves in these systems are investigated in detail.
Label differential privacy via clustering
Venue: arXiv cs.LG 2021
Year: 2021

We present new mechanisms for \emph{label differential privacy}, a relaxation of differentially private machine learning that only protects the privacy of the labels in the training set. Our mechanisms cluster the examples in the training set using their (non-private) feature vectors, randomly re-sample each label from examples in the same cluster, and output a training set with noisy labels as well as a modified version of the true loss function. We prove that when the clusters are both large and high-quality, the model that minimizes the modified loss on the noisy training set converges to small excess risk at a rate that is comparable to the rate for non-private learning. We describe both a centralized mechanism in which the entire training set is stored by a trusted curator, and a distributed mechanism where each user stores a single labeled example and replaces her label with the label of a randomly selected user from the same cluster. We also describe a learning problem in which large clusters are necessary to achieve both strong privacy and either good precision or good recall. Our experiments show that randomizing the labels within each cluster significantly improves the privacy vs. accuracy trade-off compared to applying uniform randomized response to the labels, and also compared to learning a model via DP-SGD.
Analysis and Optimization of HARQ for URLLC
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we investigate the effectiveness of the hybrid automatic repeat request (HARQ) technique in providing high-reliability and low-latency in the finite blocklength (FBL) regime in a single user uplink scenario. We characterize the packet error rate (PER), throughput, and delay performance of chase combining HARQ (CC-HARQ) and incremental redundancy HARQ (IR-HARQ) in AWGN and Rayleigh fading channel with $m$ retransmissions. Furthermore, we consider a quasi-static fading channel model, which is more accurate than the over-simplified i.i.d. block fading or same channel assumption over consecutive packets. We use finite state Markov model under the FBL regime to model correlative fading. Numerical results present interesting insight into the reliability-latency trade-off of HARQ. Furthermore, we formulate an optimization problem to maximize the throughput performance of IR-HARQ by reducing excessive retransmission overhead for a target packet error performance under different SNRs, Doppler frequencies, and rate regimes.
$Δ$-UQ: Accurate Uncertainty Quantification via Anchor Marginalization
Venue: arXiv cs.LG 2021
Year: 2021

We present $Δ$-UQ -- a novel, general-purpose uncertainty estimator using the concept of anchoring in predictive models. Anchoring works by first transforming the input into a tuple consisting of an anchor point drawn from a prior distribution, and a combination of the input sample with the anchor using a pretext encoding scheme. This encoding is such that the original input can be perfectly recovered from the tuple -- regardless of the choice of the anchor. Therefore, any predictive model should be able to predict the target response from the tuple alone (since it implicitly represents the input). Moreover, by varying the anchors for a fixed sample, we can estimate uncertainty in the prediction even using only a single predictive model. We find this uncertainty is deeply connected to improper sampling of the input data, and inherent noise, enabling us to estimate the total uncertainty in any system. With extensive empirical studies on a variety of use-cases, we demonstrate that $Δ$-UQ outperforms several competitive baselines. Specifically, we study model fitting, sequential model optimization, model based inversion in the regression setting and out of distribution detection, & calibration under distribution shifts for classification.
An explorative study on how human-robot interaction is taken into account by robot developers in praxis
Venue: arXiv cs.RO 2021
Year: 2021

How is human-robot interaction considered within the development of new robotic systems by practitioners? This study sets out to inquire, whether the development teams of robotic products have been considering human factor methods in their design and implementation process. We were specifically interested in the non-verbal communication methods they were aiming to implement, and how they have approached the design process for these. Although valuable insights on tasks and communication needs during the different phases of robot operation could be gathered, the results of this study indicate, that the perspective of the human user or bystander is very often neglected and that knowledge on methods for engineering human-robot interaction is missing. The study was conducted with eleven development teams consisting of robot manufacturers and students within a robot building course representing overall 68 individual participants.
Evolutionary Algorithms for Solving Unconstrained, Constrained and Multi-objective Noisy Combinatorial Optimisation Problems
Venue: arXiv cs.NE 2021
Year: 2021

We present an empirical study of a range of evolutionary algorithms applied to various noisy combinatorial optimisation problems. There are three sets of experiments. The first looks at several toy problems, such as OneMax and other linear problems. We find that UMDA and the Paired-Crossover Evolutionary Algorithm (PCEA) are the only ones able to cope robustly with noise, within a reasonable fixed time budget. In the second stage, UMDA and PCEA are then tested on more complex noisy problems: SubsetSum, Knapsack and SetCover. Both perform well under increasing levels of noise, with UMDA being the better of the two. In the third stage, we consider two noisy multi-objective problems (CountingOnesCountingZeros and a multi-objective formulation of SetCover). We compare several adaptations of UMDA for multi-objective problems with the Simple Evolutionary Multi-objective Optimiser (SEMO) and NSGA-II. We conclude that UMDA, and its variants, can be highly effective on a variety of noisy combinatorial optimisation, outperforming many other evolutionary algorithms.
A Formal Characterization of Black-Box System Safety Performance with Scenario Sampling
Venue: arXiv cs.RO 2021
Year: 2021

A typical scenario-based evaluation framework seeks to characterize a black-box system's safety performance (e.g., failure rate) through repeatedly sampling initialization configurations (scenario sampling) and executing a certain test policy for scenario propagation (scenario testing) with the black-box system involved as the test subject. In this letter, we first present a novel safety evaluation criterion that seeks to characterize the actual operational domain within which the test subject would remain safe indefinitely with high probability. By formulating the black-box testing scenario as a dynamic system, we show that the presented problem is equivalent to finding a certain "almost" robustly forward invariant set for the given system. Second, for an arbitrary scenario testing strategy, we propose a scenario sampling algorithm that is provably asymptotically optimal in obtaining the safe invariant set with arbitrarily high accuracy. Moreover, as one considers different testing strategies (e.g., biased sampling of safety-critical cases), we show that the proposed algorithm still converges to the unbiased approximation of the safety characterization outcome if the scenario testing satisfies a certain condition. Finally, the effectiveness of the presented scenario sampling algorithms and various theoretical properties are demonstrated in a case study of the safety evaluation of a control barrier function-based mobile robot collision avoidance system.
Adversarial defenses via a mixture of generators
Venue: arXiv cs.LG 2021
Year: 2021

In spite of the enormous success of neural networks, adversarial examples remain a relatively weakly understood feature of deep learning systems. There is a considerable effort in both building more powerful adversarial attacks and designing methods to counter the effects of adversarial examples. We propose a method to transform the adversarial input data through a mixture of generators in order to recover the correct class obfuscated by the adversarial attack. A canonical set of images is used to generate adversarial examples through potentially multiple attacks. Such transformed images are processed by a set of generators, which are trained adversarially as a whole to compete in inverting the initial transformations. To our knowledge, this is the first use of a mixture-based adversarially trained system as a defense mechanism. We show that it is possible to train such a system without supervision, simultaneously on multiple adversarial attacks. Our system is able to recover class information for previously-unseen examples with neither attack nor data labels on the MNIST dataset. The results demonstrate that this multi-attack approach is competitive with adversarial defenses tested in single-attack settings.
Language Modeling using LMUs: 10x Better Data Efficiency or Improved Scaling Compared to Transformers
Venue: arXiv cs.LG 2021
Year: 2021

Recent studies have demonstrated that the performance of transformers on the task of language modeling obeys a power-law relationship with model size over six orders of magnitude. While transformers exhibit impressive scaling, their performance hinges on processing large amounts of data, and their computational and memory requirements grow quadratically with sequence length. Motivated by these considerations, we construct a Legendre Memory Unit based model that introduces a general prior for sequence processing and exhibits an $O(n)$ and $O(n \ln n)$ (or better) dependency for memory and computation respectively. Over three orders of magnitude, we show that our new architecture attains the same accuracy as transformers with 10x fewer tokens. We also show that for the same amount of training our model improves the loss over transformers about as much as transformers improve over LSTMs. Additionally, we demonstrate that adding global self-attention complements our architecture and the augmented model improves performance even further.