OpenRaiser/Intern-Atlas huggingface dataset:
Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models
Venue: arXiv cs.CL 2021
Year: 2021

This paper proposes two intuitive metrics, skew and stereotype, that quantify and analyse the gender bias present in contextual language models when tackling the WinoBias pronoun resolution task. We find evidence that gender stereotype correlates approximately negatively with gender skew in out-of-the-box models, suggesting that there is a trade-off between these two forms of bias. We investigate two methods to mitigate bias. The first approach is an online method which is effective at removing skew at the expense of stereotype. The second, inspired by previous work on ELMo, involves the fine-tuning of BERT using an augmented gender-balanced dataset. We show that this reduces both skew and stereotype relative to its unaugmented fine-tuned counterpart. However, we find that existing gender bias benchmarks do not fully probe professional bias as pronoun resolution may be obfuscated by cross-correlations from other manifestations of gender prejudice. Our code is available online, at https://github.com/12kleingordon34/NLP_masters_project.
Longest segment of balanced parentheses -- an exercise in program inversion in a segment problem (Functional Pearl)
Venue: arXiv cs.PL 2021
Year: 2021

Given a string of parentheses, the task is to find the longest consecutive segment that is balanced, in linear time. We find this problem interesting because it involves a combination of techniques: the usual approach for solving segment problems, and a theorem for constructing the inverse of a function -- through which we derive an instance of shift-reduce parsing.
Notes on Monotone Recognition in Multi-Valued Grids
Venue: arXiv cs.DM 2021
Year: 2021

Implementation details of method of monotone recognition, based on partitioning of the grid into the discrete structures isomorphic to binary cubes is presented.
RomeBERT: Robust Training of Multi-Exit BERT
Venue: arXiv cs.CL 2021
Year: 2021

BERT has achieved superior performances on Natural Language Understanding (NLU) tasks. However, BERT possesses a large number of parameters and demands certain resources to deploy. For acceleration, Dynamic Early Exiting for BERT (DeeBERT) has been proposed recently, which incorporates multiple exits and adopts a dynamic early-exit mechanism to ensure efficient inference. While obtaining an efficiency-performance tradeoff, the performances of early exits in multi-exit BERT are significantly worse than late exits. In this paper, we leverage gradient regularized self-distillation for RObust training of Multi-Exit BERT (RomeBERT), which can effectively solve the performance imbalance problem between early and late exits. Moreover, the proposed RomeBERT adopts a one-stage joint training strategy for multi-exits and the BERT backbone while DeeBERT needs two stages that require more training time. Extensive experiments on GLUE datasets are performed to demonstrate the superiority of our approach. Our code is available at https://github.com/romebert/RomeBERT.
Termination Analysis Without the Tears
Venue: arXiv cs.PL 2021
Year: 2021

Determining whether a given program terminates is the quintessential undecidable problem. Algorithms for termination analysis are divided into two groups: (1) algorithms with strong behavioral guarantees that work in limited circumstances (e.g., complete synthesis of linear ranking functions for polyhedral loops [Podelski and Rybalchenko, 2004]), and (2) algorithms that are widely applicable, but have weak behavioral guarantees (e.g., Terminator [Cook et al., 2006]). This paper investigates the space in between: how can we design practical termination analyzers with useful behavioral guarantees? This paper presents a termination analysis that is both compositional (the result of analyzing a composite program is a function of the analysis results of its components) and monotone ("more information into the analysis yields more information out"). The paper has two key contributions. The first is an extension of Tarjan's method for solving path problems in graphs to solve infinite path problems. This provides a foundation upon which to build compositional termination analyses. The second is a collection of monotone conditional termination analyses based on this framework. We demonstrate that our tool ComPACT (Compositional and Predictable Analysis for Conditional Termination) is competitive with state-of-the-art termination tools while providing stronger behavioral guarantees.
Analytical Characterization and Design Space Exploration for Optimization of CNNs
Venue: arXiv cs.LG 2021
Year: 2021

Moving data through the memory hierarchy is a fundamental bottleneck that can limit the performance of core algorithms of machine learning, such as convolutional neural networks (CNNs). Loop-level optimization, including loop tiling and loop permutation, are fundamental transformations to reduce data movement. However, the search space for finding the best loop-level optimization configuration is explosively large. This paper develops an analytical modeling approach for finding the best loop-level optimization configuration for CNNs on multi-core CPUs. Experimental evaluation shows that this approach achieves comparable or better performance than state-of-the-art libraries and auto-tuning based optimizers for CNNs.
Black Feminist Musings on Algorithmic Oppression
Venue: arXiv cs.CY 2021
Year: 2021

This paper unapologetically reflects on the critical role that Black feminism can and should play in abolishing algorithmic oppression. Positioning algorithmic oppression in the broader field of feminist science and technology studies, I draw upon feminist philosophical critiques of science and technology and discuss histories and continuities of scientific oppression against historically marginalized people. Moreover, I examine the concepts of invisibility and hypervisibility in oppressive technologies a lá the canonical double bind. Furthermore, I discuss what it means to call for diversity as a solution to algorithmic violence, and I critique dialectics of the fairness, accountability, and transparency community. I end by inviting you to envision and imagine the struggle to abolish algorithmic oppression by abolishing oppressive systems and shifting algorithmic development practices, including engaging our communities in scientific processes, centering marginalized communities in design, and consensual data and algorithmic practices.
On the binary adder channel with complete feedback, with an application to quantitative group testing
Venue: arXiv cs.IT 2021
Year: 2021

We determine the exact value of the optimal symmetric rate point $(r, r)$ in the Dueck zero-error capacity region of the binary adder channel with complete feedback. We proved that the average zero-error capacity $r = h(1/2-δ) \approx 0.78974$, where $h(\cdot)$ is the binary entropy function and $δ= 1/(2\log_2(2+\sqrt3))$. Our motivation is a problem in quantitative group testing. Given a set of $n$ elements two of which are defective, the quantitative group testing problem asks for the identification of these two defectives through a series of tests. Each test gives the number of defectives contained in the tested subset, and the outcomes of previous tests are assumed known at the time of designing the current test. We establish that the minimum number of tests is asymptotic to $(\log_2 n) / r$ as $n \to \infty$.
Parametric Rectified Power Sigmoid Units: Learning Nonlinear Neural Transfer Analytical Forms
Venue: arXiv cs.IR 2021
Year: 2021

The paper proposes representation functionals in a dual paradigm where learning jointly concerns both linear convolutional weights and parametric forms of nonlinear activation functions. The nonlinear forms proposed for performing the functional representation are associated with a new class of parametric neural transfer functions called rectified power sigmoid units. This class is constructed to integrate both advantages of sigmoid and rectified linear unit functions, in addition with rejecting the drawbacks of these functions. Moreover, the analytic form of this new neural class involves scale, shift and shape parameters so as to obtain a wide range of activation shapes, including the standard rectified linear unit as a limit case. Parameters of this neural transfer class are considered as learnable for the sake of discovering the complex shapes that can contribute in solving machine learning issues. Performance achieved by the joint learning of convolutional and rectified power sigmoid learnable parameters are shown outstanding in both shallow and deep learning frameworks. This class opens new prospects with respect to machine learning in the sense that learnable parameters are not only attached to linear transformations, but also to suitable nonlinear operators.
Activation Functions in Artificial Neural Networks: A Systematic Overview
Venue: arXiv cs.LG 2021
Year: 2021

Activation functions shape the outputs of artificial neurons and, therefore, are integral parts of neural networks in general and deep learning in particular. Some activation functions, such as logistic and relu, have been used for many decades. But with deep learning becoming a mainstream research topic, new activation functions have mushroomed, leading to confusion in both theory and practice. This paper provides an analytic yet up-to-date overview of popular activation functions and their properties, which makes it a timely resource for anyone who studies or applies neural networks.
CHOLAN: A Modular Approach for Neural Entity Linking on Wikipedia and Wikidata
Venue: arXiv cs.CL 2021
Year: 2021

In this paper, we propose CHOLAN, a modular approach to target end-to-end entity linking (EL) over knowledge bases. CHOLAN consists of a pipeline of two transformer-based models integrated sequentially to accomplish the EL task. The first transformer model identifies surface forms (entity mentions) in a given text. For each mention, a second transformer model is employed to classify the target entity among a predefined candidates list. The latter transformer is fed by an enriched context captured from the sentence (i.e. local context), and entity description gained from Wikipedia. Such external contexts have not been used in the state of the art EL approaches. Our empirical study was conducted on two well-known knowledge bases (i.e., Wikidata and Wikipedia). The empirical results suggest that CHOLAN outperforms state-of-the-art approaches on standard datasets such as CoNLL-AIDA, MSNBC, AQUAINT, ACE2004, and T-REx.
Unsupervised Key-phrase Extraction and Clustering for Classification Scheme in Scientific Publications
Venue: arXiv cs.CL 2021
Year: 2021

Several methods have been explored for automating parts of Systematic Mapping (SM) and Systematic Review (SR) methodologies. Challenges typically evolve around the gaps in semantic understanding of text, as well as lack of domain and background knowledge necessary to bridge that gap. In this paper we investigate possible ways of automating parts of the SM/SR process, i.e. that of extracting keywords and key-phrases from scientific documents using unsupervised methods, which are then used as a basis to construct the corresponding Classification Scheme using semantic key-phrase clustering techniques. Specifically, we explore the effect of ensemble scores measure in key-phrase extraction, we explore semantic network based word embedding in embedding representation of phrase semantics and finally we also explore how clustering can be used to group related key-phrases. The evaluation is conducted on a dataset of publications pertaining the domain of "Explainable AI" which we constructed using standard publicly available digital libraries and sets of indexing terms (keywords). Results shows that: ensemble ranking score does improve the key-phrase extraction performance. Semantic-network based word embedding based on the ConceptNet Semantic Network has similar performance with contextualized word embedding, however the former are computationally more efficient. Finally Semantic key-phrase clustering at term-level can group similar terms together that can be suitable for classification scheme.
They See Me Rollin': Inherent Vulnerability of the Rolling Shutter in CMOS Image Sensors
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we describe how the electronic rolling shutter in CMOS image sensors can be exploited using a bright, modulated light source (e.g., an inexpensive, off-the-shelf laser), to inject fine-grained image disruptions. We demonstrate the attack on seven different CMOS cameras, ranging from cheap IoT to semi-professional surveillance cameras, to highlight the wide applicability of the rolling shutter attack. We model the fundamental factors affecting a rolling shutter attack in an uncontrolled setting. We then perform an exhaustive evaluation of the attack's effect on the task of object detection, investigating the effect of attack parameters. We validate our model against empirical data collected on two separate cameras, showing that by simply using information from the camera's datasheet the adversary can accurately predict the injected distortion size and optimize their attack accordingly. We find that an adversary can hide up to 75% of objects perceived by state-of-the-art detectors by selecting appropriate attack parameters. We also investigate the stealthiness of the attack in comparison to a naïve camera blinding attack, showing that common image distortion metrics can not detect the attack presence. Therefore, we present a new, accurate and lightweight enhancement to the backbone network of an object detector to recognize rolling shutter attacks. Overall, our results indicate that rolling shutter attacks can substantially reduce the performance and reliability of vision-based intelligent systems.
A Process to Facilitate Automated Automotive Cybersecurity Testing
Venue: arXiv cs.CR 2021
Year: 2021

Modern vehicles become increasingly digitalized with advanced information technology-based solutions like advanced driving assistance systems and vehicle-to-x communications. These systems are complex and interconnected. Rising complexity and increasing outside exposure has created a steadily rising demand for more cyber-secure systems. Thus, also standardization bodies and regulators issued standards and regulations to prescribe more secure development processes. This security, however, also has to be validated and verified. In order to keep pace with the need for more thorough, quicker and comparable testing, today's generally manual testing processes have to be structured and optimized. Based on existing and emerging standards for cybersecurity engineering, this paper therefore outlines a structured testing process for verifying and validating automotive cybersecurity, for which there is no standardized method so far. Despite presenting a commonly structured framework, the process is flexible in order to allow implementers to utilize their own, accustomed toolsets.
Agents.jl: A performant and feature-full agent based modelling software of minimal code complexity
Venue: arXiv cs.MA 2021
Year: 2021

Agent based modelling is a simulation method in which autonomous agents interact with their environment and one another, given a predefined set of rules. It is an integral method for modelling and simulating complex systems, such as socio-economic problems. Since agent based models are not described by simple and concise mathematical equations, code that generates them is typically complicated, large, and slow. Here we present Agents.jl, a Julia-based software that provides an ABM analysis platform with minimal code complexity. We compare our software with some of the most popular ABM software in other programming languages. We find that Agents.jl is not only the most performant, but also the least complicated software, providing the same (and sometimes more) features as the competitors with less input required from the user. Agents.jl also integrates excellently with the entire Julia ecosystem, including interactive applications, differential equations, parameter optimization, and more. This removes any ``extensions library'' requirement from Agents.jl, which is paramount in many other tools.
Unveiling Spatial Patterns of Disaster Impacts and Recovery Using Credit Card Transaction Variances
Venue: arXiv cs.CY 2021
Year: 2021

The objective of this study is to examine spatial patterns of impacts and recovery of communities based on variances in credit card transactions. Such variances could capture the collective effects of household impacts, disrupted accesses, and business closures, and thus provide an integrative measure for examining disaster impacts and community recovery in disasters. Existing studies depend mainly on survey and sociodemographic data for disaster impacts and recovery effort evaluations, although such data has limitations, including large data collection efforts and delayed timeliness results. In addition, there are very few studies have concentrated on spatial patterns and disparities of disaster impacts and short-term recovery of communities, although such investigation can enhance situational awareness during disasters and support the identification of disparate spatial patterns of disaster impacts and recovery in the impacted regions. This study examines credit card transaction data Harris County (Texas, USA) during Hurricane Harvey in 2017 to explore spatial patterns of disaster impacts and recovery during from the perspective of community residents and businesses at ZIP code and county scales, respectively, and to further investigate their spatial disparities across ZIP codes. The results indicate that individuals in ZIP codes with populations of higher income experienced more severe disaster impact and recovered more quickly than those located in lower-income ZIP codes for most business sectors. Our findings not only enhance the understanding of spatial patterns and disparities in disaster impacts and recovery for better community resilience assessment, but also could benefit emergency managers, city planners, and public officials in harnessing population activity data, using credit card transactions as a proxy for activity, to improve situational awareness and resource allocation.
Learning Outcome Oriented Programmatic Assessment
Venue: arXiv cs.CY 2021
Year: 2021

This paper describes considerations behind the organisation of a third semester BSc education. The project aims to facilitate a feedback-oriented environment using assessment for learning and for incremental measure of learner progress [Vleuten et al, 2012, "A model for programmatic assessment fit for purpose"]. Learning outcomes encourage higher order cognitive skills, following [Biggs & Tang, 2011,"Teaching for quality learning at university: what the student does"]. Embracing [Dochy et al. 2018, "Creating Impact Through Future Learning: The High Impact Learning that Lasts (HILL) Model"], several mechanisms encourage focus and motivation.
Lightweight Convolutional Neural Network with Gaussian-based Grasping Representation for Robotic Grasping Detection
Venue: arXiv cs.CV 2021
Year: 2021

The method of deep learning has achieved excellent results in improving the performance of robotic grasping detection. However, the deep learning methods used in general object detection are not suitable for robotic grasping detection. Current modern object detectors are difficult to strike a balance between high accuracy and fast inference speed. In this paper, we present an efficient and robust fully convolutional neural network model to perform robotic grasping pose estimation from an n-channel input image of the real grasping scene. The proposed network is a lightweight generative architecture for grasping detection in one stage. Specifically, a grasping representation based on Gaussian kernel is introduced to encode training samples, which embodies the principle of maximum central point grasping confidence. Meanwhile, to extract multi-scale information and enhance the feature discriminability, a receptive field block (RFB) is assembled to the bottleneck of our grasping detection architecture. Besides, pixel attention and channel attention are combined to automatically learn to focus on fusing context information of varying shapes and sizes by suppressing the noise feature and highlighting the grasping object feature. Extensive experiments on two public grasping datasets, Cornell and Jacquard demonstrate the state-of-the-art performance of our method in balancing accuracy and inference speed. The network is an order of magnitude smaller than other excellent algorithms while achieving better performance with an accuracy of 98.9$\%$ and 95.6$\%$ on the Cornell and Jacquard datasets, respectively.
COVID-19 Outbreak Prediction and Analysis using Self Reported Symptoms
Venue: arXiv cs.LG 2021
Year: 2021

It is crucial for policymakers to understand the community prevalence of COVID-19 so combative resources can be effectively allocated and prioritized during the COVID-19 pandemic. Traditionally, community prevalence has been assessed through diagnostic and antibody testing data. However, despite the increasing availability of COVID-19 testing, the required level has not been met in most parts of the globe, introducing a need for an alternative method for communities to determine disease prevalence. This is further complicated by the observation that COVID-19 prevalence and spread varies across different spatial, temporal, and demographics. In this study, we understand trends in the spread of COVID-19 by utilizing the results of self-reported COVID-19 symptoms surveys as an alternative to COVID-19 testing reports. This allows us to assess community disease prevalence, even in areas with low COVID-19 testing ability. Using individually reported symptom data from various populations, our method predicts the likely percentage of the population that tested positive for COVID-19. We do so with a Mean Absolute Error (MAE) of 1.14 and Mean Relative Error (MRE) of 60.40\% with 95\% confidence interval as (60.12, 60.67). This implies that our model predicts +/- 1140 cases than the original in a population of 1 million. In addition, we forecast the location-wise percentage of the population testing positive for the next 30 days using self-reported symptoms data from previous days. The MAE for this method is as low as 0.15 (MRE of 23.61\% with 95\% confidence interval as (23.6, 13.7)) for New York. We present an analysis of these results, exposing various clinical attributes of interest across different demographics. Lastly, we qualitatively analyze how various policy enactments (testing, curfew) affect the prevalence of COVID-19 in a community.
Violent Crime in London: An Investigation using Geographically Weighted Regression
Venue: arXiv cs.CY 2021
Year: 2021

Violent crime in London is an area of increasing interest following policing and community budget cuts in recent years. Understanding the locally-varying demographic factors that drive distribution of violent crime rate in London could be a means to more effective policy making for effective action. Using a visual analytics approach combined with Statsitical Methods, demographic features which are traditionally related to Violent Crime Rate (VCR) are identified and OLS Univariate and Multivariate Regression are used as a precursor to GWR. VIF and pearson correlation statistics show strong colinearity in many of the traditionally used features and so human reasoning is used to rectify this. Bandwidth kernel smoothing size of 67 with a Bi-Square type is best for GWR. GWR and OLS regression shows that there is local variation in VCR and K-Means clustering using 5 clusters provides an effective way of seperating violent crime in London into 5 coherent groups.