OpenRaiser/Intern-Atlas huggingface dataset:
Recovery of Binary Sparse Signals from Structured Biased Measurements
Venue: arXiv cs.IT 2020
Year: 2020

In this paper we study the reconstruction of binary sparse signals from partial random circulant measurements. We show that the reconstruction via the least-squares algorithm is as good as the reconstruction via the usually used program basis pursuit. We further show that we need as many measurements to recover an $s$-sparse signal $x_0\in\mathbb{R}^N$ as we need to recover a dense signal, more-precisely an $N-s$-sparse signal $x_0\in\mathbb{R}^N$. We further establish stability with respect to noisy measurements.
Biologically Plausible Learning of Text Representation with Spiking Neural Networks
Venue: arXiv cs.NE 2020
Year: 2020

This study proposes a novel biologically plausible mechanism for generating low-dimensional spike-based text representation. First, we demonstrate how to transform documents into series of spikes spike trains which are subsequently used as input in the training process of a spiking neural network (SNN). The network is composed of biologically plausible elements, and trained according to the unsupervised Hebbian learning rule, Spike-Timing-Dependent Plasticity (STDP). After training, the SNN can be used to generate low-dimensional spike-based text representation suitable for text/document classification. Empirical results demonstrate that the generated text representation may be effectively used in text classification leading to an accuracy of $80.19\%$ on the bydate version of the 20 newsgroups data set, which is a leading result amongst approaches that rely on low-dimensional text representations.
The Sci-hub Effect: Sci-hub downloads lead to more article citations
Venue: arXiv cs.DL 2020
Year: 2020

Citations are often used as a metric of the impact of scientific publications. Here, we examine how the number of downloads from Sci-hub as well as various characteristics of publications and their authors predicts future citations. Using data from 12 leading journals in economics, consumer research, neuroscience, and multidisciplinary research, we found that articles downloaded from Sci-hub were cited 1.72 times more than papers not downloaded from Sci-hub and that the number of downloads from Sci-hub was a robust predictor of future citations. Among other characteristics of publications, the number of figures in a manuscript consistently predicts its future citations. The results suggest that limited access to publications may limit some scientific research from achieving its full impact.
Tensor estimation with structured priors
Venue: arXiv cs.IT 2020
Year: 2020

We consider rank-one symmetric tensor estimation when the tensor is corrupted by Gaussian noise and the spike forming the tensor is a structured signal coming from a generalized linear model. The latter is a mathematically tractable model of a non-trivial hidden lower-dimensional latent structure in a signal. We work in a large dimensional regime with fixed ratio of signal-to-latent space dimensions. Remarkably, in this asymptotic regime, the mutual information between the spike and the observations can be expressed as a finite-dimensional variational problem, and it is possible to deduce the minimum-mean-square-error from its solution. We discuss, on examples, properties of the phase transitions as a function of the signal-to-noise ratio. Typically, the critical signal-to-noise ratio decreases with increasing signal-to-latent space dimensions. We discuss the limit of vanishing ratio of signal-to-latent space dimensions and determine the limiting tensor estimation problem. We also point out similarities and differences with the case of matrices.
Deep-learning Autoencoder for Coherent and Nonlinear Optical Communication
Venue: arXiv cs.IT 2020
Year: 2020

Motivated by the recent success of end-to-end training of communications in the wireless domain, we strive to adapt the end-to-end-learning idea from the wireless case (i.e., linear) to coherent optical fiber links (i.e., nonlinear). Although, at first glance, it sounds like a straightforward extension, it turns out that several pitfalls exist - in terms of theory but also in terms of practical implementation. This paper analyzes the potential of an autoencoder and limitations for the optical fiber under the influence of Kerr-nonlinearity and chromatic dispersion. As there is no exact capacity limit known and, hence, no analytical perfect system solution available, we set great value to the interpretability on the learnings of the autoencoder. Therefore, we design its architecture to be as close as possible to the structure of a classic communication system, knowing that this may limit its degree of freedom and, thus, its performance. Nevertheless, we were able to achieve an unexpected high gain in terms of spectral efficiency compared to a conventional reference system.
Cross-Supervised Object Detection
Venue: arXiv cs.CV 2020
Year: 2020

After learning a new object category from image-level annotations (with no object bounding boxes), humans are remarkably good at precisely localizing those objects. However, building good object localizers (i.e., detectors) currently requires expensive instance-level annotations. While some work has been done on learning detectors from weakly labeled samples (with only class labels), these detectors do poorly at localization. In this work, we show how to build better object detectors from weakly labeled images of new categories by leveraging knowledge learned from fully labeled base categories. We call this novel learning paradigm cross-supervised object detection. We propose a unified framework that combines a detection head trained from instance-level annotations and a recognition head learned from image-level annotations, together with a spatial correlation module that bridges the gap between detection and recognition. These contributions enable us to better detect novel objects with image-level annotations in complex multi-object scenes such as the COCO dataset.
Computing Light Transport Gradients using the Adjoint Method
Venue: arXiv cs.GR 2020
Year: 2020

This paper proposes a new equation from continuous adjoint theory to compute the gradient of quantities governed by the Transport Theory of light. Unlike discrete gradients ala autograd, which work at the code level, we first formulate the continuous theory and then discretize it. The key insight of this paper is that computing gradients in Transport Theory is akin to computing the importance, a quantity adjoint to radiance that satisfies an adjoint equation. Importance tells us where to look for light that matters. This is one of the key insights of this paper. In fact, this mathematical journey started from a whimsical thought that these adjoints might be related. Computing gradients is therefore no more complicated than computing the importance field. This insight and the following paper hopefully will shed some light on this complicated problem and ease the implementations of gradient computations in existing path tracers.
ULSAM: Ultra-Lightweight Subspace Attention Module for Compact Convolutional Neural Networks
Venue: arXiv cs.CV 2020
Year: 2020

The capability of the self-attention mechanism to model the long-range dependencies has catapulted its deployment in vision models. Unlike convolution operators, self-attention offers infinite receptive field and enables compute-efficient modeling of global dependencies. However, the existing state-of-the-art attention mechanisms incur high compute and/or parameter overheads, and hence unfit for compact convolutional neural networks (CNNs). In this work, we propose a simple yet effective "Ultra-Lightweight Subspace Attention Mechanism" (ULSAM), which infers different attention maps for each feature map subspace. We argue that leaning separate attention maps for each feature subspace enables multi-scale and multi-frequency feature representation, which is more desirable for fine-grained image classification. Our method of subspace attention is orthogonal and complementary to the existing state-of-the-arts attention mechanisms used in vision models. ULSAM is end-to-end trainable and can be deployed as a plug-and-play module in the pre-existing compact CNNs. Notably, our work is the first attempt that uses a subspace attention mechanism to increase the efficiency of compact CNNs. To show the efficacy of ULSAM, we perform experiments with MobileNet-V1 and MobileNet-V2 as backbone architectures on ImageNet-1K and three fine-grained image classification datasets. We achieve $\approx$13% and $\approx$25% reduction in both the FLOPs and parameter counts of MobileNet-V2 with a 0.27% and more than 1% improvement in top-1 accuracy on the ImageNet-1K and fine-grained image classification datasets (respectively). Code and trained models are available at https://github.com/Nandan91/ULSAM.
Learning predictive representations in autonomous driving to improve deep reinforcement learning
Venue: arXiv cs.LG 2020
Year: 2020

Reinforcement learning using a novel predictive representation is applied to autonomous driving to accomplish the task of driving between lane markings where substantial benefits in performance and generalization are observed on unseen test roads in both simulation and on a real Jackal robot. The novel predictive representation is learned by general value functions (GVFs) to provide out-of-policy, or counter-factual, predictions of future lane centeredness and road angle that form a compact representation of the state of the agent improving learning in both online and offline reinforcement learning to learn to drive an autonomous vehicle with methods that generalizes well to roads not in the training data. Experiments in both simulation and the real-world demonstrate that predictive representations in reinforcement learning improve learning efficiency, smoothness of control and generalization to roads that the agent was never shown during training, including damaged lane markings. It was found that learning a predictive representation that consists of several predictions over different time scales, or discount factors, improves the performance and smoothness of the control substantially. The Jackal robot was trained in a two step process where the predictive representation is learned first followed by a batch reinforcement learning algorithm (BCQ) from data collected through both automated and human-guided exploration in the environment. We conclude that out-of-policy predictive representations with GVFs offer reinforcement learning many benefits in real-world problems.
Generating induction principles and subterm relations for inductive types using MetaCoq
Venue: arXiv cs.LO 2020
Year: 2020

We implement three Coq plugins regarding inductive types in MetaCoq. The first plugin is a simple syntax transformation generating alternative constructors for inductive types by abstracting over concrete indices in the types of the constructors. The second plugin re-implements Coq's $\texttt{Scheme Induction}$ command in MetaCoq, and extends it to nested inductive types, e.g. types like rose trees which use $\texttt{list}$ in their definition, similar to the Elpi-plugin by Tassi. The third plugin implements the $\texttt{Derive Subterm}$ command provided by the Equations package in MetaCoq.
Online learning with Corrupted context: Corrupted Contextual Bandits
Venue: arXiv cs.LG 2020
Year: 2020

We consider a novel variant of the contextual bandit problem (i.e., the multi-armed bandit with side-information, or context, available to a decision-maker) where the context used at each decision may be corrupted ("useless context"). This new problem is motivated by certain on-line settings including clinical trial and ad recommendation applications. In order to address the corrupted-context setting,we propose to combine the standard contextual bandit approach with a classical multi-armed bandit mechanism. Unlike standard contextual bandit methods, we are able to learn from all iteration, even those with corrupted context, by improving the computing of the expectation for each arm. Promising empirical results are obtained on several real-life datasets.
Mapping the South African health landscape in response to COVID-19
Venue: arXiv cs.CY 2020
Year: 2020

When the COVID-19 disease pandemic infiltrated the world, there was an immediate need for accurate information. As with any outbreak, the outbreak follows a clear trajectory, and subsequently, the supporting information for that outbreak needs to address the needs associated with that stage of the outbreak. At first, there was a need to inform the public of the information related to the initial situation related to the "who" of the COVID-19 disease. However, as time continued, the "where", "when" and "how to" related questions started to emerge in relation to the public healthcare system themselves. Questions surrounding the health facilities including COVID-19 hospital bed capacity, locations of designated COVID-19 facilities, and general information related to these facilities were not easily accessible to the general public. Furthermore, the available information was found to be outdated, fragmented across several platforms, and still had gaps in the data related to these facilities. To rectify this problem, a group of volunteers working on the covid19za project stepped in to assist. Each member leading a part of the project chose to focus on one of four problems related to the challenges associated with the Hospital information including: data quality, data completeness, data source validation and data visualisation capacity. As the project developed, so did the sophistication of the data, visualisation and core function of the project. The future prospects of this project relate to a Progressive Web Application that will avail this information for the public as well as healthcare workers through comprehensive mapping and data quality.
On the Relationship Between Probabilistic Circuits and Determinantal Point Processes
Venue: arXiv cs.AI 2020
Year: 2020

Scaling probabilistic models to large realistic problems and datasets is a key challenge in machine learning. Central to this effort is the development of tractable probabilistic models (TPMs): models whose structure guarantees efficient probabilistic inference algorithms. The current landscape of TPMs is fragmented: there exist various kinds of TPMs with different strengths and weaknesses. Two of the most prominent classes of TPMs are determinantal point processes (DPPs) and probabilistic circuits (PCs). This paper provides the first systematic study of their relationship. We propose a unified analysis and shared language for discussing DPPs and PCs. Then we establish theoretical barriers for the unification of these two families, and prove that there are cases where DPPs have no compact representation as a class of PCs. We close with a perspective on the central problem of unifying these tractable models.
Optimizing Cuckoo Filter for high burst tolerance,low latency, and high throughput
Venue: arXiv cs.DC 2020
Year: 2020

In this paper, we present an implementation of a cuckoo filter for membership testing, optimized for distributed data stores operating in high workloads. In large databases, querying becomes inefficient using traditional search methods. To achieve optimal performance it is necessary to use probabilistic data structures to test the membership of a given key, at the cost of getting false positives while querying data. The widely used bloom filters can be used for this, but they have limitations like no support for deletes. To improve upon this we use a modified version of the cuckoo filter that gives better amortized times for search, with less false positives.
Software Enabled Security Architecture and Mechanisms for Securing 5G Network Services
Venue: arXiv cs.CR 2020
Year: 2020

The 5G network systems are evolving and have complex network infrastructures. There is a great deal of work in this area focused on meeting the stringent service requirements for the 5G networks. Within this context, security requirements play a critical role as 5G networks can support a range of services such as healthcare services, financial and critical infrastructures. 3GPP and ETSI have been developing security frameworks for 5G networks. Our work in 5G security has been focusing on the design of security architecture and mechanisms enabling dynamic establishment of secure and trusted end to end services as well as development of mechanisms to proactively detect and mitigate security attacks in virtualised network infrastructures. The focus of this paper is on the latter, namely the facilities and mechanisms, and the design of a security architecture providing facilities and mechanisms to detect and mitigate specific security attacks. We have developed and implemented a simplified version of the security architecture using Software Defined Networks (SDN) and Network Function Virtualisation (NFV) technologies. The specific security functions developed in this architecture can be directly integrated into the 5G core network facilities enhancing its security. We describe the design and implementation of the security architecture and demonstrate how it can efficiently mitigate specific types of attacks.
Software Enabled Security Architecture for Counteracting Attacks in Control Systems
Venue: arXiv cs.CR 2020
Year: 2020

Increasingly Industrial Control Systems (ICS) systems are being connected to the Internet to minimise the operational costs and provide additional flexibility. These control systems such as the ones used in power grids, manufacturing and utilities operate continually and have long lifespans measured in decades rather than years as in the case of IT systems. Such industrial control systems require uninterrupted and safe operation. However, they can be vulnerable to a variety of attacks, as successful attacks on critical control infrastructures could have devastating consequences to the safety of human lives as well as a nation's security and prosperity. Furthermore, there can be a range of attacks that can target ICS and it is not easy to secure these systems against all known attacks let alone unknown ones. In this paper, we propose a software enabled security architecture using Software Defined Networking (SDN) and Network Function Virtualisation (NFV) that can enhance the capability to secure industrial control systems. We have designed such an SDN/NFV enabled security architecture and developed a Control System Security Application (CSSA) in SDN Controller for enhancing security in ICS against certain specific attacks namely denial of service attacks, from unpatched vulnerable control system components and securing the communication flows from the legacy devices that do not support any security functionality. In this paper, we discuss the prototype implementation of the proposed architecture and the results obtained from our analysis.
A Blockchain-based Approach for Assessing Compliance with SLA-guaranteed IoT Services
Venue: arXiv cs.DC 2020
Year: 2020

Within cloud-based internet of things (IoT) applications, typically cloud providers employ Service Level Agreements (SLAs) to ensure the quality of their provisioned services. Similar to any other contractual method, an SLA is not immune to breaches. Ideally, an SLA stipulates consequences (e.g. penalties) imposed on cloud providers when they fail to conform to SLA terms. The current practice assumes trust in service providers to acknowledge SLA breach incidents and executing associated consequences. Recently, the Blockchain paradigm has introduced compelling capabilities that may enable us to address SLA enforcement more elegantly. This paper proposes and implements a blockchain-based approach for assessing SLA compliance and enforcing consequences. It employs a diagnostic accuracy method for validating the dependability of the proposed solution. The paper also benchmarks Hyperledger Fabric to investigate its feasibility as an underlying blockchain infrastructure concerning latency and transaction success/fail rates.
Interactive Deep Refinement Network for Medical Image Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

Deep learning techniques have successfully been employed in numerous computer vision tasks including image segmentation. The techniques have also been applied to medical image segmentation, one of the most critical tasks in computer-aided diagnosis. Compared with natural images, the medical image is a gray-scale image with low-contrast (even with some invisible parts). Because some organs have similar intensity and texture with neighboring organs, there is usually a need to refine automatic segmentation results. In this paper, we propose an interactive deep refinement framework to improve the traditional semantic segmentation networks such as U-Net and fully convolutional network. In the proposed framework, we added a refinement network to traditional segmentation network to refine the segmentation results.Experimental results with public dataset revealed that the proposed method could achieve higher accuracy than other state-of-the-art methods.
Iterative Machine Teaching without Teachers
Venue: arXiv cs.LG 2020
Year: 2020

Iterative machine teaching is a method for selecting an optimal teaching example that enables a student to efficiently learn a target concept at each iteration. Existing studies on iterative machine teaching are based on supervised machine learning and assume that there are teachers who know the true answers of all teaching examples. In this study, we consider an unsupervised case where such teachers do not exist; that is, we cannot access the true answer of any teaching example. Students are given a teaching example at each iteration, but there is no guarantee if the corresponding label is correct. Recent studies on crowdsourcing have developed methods for estimating the true answers from crowdsourcing responses. In this study, we apply these to iterative machine teaching for estimating the true labels of teaching examples along with student models that are used for teaching. Our method supports the collaborative learning of students without teachers. The experimental results show that the teaching performance of our method is particularly effective for low-level students in particular.
MiniNet: An extremely lightweight convolutional neural network for real-time unsupervised monocular depth estimation
Venue: arXiv cs.CV 2020
Year: 2020

Predicting depth from a single image is an attractive research topic since it provides one more dimension of information to enable machines to better perceive the world. Recently, deep learning has emerged as an effective approach to monocular depth estimation. As obtaining labeled data is costly, there is a recent trend to move from supervised learning to unsupervised learning to obtain monocular depth. However, most unsupervised learning methods capable of achieving high depth prediction accuracy will require a deep network architecture which will be too heavy and complex to run on embedded devices with limited storage and memory spaces. To address this issue, we propose a new powerful network with a recurrent module to achieve the capability of a deep network while at the same time maintaining an extremely lightweight size for real-time high performance unsupervised monocular depth prediction from video sequences. Besides, a novel efficient upsample block is proposed to fuse the features from the associated encoder layer and recover the spatial size of features with the small number of model parameters. We validate the effectiveness of our approach via extensive experiments on the KITTI dataset. Our new model can run at a speed of about 110 frames per second (fps) on a single GPU, 37 fps on a single CPU, and 2 fps on a Raspberry Pi 3. Moreover, it achieves higher depth accuracy with nearly 33 times fewer model parameters than state-of-the-art models. To the best of our knowledge, this work is the first extremely lightweight neural network trained on monocular video sequences for real-time unsupervised monocular depth estimation, which opens up the possibility of implementing deep learning-based real-time unsupervised monocular depth prediction on low-cost embedded devices.