OpenRaiser/Intern-Atlas huggingface dataset:
Real-Time Nonparametric Anomaly Detection in High-Dimensional Settings
Venue: arXiv cs.LG 2020
Year: 2020

Timely detection of abrupt anomalies is crucial for real-time monitoring and security of modern systems producing high-dimensional data. With this goal, we propose effective and scalable algorithms. Proposed algorithms are nonparametric as both the nominal and anomalous multivariate data distributions are assumed unknown. We extract useful univariate summary statistics and perform anomaly detection in a single-dimensional space. We model anomalies as persistent outliers and propose to detect them via a cumulative sum-like algorithm. In case the observed data have a low intrinsic dimensionality, we learn a submanifold in which the nominal data are embedded and evaluate whether the sequentially acquired data persistently deviate from the nominal submanifold. Further, in the general case, we learn an acceptance region for nominal data via Geometric Entropy Minimization and evaluate whether the sequentially observed data persistently fall outside the acceptance region. We provide an asymptotic lower bound and an asymptotic approximation for the average false alarm period of the proposed algorithm. Moreover, we provide a sufficient condition to asymptotically guarantee that the decision statistic of the proposed algorithm does not diverge in the absence of anomalies. Experiments illustrate the effectiveness of the proposed schemes in quick and accurate anomaly detection in high-dimensional settings.
Kernelization and approximation of distance-$r$ independent sets on nowhere dense graphs
Venue: arXiv cs.DM 2020
Year: 2020

For a positive integer $r$, a distance-$r$ independent set in an undirected graph $G$ is a set $I\subseteq V(G)$ of vertices pairwise at distance greater than $r$, while a distance-$r$ dominating set is a set $D\subseteq V(G)$ such that every vertex of the graph is within distance at most $r$ from a vertex from $D$. We study the duality between the maximum size of a distance-$2r$ independent set and the minimum size of a distance-$r$ dominating set in nowhere dense graph classes, as well as the kernelization complexity of the distance-$r$ independent set problem on these graph classes. Specifically, we prove that the distance-$r$ independent set problem admits an almost linear kernel on every nowhere dense graph class.
Sequence-Subset Distance and Coding for Error Control in DNA-based Data Storage
Venue: arXiv cs.IT 2020
Year: 2020

The process of DNA-based data storage (DNA storage for short) can be mathematically modelled as a communication channel, termed DNA storage channel, whose inputs and outputs are sets of unordered sequences. To design error correcting codes for DNA storage channel, a new metric, termed the sequence-subset distance, is introduced, which generalizes the Hamming distance to a distance function defined between any two sets of unordered vectors and helps to establish a uniform framework to design error correcting codes for DNA storage channel. We further introduce a family of error correcting codes, referred to as \emph{sequence-subset codes}, for DNA storage and show that the error-correcting ability of such codes is completely determined by their minimum distance. We derive some upper bounds on the size of the sequence-subset codes including a tight bound for a special case, a Singleton-like bound and a Plotkin-like bound. We also propose some constructions, including an optimal construction for that special case, which imply lower bounds on the size of such codes.
On the Learning Dynamics of Deep Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

While a lot of progress has been made in recent years, the dynamics of learning in deep nonlinear neural networks remain to this day largely misunderstood. In this work, we study the case of binary classification and prove various properties of learning in such networks under strong assumptions such as linear separability of the data. Extending existing results from the linear case, we confirm empirical observations by proving that the classification error also follows a sigmoidal shape in nonlinear architectures. We show that given proper initialization, learning expounds parallel independent modes and that certain regions of parameter space might lead to failed training. We also demonstrate that input norm and features' frequency in the dataset lead to distinct convergence speeds which might shed some light on the generalization capabilities of deep neural networks. We provide a comparison between the dynamics of learning with cross-entropy and hinge losses, which could prove useful to understand recent progress in the training of generative adversarial networks. Finally, we identify a phenomenon that we baptize gradient starvation where the most frequent features in a dataset prevent the learning of other less frequent but equally informative features.
Modeling Online Discourse with Coupled Distributed Topics
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we propose a deep, globally normalized topic model that incorporates structural relationships connecting documents in socially generated corpora, such as online forums. Our model (1) captures discursive interactions along observed reply links in addition to traditional topic information, and (2) incorporates latent distributed representations arranged in a deep architecture, which enables a GPU-based mean-field inference procedure that scales efficiently to large data. We apply our model to a new social media dataset consisting of 13M comments mined from the popular internet forum Reddit, a domain that poses significant challenges to models that do not account for relationships connecting user comments. We evaluate against existing methods across multiple metrics including perplexity and metadata prediction, and qualitatively analyze the learned interaction patterns.
Faster RER-CNN: application to the detection of vehicles in aerial images
Venue: arXiv cs.CV 2020
Year: 2020

Detecting small vehicles in aerial images is a difficult job that can be challenging even for humans. Rotating objects, low resolution, small inter-class variability and very large images comprising complicated backgrounds render the work of photo-interpreters tedious and wearisome. Unfortunately even the best classical detection pipelines like Faster R-CNN cannot be used off-the-shelf with good results because they were built to process object centric images from day-to-day life with multi-scale vertical objects. In this work we build on the Faster R-CNN approach to turn it into a detection framework that deals appropriately with the rotation equivariance inherent to any aerial image task. This new pipeline (Faster Rotation Equivariant Regions CNN) gives, without any bells and whistles, state-of-the-art results on one of the most challenging aerial imagery datasets: VeDAI and give good results w.r.t. the baseline Faster R-CNN on two others: Munich and GoogleEarth .
Understanding Convolutional Neural Networks for Text Classification
Venue: arXiv cs.CL 2020
Year: 2020

We present an analysis into the inner workings of Convolutional Neural Networks (CNNs) for processing text. CNNs used for computer vision can be interpreted by projecting filters into image space, but for discrete sequence inputs CNNs remain a mystery. We aim to understand the method by which the networks process and classify text. We examine common hypotheses to this problem: that filters, accompanied by global max-pooling, serve as ngram detectors. We show that filters may capture several different semantic classes of ngrams by using different activation patterns, and that global max-pooling induces behavior which separates important ngrams from the rest. Finally, we show practical use cases derived from our findings in the form of model interpretability (explaining a trained model by deriving a concrete identity for each filter, bridging the gap between visualization tools in vision tasks and NLP) and prediction interpretability (explaining predictions). Code implementation is available online at github.com/sayaendo/interpreting-cnn-for-text.
Legal Assignments and fast EADAM with consent via classical theory of stable matchings
Venue: arXiv cs.DS 2020
Year: 2020

Gale and Shapley's stable assignment problem has been extensively studied, applied, and extended. In the context of school choice, mechanisms often aim at finding an assignment that is more favorable to students. We investigate two extensions introduced in this framework -- legal assignments and the EADAM algorithm -- through the lens of classical theory of stable matchings. In any instance, the set ${\cal L}$ of legal assignments is known to contain all stable assignments. We prove that ${\cal L}$ is exactly the set of stable assignments in another instance. Moreover, we show that essentially all optimization problems over ${\cal L}$ can be solved within the same time bound needed for solving it over the set of stable assignments. A key tool for this latter result is an algorithm that finds the student-optimal legal assignment. We then generalize our algorithm to obtain the assignment output of EADAM with any given set of consenting students without sacrificing the running time, hence largely improving in both theory and practice over known algorithms. Lastly, we show that the set ${\cal L}$ can be much larger than the set of stable matchings, connecting legal matchings with certain concepts and open problems in the literature.
An Adaptive Locally Connected Neuron Model: Focusing Neuron
Venue: arXiv cs.NE 2020
Year: 2020

This paper presents a new artificial neuron model capable of learning its receptive field in the topological domain of inputs. The model provides adaptive and differentiable local connectivity (plasticity) applicable to any domain. It requires no other tool than the backpropagation algorithm to learn its parameters which control the receptive field locations and apertures. This research explores whether this ability makes the neuron focus on informative inputs and yields any advantage over fully connected neurons. The experiments include tests of focusing neuron networks of one or two hidden layers on synthetic and well-known image recognition data sets. The results demonstrated that the focusing neurons can move their receptive fields towards more informative inputs. In the simple two-hidden layer networks, the focusing layers outperformed the dense layers in the classification of the 2D spatial data sets. Moreover, the focusing networks performed better than the dense networks even when 70$\%$ of the weights were pruned. The tests on convolutional networks revealed that using focusing layers instead of dense layers for the classification of convolutional features may work better in some data sets.
A Successive-Elimination Approach to Adaptive Robotic Sensing
Venue: arXiv cs.LG 2020
Year: 2020

We study an adaptive source seeking problem, in which a mobile robot must identify the strongest emitter(s) of a signal in an environment with background emissions. Background signals may be highly heterogeneous and can mislead algorithms that are based on receding horizon control. We propose AdaSearch, a general algorithm for adaptive source seeking in the face of heterogeneous background noise. AdaSearch combines global trajectory planning with principled confidence intervals in order to concentrate measurements in promising regions while guaranteeing sufficient coverage of the entire area. Theoretical analysis shows that AdaSearch confers gains over a uniform sampling strategy when the distribution of background signals is highly variable. Simulation experiments demonstrate that when applied to the problem of radioactive source seeking, AdaSearch outperforms both uniform sampling and a receding time horizon information-maximization approach based on the current literature. We also demonstrate AdaSearch in hardware, providing further evidence of its potential for real-time implementation.
Is Your Load Generator Launching Web Requests in Bunches?
Venue: arXiv cs.PF 2020
Year: 2020

One problem with load test quality, almost always overlooked, is the potential for the load generator's user thread pool to sync up and dispatch queries in bunches rather than independently from each other like real users initiate their requests. A spiky launch pattern misrepresents workload flow as well as yields erroneous application response time statistics. This paper describes what a real user request timing pattern looks like, illustrates how to identify it in the load generation environment, and exercises a free downloadable tool which measures how well the load generator is mimicking the timing pattern of real web user requests.
On Exact and $\infty$-Rényi Common Informations
Venue: arXiv cs.IT 2020
Year: 2020

Recently, two extensions of Wyner's common information\textemdash exact and Rényi common informations\textemdash were introduced respectively by Kumar, Li, and El Gamal (KLE), and the present authors. The class of common information problems involves determining the minimum rate of the common input to two independent processors needed to exactly or approximately generate a target joint distribution. For the exact common information problem, exact generation of the target distribution is required, while for Wyner's and $α$-Rényi common informations, the relative entropy and Rényi divergence with order $α$ were respectively used to quantify the discrepancy between the synthesized and target distributions. The exact common information is larger than or equal to Wyner's common information. However, it was hitherto unknown whether the former is strictly larger than the latter for some joint distributions. In this paper, we first establish the equivalence between the exact and $\infty$-Rényi common informations, and then provide single-letter upper and lower bounds for these two quantities. For doubly symmetric binary sources, we show that the upper and lower bounds coincide, which implies that for such sources, the exact and $\infty$-Rényi common informations are completely characterized. Interestingly, we observe that for such sources, these two common informations are strictly larger than Wyner's. This answers an open problem posed by KLE. Furthermore, we extend Wyner's, $\infty$-Rényi, and exact common informations to sources with countably infinite or continuous alphabets, including Gaussian sources.
Enumerating minimal dominating sets in $K_t$-free graphs and variants
Venue: arXiv cs.DM 2020
Year: 2020

It is a long-standing open problem whether the minimal dominating sets of a graph can be enumerated in output-polynomial time. In this paper we investigate this problem in graph classes defined by forbidding an induced subgraph. In particular, we provide output-polynomial time algorithms for $K_t$-free graphs and variants. This answers a question of Kanté et al. about enumeration in bipartite graphs.
Time Reversal as Self-Supervision
Venue: arXiv cs.RO 2020
Year: 2020

A longstanding challenge in robot learning for manipulation tasks has been the ability to generalize to varying initial conditions, diverse objects, and changing objectives. Learning based approaches have shown promise in producing robust policies, but require heavy supervision to efficiently learn precise control, especially from visual inputs. We propose a novel self-supervision technique that uses time-reversal to learn goals and provide a high level plan to reach them. In particular, we introduce the time-reversal model (TRM), a self-supervised model which explores outward from a set of goal states and learns to predict these trajectories in reverse. This provides a high level plan towards goals, allowing us to learn complex manipulation tasks with no demonstrations or exploration at test time. We test our method on the domain of assembly, specifically the mating of tetris-style block pairs. Using our method operating atop visual model predictive control, we are able to assemble tetris blocks on a physical robot using only uncalibrated RGB camera input, and generalize to unseen block pairs. sites.google.com/view/time-reversal
Approximating the Existential Theory of the Reals
Venue: arXiv cs.CC 2020
Year: 2020

The Existential Theory of the Reals (ETR) consists of existentially quantified Boolean formulas over equalities and inequalities of polynomial functions of variables in $\mathbb{R}$. In this paper we propose and study the approximate existential theory of the reals ($ε$-ETR), in which the constraints only need to be satisfied approximately. We first show that when the domain of the variables is $\mathbb{R}$ then $ε$-ETR = ETR under polynomial time reductions, and then study the constrained $ε$-ETR problem when the variables are constrained to lie in a given bounded convex set. Our main theorem is a sampling theorem, similar to those that have been proved for approximate equilibria in normal form games. It discretizes the domain in a grid-like manner whose density depends on various properties of the formula. A consequence of our theorem is that we obtain a quasi-polynomial time approximation scheme (QPTAS) for a fragment of constrained $ε$-ETR. We use our theorem to create several new PTAS and QPTAS algorithms for problems from a variety of fields.
Deep Learning Based Caching for Self-Driving Car in Multi-access Edge Computing
Venue: arXiv cs.NI 2020
Year: 2020

Once self-driving car becomes a reality and passengers are no longer worry about it, they will need to find new ways of entertainment. However, retrieving entertainment contents at the Data Center (DC) can hinder content delivery service due to high delay of car-to-DC communication. To address these challenges, we propose a deep learning based caching for self-driving car, by using Deep Learning approaches deployed on the Multi-access Edge Computing (MEC) structure. First, at DC, Multi-Layer Perceptron (MLP) is used to predict the probabilities of contents to be requested in specific areas. To reduce the car-DC delay, MLP outputs are logged into MEC servers attached to roadside units. Second, in order to cache entertainment contents stylized for car passengers' features such as age and gender, Convolutional Neural Network (CNN) is used to predict age and gender of passengers. Third, each car requests MLP output from MEC server and compares its CNN and MLP outputs by using k-means and binary classification. Through this, the self-driving car can identify the contents need to be downloaded from the MEC server and cached. Finally, we formulate deep learning based caching in the self-driving car that enhances entertainment services as an optimization problem whose goal is to minimize content downloading delay. To solve the formulated problem, a Block Successive Majorization-Minimization (BS-MM) technique is applied. The simulation results show that the accuracy of our prediction for the contents need to be cached in the areas of the self-driving car is achieved at 98.04% and our approach can minimize delay.
Compact packings of the plane with three sizes of discs
Venue: arXiv cs.DM 2020
Year: 2020

A compact packing is a set of non-overlapping discs where all the holes between discs are curvilinear triangles. There is only one compact packing by discs of size $1$. There are exactly $9$ values of $r$ which allow a compact packing by discs of sizes $1$ and $r$. We prove here that there are exactly $164$ pairs $(r,s)$ allowing a compact packing by discs of sizes $1$, $r$ and $s$.
Corrections to "Wyner's Common Information under Rényi Divergence Measures"
Venue: arXiv cs.IT 2020
Year: 2020

In this correspondence, we correct an erroneous result on the achievability part of the Rényi common information with order $1+s\in(1,2]$ in [1]. The new achievability result (upper bound) of the Rényi common information no longer coincides with Wyner's common information. We also provide a new converse result (lower bound) in this correspondence for the Rényi common information with order $1+s\in(1,\infty]$. Numerical results show that for doubly symmetric binary sources, the new upper and lower bounds coincide for the order $1+s\in(1,2]$ and they are both strictly larger than Wyner's common information for this case.
PPO-CMA: Proximal Policy Optimization with Covariance Matrix Adaptation
Venue: arXiv cs.LG 2020
Year: 2020

Proximal Policy Optimization (PPO) is a highly popular model-free reinforcement learning (RL) approach. However, we observe that in a continuous action space, PPO can prematurely shrink the exploration variance, which leads to slow progress and may make the algorithm prone to getting stuck in local optima. Drawing inspiration from CMA-ES, a black-box evolutionary optimization method designed for robustness in similar situations, we propose PPO-CMA, a proximal policy optimization approach that adaptively expands the exploration variance to speed up progress. With only minor changes to PPO, our algorithm considerably improves performance in Roboschool continuous control benchmarks. Our results also show that PPO-CMA, as opposed to PPO, is significantly less sensitive to the choice of hyperparameters, allowing one to use it in complex movement optimization tasks without requiring tedious tuning.
Understanding Recurrent Neural Architectures by Analyzing and Synthesizing Long Distance Dependencies in Benchmark Sequential Datasets
Venue: arXiv cs.LG 2020
Year: 2020

In order to build efficient deep recurrent neural architectures, it is essential to analyze the complexityof long distance dependencies (LDDs) of the dataset being modeled. In this paper, we presentdetailed analysis of the dependency decay curve exhibited by various datasets. The datasets sampledfrom a similar process (e.g. natural language, sequential MNIST, Strictlyk-Piecewise languages,etc) display variations in the properties of the dependency decay curve. Our analysis reveal thefactors resulting in these variations; such as (i) number of unique symbols in a dataset, (ii) size ofthe dataset, (iii) number of interacting symbols within a given LDD, and (iv) the distance betweenthe interacting symbols. We test these factors by generating synthesized datasets of the Strictlyk-Piecewise languages. Another advantage of these synthesized datasets is that they enable targetedtesting of deep recurrent neural architectures in terms of their ability to model LDDs with differentcharacteristics. We also demonstrate that analysing dependency decay curves can inform the selectionof optimal hyper-parameters for SOTA deep recurrent neural architectures. This analysis can directlycontribute to the development of more accurate and efficient sequential models.