OpenRaiser/Intern-Atlas huggingface dataset:
SemTK: An Ontology-first, Open Source Semantic Toolkit for Managing and Querying Knowledge Graphs
Venue: arXiv cs.AI 2018
Year: 2018

The relatively recent adoption of Knowledge Graphs as an enabling technology in multiple high-profile artificial intelligence and cognitive applications has led to growing interest in the Semantic Web technology stack. Many semantics-related tools, however, are focused on serving experts with a deep understanding of semantic technologies. For example, triplification of relational data is available but there is no open source tool that allows a user unfamiliar with OWL/RDF to import data into a semantic triple store in an intuitive manner. Further, many tools require users to have a working understanding of SPARQL to query data. Casual users interested in benefiting from the power of Knowledge Graphs have few tools available for exploring, querying, and managing semantic data. We present SemTK, the Semantics Toolkit, a user-friendly suite of tools that allow both expert and non-expert semantics users convenient ingestion of relational data, simplified query generation, and more. The exploration of ontologies and instance data is performed through SPARQLgraph, an intuitive web-based user interface in SemTK understandable and navigable by a lay user. The open source version of SemTK is available at http://semtk.research.ge.com
Learning Depthwise Separable Graph Convolution from Data Manifold
Venue: arXiv cs.LG 2018
Year: 2018

Convolution Neural Network (CNN) has gained tremendous success in computer vision tasks with its outstanding ability to capture the local latent features. Recently, there has been an increasing interest in extending convolution operations to the non-Euclidean geometry. Although various types of convolution operations have been proposed for graphs or manifolds, their connections with traditional convolution over grid-structured data are not well-understood. In this paper, we show that depthwise separable convolution can be successfully generalized for the unification of both graph-based and grid-based convolution methods. Based on this insight we propose a novel Depthwise Separable Graph Convolution (DSGC) approach which is compatible with the tradition convolution network and subsumes existing convolution methods as special cases. It is equipped with the combined strengths in model expressiveness, compatibility (relatively small number of parameters), modularity and computational efficiency in training. Extensive experiments show the outstanding performance of DSGC in comparison with strong baselines on multi-domain benchmark datasets.
The Cost of Uncertainty in Curing Epidemics
Venue: arXiv cs.SI 2018
Year: 2018

Motivated by the study of controlling (curing) epidemics, we consider the spread of an SI process on a known graph, where we have a limited budget to use to transition infected nodes back to the susceptible state (i.e., to cure nodes). Recent work has demonstrated that under perfect and instantaneous information (which nodes are/are not infected), the budget required for curing a graph precisely depends on a combinatorial property called the CutWidth. We show that this assumption is in fact necessary: even a minor degradation of perfect information, e.g., a diagnostic test that is 99% accurate, drastically alters the landscape. Infections that could previously be cured in sublinear time now may require exponential time, or orderwise larger budget to cure. The crux of the issue comes down to a tension not present in the full information case: if a node is suspected (but not certain) to be infected, do we risk wasting our budget to try to cure an uninfected node, or increase our certainty by longer observation, at the risk that the infection spreads further? Our results present fundamental, algorithm-independent bounds that tradeoff budget required vs. uncertainty.
Efficient $\widetilde{O}(n/ε)$ Spectral Sketches for the Laplacian and its Pseudoinverse
Venue: arXiv cs.DS 2018
Year: 2018

In this paper we consider the problem of efficiently computing $ε$-sketches for the Laplacian and its pseudoinverse. Given a Laplacian and an error tolerance $ε$, we seek to construct a function $f$ such that for any vector $x$ (chosen obliviously from $f$), with high probability $(1-ε) x^\top A x \leq f(x) \leq (1 + ε) x^\top A x$ where $A$ is either the Laplacian or its pseudoinverse. Our goal is to construct such a sketch $f$ efficiently and to store it in the least space possible. We provide nearly-linear time algorithms that, when given a Laplacian matrix $\mathcal{L} \in \mathbb{R}^{n \times n}$ and an error tolerance $ε$, produce $\tilde{O}(n/ε)$-size sketches of both $\mathcal{L}$ and its pseudoinverse. Our algorithms improve upon the previous best sketch size of $\widetilde{O}(n / ε^{1.6})$ for sketching the Laplacian form by Andoni et al (2015) and $O(n / ε^2)$ for sketching the Laplacian pseudoinverse by Batson, Spielman, and Srivastava (2008). Furthermore we show how to compute all-pairs effective resistances from $\widetilde{O}(n/ε)$ size sketch in $\widetilde{O}(n^2/ε)$ time. This improves upon the previous best running time of $\widetilde{O}(n^2/ε^2)$ by Spielman and Srivastava (2008).
Enabling rootless Linux Containers in multi-user environments: the udocker tool
Venue: arXiv cs.SE 2018
Year: 2018

Containers are increasingly used as means to distribute and run Linux services and applications. In this paper we describe the architectural design and implementation of udocker, a tool which enables the user to execute Linux containers in user mode. We also present a few practical applications, using a range of scientific codes characterized by different requirements: from single core execution to MPI parallel execution and execution on GPGPUs.
New Classes of Distributed Time Complexity
Venue: arXiv cs.DC 2018
Year: 2018

A number of recent papers -- e.g. Brandt et al. (STOC 2016), Chang et al. (FOCS 2016), Ghaffari & Su (SODA 2017), Brandt et al. (PODC 2017), and Chang & Pettie (FOCS 2017) -- have advanced our understanding of one of the most fundamental questions in theory of distributed computing: what are the possible time complexity classes of LCL problems in the LOCAL model? In essence, we have a graph problem $Π$ in which a solution can be verified by checking all radius-$O(1)$ neighbourhoods, and the question is what is the smallest $T$ such that a solution can be computed so that each node chooses its own output based on its radius-$T$ neighbourhood. Here $T$ is the distributed time complexity of $Π$. The time complexity classes for deterministic algorithms in bounded-degree graphs that are known to exist by prior work are $Θ(1)$, $Θ(\log^* n)$, $Θ(\log n)$, $Θ(n^{1/k})$, and $Θ(n)$. It is also known that there are two gaps: one between $ω(1)$ and $o(\log \log^* n)$, and another between $ω(\log^* n)$ and $o(\log n)$. It has been conjectured that many more gaps exist, and that the overall time hierarchy is relatively simple -- indeed, this is known to be the case in restricted graph families such as cycles and grids. We show that the picture is much more diverse than previously expected. We present a general technique for engineering LCL problems with numerous different deterministic time complexities, including $Θ(\log^αn)$ for any $α\ge1$, $2^{Θ(\log^αn)}$ for any $α\le 1$, and $Θ(n^α)$ for any $α<1/2$ in the high end of the complexity spectrum, and $Θ(\log^α\log^* n)$ for any $α\ge 1$, $\smash{2^{Θ(\log^α\log^* n)}}$ for any $α\le 1$, and $Θ((\log^* n)^α)$ for any $α\le 1$ in the low end; here $α$ is a positive rational number.
Radical analysis network for zero-shot learning in printed Chinese character recognition
Venue: arXiv cs.CV 2018
Year: 2018

Chinese characters have a huge set of character categories, more than 20,000 and the number is still increasing as more and more novel characters continue being created. However, the enormous characters can be decomposed into a compact set of about 500 fundamental and structural radicals. This paper introduces a novel radical analysis network (RAN) to recognize printed Chinese characters by identifying radicals and analyzing two-dimensional spatial structures among them. The proposed RAN first extracts visual features from input by employing convolutional neural networks as an encoder. Then a decoder based on recurrent neural networks is employed, aiming at generating captions of Chinese characters by detecting radicals and two-dimensional structures through a spatial attention mechanism. The manner of treating a Chinese character as a composition of radicals rather than a single character class largely reduces the size of vocabulary and enables RAN to possess the ability of recognizing unseen Chinese character classes, namely zero-shot learning.
Constant-Factor Approximation for Ordered k-Median
Venue: arXiv cs.DS 2018
Year: 2018

We study the Ordered k-Median problem, in which the solution is evaluated by first sorting the client connection costs and then multiplying them with a predefined non-increasing weight vector (higher connection costs are taken with larger weights). Since the 1990s, this problem has been studied extensively in the discrete optimization and operations research communities and has emerged as a framework unifying many fundamental clustering and location problems such as k-Median and k-Center. This generality, however, renders the problem intriguing from the algorithmic perspective and obtaining non-trivial approximation algorithms was an open problem even for simple topologies such as trees. Recently, Aouad and Segev were able to obtain an O(log n) approximation algorithm for Ordered k-Median using a sophisticated local-search approach and the concept of surrogate models thereby extending the result by Tamir (2001) for the case of a rectangular weight vector, also known as k-Facility p-Centrum. In this paper, we provide an LP-rounding constant-factor approximation algorithm for the Ordered k-Median problem. We first provide a new analysis of the rounding process by Charikar and Li (2012) for k-Median, when applied to a fractional solution obtained from solving an LP relaxation over a non-metric, truncated cost vector, resulting in an elegant 15-approximation for the rectangular case. Then, we show that a simple weight bucketing can be applied to the general case resulting in O(log n) rectangles and hence in a constant-factor approximation in quasi-polynomial time. Finally, we show that also the clever distance bucketing by Aouad and Segev can be combined with the objective-oblivious version of our LP-rounding for the rectangular case, and that it results in a true, polynomial time, constant-factor approximation algorithm.
NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm
Venue: arXiv cs.NE 2018
Year: 2018

Deep neural networks (DNNs) have begun to have a pervasive impact on various applications of machine learning. However, the problem of finding an optimal DNN architecture for large applications is challenging. Common approaches go for deeper and larger DNN architectures but may incur substantial redundancy. To address these problems, we introduce a network growth algorithm that complements network pruning to learn both weights and compact DNN architectures during training. We propose a DNN synthesis tool (NeST) that combines both methods to automate the generation of compact and accurate DNNs. NeST starts with a randomly initialized sparse network called the seed architecture. It iteratively tunes the architecture with gradient-based growth and magnitude-based pruning of neurons and connections. Our experimental results show that NeST yields accurate, yet very compact DNNs, with a wide range of seed architecture selection. For the LeNet-300-100 (LeNet-5) architecture, we reduce network parameters by 70.2x (74.3x) and floating-point operations (FLOPs) by 79.4x (43.7x). For the AlexNet and VGG-16 architectures, we reduce network parameters (FLOPs) by 15.7x (4.6x) and 30.2x (8.6x), respectively. NeST's grow-and-prune paradigm delivers significant additional parameter and FLOPs reduction relative to pruning-only methods.
A Joint 3D-2D based Method for Free Space Detection on Roads
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we address the problem of road segmentation and free space detection in the context of autonomous driving. Traditional methods either use 3-dimensional (3D) cues such as point clouds obtained from LIDAR, RADAR or stereo cameras or 2-dimensional (2D) cues such as lane markings, road boundaries and object detection. Typical 3D point clouds do not have enough resolution to detect fine differences in heights such as between road and pavement. Image based 2D cues fail when encountering uneven road textures such as due to shadows, potholes, lane markings or road restoration. We propose a novel free road space detection technique combining both 2D and 3D cues. In particular, we use CNN based road segmentation from 2D images and plane/box fitting on sparse depth data obtained from SLAM as priors to formulate an energy minimization using conditional random field (CRF), for road pixels classification. While the CNN learns the road texture and is unaffected by depth boundaries, the 3D information helps in overcoming texture based classification failures. Finally, we use the obtained road segmentation with the 3D depth data from monocular SLAM to detect the free space for the navigation purposes. Our experiments on KITTI odometry dataset, Camvid dataset, as well as videos captured by us, validate the superiority of the proposed approach over the state of the art.
Sketching Linear Classifiers over Data Streams
Venue: arXiv cs.LG 2018
Year: 2018

We introduce a new sub-linear space sketch---the Weight-Median Sketch---for learning compressed linear classifiers over data streams while supporting the efficient recovery of large-magnitude weights in the model. This enables memory-limited execution of several statistical analyses over streams, including online feature selection, streaming data explanation, relative deltoid detection, and streaming estimation of pointwise mutual information. Unlike related sketches that capture the most frequently-occurring features (or items) in a data stream, the Weight-Median Sketch captures the features that are most discriminative of one stream (or class) compared to another. The Weight-Median Sketch adopts the core data structure used in the Count-Sketch, but, instead of sketching counts, it captures sketched gradient updates to the model parameters. We provide a theoretical analysis that establishes recovery guarantees for batch and online learning, and demonstrate empirical improvements in memory-accuracy trade-offs over alternative memory-budgeted methods, including count-based sketches and feature hashing.
Single Image Super-Resolution Using Lightweight CNN with Maxout Units
Venue: arXiv cs.CV 2018
Year: 2018

Rectified linear units (ReLU) are well-known to be helpful in obtaining faster convergence and thus higher performance for many deep-learning-based applications. However, networks with ReLU tend to perform poorly when the number of filter parameters is constrained to a small number. To overcome it, in this paper, we propose a novel network utilizing maxout units (MU), and show its effectiveness on super-resolution (SR) applications. In general, the MU has been known to make the filter sizes doubled in generating the feature maps of the same sizes in classification problems. In this paper, we first reveal that the MU can even make the filter sizes halved in restoration problems thus leading to compaction of the network sizes. To show this, our SR network is designed without increasing the filter sizes with MU, which outperforms the state of the art SR methods with a smaller number of filter parameters. To the best of our knowledge, we are the first to incorporate MU into SR applications and show promising performance results. In MU, feature maps from a previous convolutional layer are divided into two parts along channels, which are then compared element-wise and only their max values are passed to a next layer. Along with some interesting properties of MU to be analyzed, we further investigate other variants of MU and their effects. In addition, while ReLU have a trouble for learning in networks with a very small number of convolutional filter parameters, MU do not. For SR applications, our MU-based network reconstructs high-resolution images with comparable quality compared to previous deep-learning-based SR methods, with lower filter parameters.
Revisionist Simulations: A New Approach to Proving Space Lower Bounds
Venue: arXiv cs.DC 2018
Year: 2018

Determining the space complexity of $x$-obstruction-free $k$-set agreement for $x\leq k$ is an open problem. In $x$-obstruction-free protocols, processes are required to return in executions where at most $x$ processes take steps. The best known upper bound on the number of registers needed to solve this problem among $n>k$ processes is $n-k+x$ registers. No general lower bound better than $2$ was known. We prove that any $x$-obstruction-free protocol solving $k$-set agreement among $n>k$ processes uses at least $\lfloor(n-x)/(k+1-x)\rfloor+1$ registers. Our main tool is a simulation that serves as a reduction from the impossibility of deterministic wait-free $k$-set agreement: if a protocol uses fewer registers, then it is possible for $k+1$ processes to simulate the protocol and deterministically solve $k$-set agreement in a wait-free manner, which is impossible. A critical component of the simulation is the ability of simulating processes to revise the past of simulated processes. We introduce a new augmented snapshot object, which facilitates this. We also prove that any space lower bound on the number of registers used by obstruction-free protocols applies to protocols that satisfy nondeterministic solo termination. Hence, our lower bound of $\lfloor(n-1)/k\rfloor+1$ for the obstruction-free ($x=1$) case also holds for randomized wait-free free protocols. In particular, this gives a tight lower bound of exactly $n$ registers for solving obstruction-free and randomized wait-free consensus. Finally, our new techniques can be applied to get a space lower of $\lfloor n/2\rfloor+1$ for $ε$-approximate agreement, for sufficiently small $ε$. It requires participating processes to return values within $ε$ of each other. The best known upper bounds are $\lceil\log(1/ε)\rceil$ and $n$, while no general lower bounds were known.
End-to-end learning for music audio tagging at scale
Venue: arXiv cs.SD 2018
Year: 2018

The lack of data tends to limit the outcomes of deep learning research, particularly when dealing with end-to-end learning stacks processing raw data such as waveforms. In this study, 1.2M tracks annotated with musical labels are available to train our end-to-end models. This large amount of data allows us to unrestrictedly explore two different design paradigms for music auto-tagging: assumption-free models - using waveforms as input with very small convolutional filters; and models that rely on domain knowledge - log-mel spectrograms with a convolutional neural network designed to learn timbral and temporal features. Our work focuses on studying how these two types of deep architectures perform when datasets of variable size are available for training: the MagnaTagATune (25k songs), the Million Song Dataset (240k songs), and a private dataset of 1.2M songs. Our experiments suggest that music domain assumptions are relevant when not enough training data are available, thus showing how waveform-based models outperform spectrogram-based ones in large-scale data scenarios.
Convex Optimization with Unbounded Nonconvex Oracles using Simulated Annealing
Venue: arXiv cs.DS 2018
Year: 2018

We consider the problem of minimizing a convex objective function $F$ when one can only evaluate its noisy approximation $\hat{F}$. Unless one assumes some structure on the noise, $\hat{F}$ may be an arbitrary nonconvex function, making the task of minimizing $F$ intractable. To overcome this, prior work has often focused on the case when $F(x)-\hat{F}(x)$ is uniformly-bounded. In this paper we study the more general case when the noise has magnitude $αF(x) + β$ for some $α, β> 0$, and present a polynomial time algorithm that finds an approximate minimizer of $F$ for this noise model. Previously, Markov chains, such as the stochastic gradient Langevin dynamics, have been used to arrive at approximate solutions to these optimization problems. However, for the noise model considered in this paper, no single temperature allows such a Markov chain to both mix quickly and concentrate near the global minimizer. We bypass this by combining "simulated annealing" with the stochastic gradient Langevin dynamics, and gradually decreasing the temperature of the chain in order to approach the global minimizer. As a corollary one can approximately minimize a nonconvex function that is close to a convex function; however, the closeness can deteriorate as one moves away from the optimum.
Constructive Discrepancy Minimization with Hereditary L2 Guarantees
Venue: arXiv cs.DS 2018
Year: 2018

In discrepancy minimization problems, we are given a family of sets $\mathcal{S} = \{S_1,\dots,S_m\}$, with each $S_i \in \mathcal{S}$ a subset of some universe $U = \{u_1,\dots,u_n\}$ of $n$ elements. The goal is to find a coloring $χ: U \to \{-1,+1\}$ of the elements of $U$ such that each set $S \in \mathcal{S}$ is colored as evenly as possible. Two classic measures of discrepancy are $\ell_\infty$-discrepancy defined as $\textrm{disc}_\infty(\mathcal{S},χ):=\max_{S \in \mathcal{S}} | \sum_{u_i \in S} χ(u_i) |$ and $\ell_2$-discrepancy defined as $\textrm{disc}_2(\mathcal{S},χ):=\sqrt{(1/|\mathcal{S}|)\sum_{S \in \mathcal{S}} \left(\sum_{u_i \in S}χ(u_i)\right)^2}$. Breakthrough work by Bansal gave a polynomial time algorithm, based on rounding an SDP, for finding a coloring $χ$ such that $\textrm{disc}_\infty(\mathcal{S},χ) = O(\lg n \cdot \textrm{herdisc}_\infty(\mathcal{S}))$ where $\textrm{herdisc}_\infty(\mathcal{S})$ is the hereditary $\ell_\infty$-discrepancy of $\mathcal{S}$. We complement his work by giving a simple $O((m+n)n^2)$ time algorithm for finding a coloring $χ$ such $\textrm{disc}_2(\mathcal{S},χ) = O(\sqrt{\lg n} \cdot \textrm{herdisc}_2(\mathcal{S}))$ where $\textrm{herdisc}_2(\mathcal{S})$ is the hereditary $\ell_2$-discrepancy of $\mathcal{S}$. Interestingly, our algorithm avoids solving an SDP and instead relies on computing eigendecompositions of matrices. Moreover, we use our ideas to speed up the Edge-Walk algorithm by Lovett and Meka [SICOMP'15]. To prove that our algorithm has the claimed guarantees, we show new inequalities relating $\textrm{herdisc}_\infty$ and $\textrm{herdisc}_2$ to the eigenvalues of the matrix corresponding to $\mathcal{S}$. Our inequalities improve over previous work by Chazelle and Lvov, and by Matousek et al. Finally, we also implement our algorithm and show that it far outperforms random sampling.
Run Compressed Rank/Select for Large Alphabets
Venue: arXiv cs.DS 2018
Year: 2018

Given a string of length $n$ that is composed of $r$ runs of letters from the alphabet $\{0,1,\ldots,σ{-}1\}$ such that $2 \le σ\le r$, we describe a data structure that, provided $r \le n / \log^{ω(1)} n$, stores the string in $r\log\frac{nσ}{r} + o(r\log\frac{nσ}{r})$ bits and supports select and access queries in $O(\log\frac{\log(n/r)}{\log\log n})$ time and rank queries in $O(\log\frac{\log(nσ/r)}{\log\log n})$ time. We show that $r\log\frac{n(σ-1)}{r} - O(\log\frac{n}{r})$ bits are necessary for any such data structure and, thus, our solution is succinct. We also describe a data structure that uses $(1 + ε)r\log\frac{nσ}{r} + O(r)$ bits, where $ε> 0$ is an arbitrary constant, with the same query times but without the restriction $r \le n / \log^{ω(1)} n$. By simple reductions to the colored predecessor problem, we show that the query times are optimal in the important case $r \ge 2^{\log^δn}$, for an arbitrary constant $δ> 0$. We implement our solution and compare it with the state of the art, showing that the closest competitors consume 31-46% more space.