OpenRaiser/Intern-Atlas huggingface dataset:
Reverse Ranking by Graph Structure: Model and Scalable Algorithms
Venue: arXiv cs.SI 2016
Year: 2016

Distances in a network capture relations between nodes and are the basis of centrality, similarity, and influence measures. Often, however, the relevance of a node $u$ to a node $v$ is more precisely measured not by the magnitude of the distance, but by the number of nodes that are closer to $v$ than $u$. That is, by the {\em rank} of $u$ in an ordering of nodes by increasing distance from $v$. We identify and address fundamental challenges in rank-based graph mining. We first consider single-source computation of reverse-ranks and design a "Dijkstra-like" algorithm which computes nodes in order of increasing approximate reverse rank while only traversing edges adjacent to returned nodes. We then define {\em reverse-rank influence}, which naturally extends reverse nearest neighbors influence [Korn and Muthukrishnan 2000] and builds on a well studied distance-based influence. We present near-linear algorithms for greedy approximate reverse-rank influence maximization. The design relies on our single-source algorithm. Our algorithms utilize near-linear preprocessing of the network to compute all-distance sketches. As a contribution of independent interest, we present a novel algorithm for computing these sketches, which have many other applications, on multi-core architectures. We complement our algorithms by establishing the hardness of computing {\em exact} reverse-ranks for a single source and {\em exact} reverse-rank influence. This implies that when using near-linear algorithms, the small relative errors we obtain are the best we can currently hope for. Finally, we conduct an experimental evaluation on graphs with tens of millions of edges, demonstrating both scalability and accuracy.
Provable Bayesian Inference via Particle Mirror Descent
Venue: arXiv cs.LG 2016
Year: 2016

Bayesian methods are appealing in their flexibility in modeling complex data and ability in capturing uncertainty in parameters. However, when Bayes' rule does not result in tractable closed-form, most approximate inference algorithms lack either scalability or rigorous guarantees. To tackle this challenge, we propose a simple yet provable algorithm, \emph{Particle Mirror Descent} (PMD), to iteratively approximate the posterior density. PMD is inspired by stochastic functional mirror descent where one descends in the density space using a small batch of data points at each iteration, and by particle filtering where one uses samples to approximate a function. We prove result of the first kind that, with $m$ particles, PMD provides a posterior density estimator that converges in terms of $KL$-divergence to the true posterior in rate $O(1/\sqrt{m})$. We demonstrate competitive empirical performances of PMD compared to several approximate inference algorithms in mixture models, logistic regression, sparse Gaussian processes and latent Dirichlet allocation on large scale datasets.
On optimal nonlinear systematic codes
Venue: arXiv cs.IT 2016
Year: 2016

Most bounds on the size of codes hold for any code, whether linear or not. Notably, the Griesmer bound holds only in the linear case and so optimal linear codes are not necessarily optimal codes. In this paper we identify code parameters $(q,d,k)$, namely field size, minimum distance and dimension, for which the Griesmer bound holds also in the (systematic) nonlinear case. Moreover, we show that the Griesmer bound does not necessarily hold for a systematic code by explicit construction of a family of optimal systematic binary codes. On the other hand, we are able to provide some versions of the Griesmer bound holding for all systematic codes.
Fundamental Limits of Communication with Low Probability of Detection
Venue: arXiv cs.IT 2016
Year: 2016

This paper considers the problem of communication over a discrete memoryless channel (DMC) or an additive white Gaussian noise (AWGN) channel subject to the constraint that the probability that an adversary who observes the channel outputs can detect the communication is low. Specifically, the relative entropy between the output distributions when a codeword is transmitted and when no input is provided to the channel must be sufficiently small. For a DMC whose output distribution induced by the "off" input symbol is not a mixture of the output distributions induced by other input symbols, it is shown that the maximum amount of information that can be transmitted under this criterion scales like the square root of the blocklength. The same is true for the AWGN channel. Exact expressions for the scaling constant are also derived.
Variance Reduced Stochastic Gradient Descent with Neighbors
Venue: arXiv cs.LG 2016
Year: 2016

Stochastic Gradient Descent (SGD) is a workhorse in machine learning, yet its slow convergence can be a computational bottleneck. Variance reduction techniques such as SAG, SVRG and SAGA have been proposed to overcome this weakness, achieving linear convergence. However, these methods are either based on computations of full gradients at pivot points, or on keeping per data point corrections in memory. Therefore speed-ups relative to SGD may need a minimal number of epochs in order to materialize. This paper investigates algorithms that can exploit neighborhood structure in the training data to share and re-use information about past stochastic gradients across data points, which offers advantages in the transient optimization phase. As a side-product we provide a unified convergence analysis for a family of variance reduction algorithms, which we call memorization algorithms. We provide experimental results supporting our theory.
Optimal Online Algorithms for the Multi-Objective Time Series Search Problem
Venue: arXiv cs.DS 2016
Year: 2016

Tiedemann, et al. [Proc. of WALCOM, LNCS 8973, 2015, pp.210-221] defined multi-objective online problems (as an online version of multi-objective optimization problems) and the competitive analysis for multi-objective online problems and showed that (1) with respect to the worst component competitive analysis, the online algorithm RPP-HIGH is best possible for the multi-objective time series search~problem; (2) with respect to the arithmetic mean component competitive analysis, the online algorithm RPP-MULT is best possible for the bi-objective time series search problem; (3) with respect to the geometric mean component competitive analysis, the online algorithm RPP-MULT is best possible for the bi-objective time series search problem. In this paper, we first point out that the definitions and frameworks of the competitive analysis due to Tiedemann, et al. do not necessarily capture the efficiency of online algorithms for multi-objective online problems and provide modified definitions of the competitive analysis for multi-objective online problems. Then under the modified framework, we present a simple online algorithm Balanced Price Policy BPP_{k} for the multi-objective (k-objective) time series search problem, and show that the algorithm BPP_{k} is best possible with respect to any measure of the competitive analysis (defined by a monotone continuous function f). Under the modified framework, we derive exact values of the competitive ratio for the multi-objective time series search problem with respect to the worst component competitive analysis, the arithmetic mean component competitive analysis, and the geometric mean component competitive analysis.
Learning with Clustering Structure
Venue: arXiv cs.LG 2016
Year: 2016

We study supervised learning problems using clustering constraints to impose structure on either features or samples, seeking to help both prediction and interpretation. The problem of clustering features arises naturally in text classification for instance, to reduce dimensionality by grouping words together and identify synonyms. The sample clustering problem on the other hand, applies to multiclass problems where we are allowed to make multiple predictions and the performance of the best answer is recorded. We derive a unified optimization formulation highlighting the common structure of these problems and produce algorithms whose core iteration complexity amounts to a k-means clustering step, which can be approximated efficiently. We extend these results to combine sparsity and clustering constraints, and develop a new projection algorithm on the set of clustered sparse vectors. We prove convergence of our algorithms on random instances, based on a union of subspaces interpretation of the clustering structure. Finally, we test the robustness of our methods on artificial data sets as well as real data extracted from movie reviews.
A Decentralized Parallelization-in-Time Approach with Parareal
Venue: arXiv cs.DC 2016
Year: 2016

With steadily increasing parallelism for high-performance architectures, simulations requiring a good strong scalability are prone to be limited in scalability with standard spatial-decomposition strategies at a certain amount of parallel processors. This can be a show-stopper if the simulation results have to be computed with wallclock time restrictions (e.g.\,for weather forecasts) or as fast as possible (e.g. for urgent computing). Here, the time-dimension is the only one left for parallelization and we focus on Parareal as one particular parallelization-in-time method. We discuss a software approach for making Parareal parallelization transparent for application developers, hence allowing fast prototyping for Parareal. Further, we introduce a decentralized Parareal which results in autonomous simulation instances which only require communicating with the previous and next simulation instances, hence with strong locality for communication. This concept is evaluated by a prototypical solver for the rotational shallow-water equations which we use as a representative black-box solver.
The Fractality of Polar and Reed-Muller Codes
Venue: arXiv cs.IT 2016
Year: 2016

The generator matrices of polar codes and Reed-Muller codes are obtained by selecting rows from the Kronecker product of a lower-triangular binary square matrix. For polar codes, the selection is based on the Bhattacharyya parameter of the row, which is closely related to the error probability of the corresponding input bit under sequential decoding. For Reed-Muller codes, the selection is based on the Hamming weight of the row. This work investigates the properties of the index sets pointing to those rows in the infinite blocklength limit. In particular, the Lebesgue measure, the Hausdorff dimension, and the self-similarity of these sets will be discussed. It is shown that these index sets have several properties that are common to fractals.
A parameterized approximation algorithm for the mixed and windy Capacitated Arc Routing Problem: theory and experiments
Venue: arXiv cs.DS 2016
Year: 2016

We prove that any polynomial-time $α(n)$-approximation algorithm for the $n$-vertex metric asymmetric Traveling Salesperson Problem yields a polynomial-time $O(α(C))$-approximation algorithm for the mixed and windy Capacitated Arc Routing Problem, where $C$ is the number of weakly connected components in the subgraph induced by the positive-demand arcs---a small number in many applications. In conjunction with known results, we obtain constant-factor approximations for $C\in O(\log n)$ and $O(\log C/\log\log C)$-approximations in general. Experiments show that our algorithm, together with several heuristic enhancements, outperforms many previous polynomial-time heuristics. Finally, since the solution quality achievable in polynomial time appears to mainly depend on $C$ and since $C=1$ in almost all benchmark instances, we propose the Ob benchmark set, simulating cities that are divided into several components by a river.
Nonparametric Bayesian Double Articulation Analyzer for Direct Language Acquisition from Continuous Speech Signals
Venue: arXiv cs.AI 2016
Year: 2016

Human infants can discover words directly from unsegmented speech signals without any explicitly labeled data. In this paper, we develop a novel machine learning method called nonparametric Bayesian double articulation analyzer (NPB-DAA) that can directly acquire language and acoustic models from observed continuous speech signals. For this purpose, we propose an integrative generative model that combines a language model and an acoustic model into a single generative model called the "hierarchical Dirichlet process hidden language model" (HDP-HLM). The HDP-HLM is obtained by extending the hierarchical Dirichlet process hidden semi-Markov model (HDP-HSMM) proposed by Johnson et al. An inference procedure for the HDP-HLM is derived using the blocked Gibbs sampler originally proposed for the HDP-HSMM. This procedure enables the simultaneous and direct inference of language and acoustic models from continuous speech signals. Based on the HDP-HLM and its inference procedure, we developed a novel double articulation analyzer. By assuming HDP-HLM as a generative model of observed time series data, and by inferring latent variables of the model, the method can analyze latent double articulation structure, i.e., hierarchically organized latent words and phonemes, of the data in an unsupervised manner. The novel unsupervised double articulation analyzer is called NPB-DAA. The NPB-DAA can automatically estimate double articulation structure embedded in speech signals. We also carried out two evaluation experiments using synthetic data and actual human continuous speech signals representing Japanese vowel sequences. In the word acquisition and phoneme categorization tasks, the NPB-DAA outperformed a conventional double articulation analyzer (DAA) and baseline automatic speech recognition system whose acoustic model was trained in a supervised manner.
Bounds on the Parameters of Locally Recoverable Codes
Venue: arXiv cs.IT 2016
Year: 2016

A locally recoverable code (LRC code) is a code over a finite alphabet such that every symbol in the encoding is a function of a small number of other symbols that form a recovering set. In this paper we derive new finite-length and asymptotic bounds on the parameters of LRC codes. For LRC codes with a single recovering set for every coordinate, we derive an asymptotic Gilbert-Varshamov type bound for LRC codes and find the maximum attainable relative distance of asymptotically good LRC codes. Similar results are established for LRC codes with two disjoint recovering sets for every coordinate. For the case of multiple recovering sets we derive a lower bound on the parameters using expander graph arguments. Finally, we also derive finite-length upper bounds on the rate and distance of LRC codes with multiple recovering sets.
Flexible Multi-layer Sparse Approximations of Matrices and Applications
Venue: arXiv cs.LG 2016
Year: 2016

The computational cost of many signal processing and machine learning techniques is often dominated by the cost of applying certain linear operators to high-dimensional vectors. This paper introduces an algorithm aimed at reducing the complexity of applying linear operators in high dimension by approximately factorizing the corresponding matrix into few sparse factors. The approach relies on recent advances in non-convex optimization. It is first explained and analyzed in details and then demonstrated experimentally on various problems including dictionary learning for image denoising, and the approximation of large matrices arising in inverse problems.
Passivity-Based Adaptive Control for Visually Servoed Robotic Systems
Venue: arXiv cs.SY 2016
Year: 2016

This paper investigates the visual servoing problem for robotic systems with uncertain kinematic, dynamic, and camera parameters. We first present the passivity properties associated with the overall kinematics of the system, and then propose two passivity-based adaptive control schemes to resolve the visual tracking problem. One scheme employs the adaptive inverse-Jacobian-like feedback, and the other employs the adaptive transpose Jacobian feedback. With the Lyapunov analysis approach, it is shown that under either of the proposed control schemes, the image-space tracking errors converge to zero without relying on the assumption of the invertibility of the estimated depth. Numerical simulations are performed to show the tracking performance of the proposed adaptive controllers.
Implementation of Nonlinear Model Predictive Path-Following Control for an Industrial Robot
Venue: arXiv cs.SY 2016
Year: 2016

Many robotic applications, such as milling, gluing, or high precision measurements, require the exact following of a pre-defined geometric path. In this paper, we investigate the real-time feasible implementation of model predictive path-following control for an industrial robot. We consider constrained output path following with and without reference speed assignment. We present results from an implementation of the proposed model predictive path-following controller on a KUKA LWR IV robot.
Modeling the Mind: A brief review
Venue: arXiv cs.AI 2016
Year: 2016

The brain is a powerful tool used to achieve amazing feats. There have been several significant advances in neuroscience and artificial brain research in the past two decades. This article is a review of such advances, ranging from the concepts of connectionism, to neural network architectures and high-dimensional representations. There have also been advances in biologically inspired cognitive architectures of which we will cite a few. We will be positioning relatively specific models in a much broader perspective, while comparing and contrasting their advantages and weaknesses. The projects presented are targeted to model the brain at different levels, utilizing different methodologies.
Energy Dissipation by Metamorphic Micro-Robots in Viscous Fluids
Venue: arXiv cs.RO 2016
Year: 2016

Microscopic robots could perform tasks with high spatial precision, such as acting on precisely-targeted cells in biological tissues. Some tasks may benefit from robots that change shape, such as elongating to improve chemical gradient sensing or contracting to squeeze through narrow channels. This paper evaluates the energy dissipation for shape-changing (i.e., metamorphic) robots whose size is comparable to bacteria. Unlike larger robots, surface forces dominate the dissipation. Theoretical estimates indicate that the power likely to be available to the robots, as determined by previous studies, is sufficient to change shape fairly rapidly even in highly-viscous biological fluids. Achieving this performance will require significant improvements in manufacturing and material properties compared to current micromachines. Furthermore, optimally varying the speed of shape change only slightly reduces energy use compared to uniform speed, thereby simplifying robot controllers.
Estimating the Directed Information and Testing for Causality
Venue: arXiv cs.IT 2016
Year: 2016

The problem of estimating the directed information rate between two discrete processes $\{X_n\}$ and $\{Y_n\}$ via the plug-in (or maximum-likelihood) estimator is considered. When the joint process $\{(X_n,Y_n)\}$ is a Markov chain of a given memory length, the plug-in estimator is shown to be asymptotically Gaussian and to converge at the optimal rate $O(1/\sqrt{n})$ under appropriate conditions; this is the first estimator that has been shown to achieve this rate. An important connection is drawn between the problem of estimating the directed information rate and that of performing a hypothesis test for the presence of causal influence between the two processes. Under fairly general conditions, the null hypothesis, which corresponds to the absence of causal influence, is equivalent to the requirement that the directed information rate be equal to zero. In that case a finer result is established, showing that the plug-in converges at the faster rate $O(1/n)$ and that it is asymptotically $χ^2$-distributed. This is proved by showing that this estimator is equal to (a scalar multiple of) the classical likelihood ratio statistic for the above hypothesis test. Finally it is noted that these results facilitate the design of an actual likelihood ratio test for the presence or absence of causal influence.
Latent Belief Theory and Belief Dependencies: A Solution to the Recovery Problem in the Belief Set Theories
Venue: arXiv cs.AI 2016
Year: 2016

The AGM recovery postulate says: assume a set of propositions X; assume that it is consistent and that it is closed under logical consequences; remove a belief P from the set minimally, but make sure that the resultant set is again some set of propositions X' which is closed under the logical consequences; now add P again and close the set under the logical consequences; and we should get a set of propositions that contains all the propositions that were in X. This postulate has since met objections; many have observed that it could bear counter-intuitive results. Nevertheless, the attempts that have been made so far to amend it either recovered the postulate in full, had to relinquish the assumption of the logical closure altogether, or else had to introduce fresh controversies of their own. We provide a solution to the recovery paradox in this work. Our theoretical basis is the recently proposed belief theory with latent beliefs (simply the latent belief theory for short). Firstly, through examples, we will illustrate that the vanilla latent belief theory can be made more expressive. We will identify that a latent belief, when it becomes visible, may remain visible only while the beliefs that triggered it into the agent's consciousness are in the agent's belief set. In order that such situations can be also handled, we will enrich the latent belief theory with belief dependencies among attributive beliefs, recording the information as to which belief is supported of its existence by which beliefs. We will show that the enriched latent belief theory does not possess the recovery property. The closure by logical consequences is maintained in the theory, however. Hence it serves as a solution to the open problem in the belief set theories.
Learning Leading Indicators for Time Series Predictions
Venue: arXiv cs.LG 2016
Year: 2016

We consider the problem of learning models for forecasting multiple time-series systems together with discovering the leading indicators that serve as good predictors for the system. We model the systems by linear vector autoregressive models (VAR) and link the discovery of leading indicators to inferring sparse graphs of Granger-causality. We propose new problem formulations and develop two new methods to learn such models, gradually increasing the complexity of assumptions and approaches. While the first method assumes common structures across the whole system, our second method uncovers model clusters based on the Granger-causality and leading indicators together with learning the model parameters. We study the performance of our methods on a comprehensive set of experiments and confirm their efficacy and their advantages over state-of-the-art sparse VAR and graphical Granger learning methods.
A review of the literature on citation impact indicators
Venue: arXiv cs.DL 2016
Year: 2016

Citation impact indicators nowadays play an important role in research evaluation, and consequently these indicators have received a lot of attention in the bibliometric and scientometric literature. This paper provides an in-depth review of the literature on citation impact indicators. First, an overview is given of the literature on bibliographic databases that can be used to calculate citation impact indicators (Web of Science, Scopus, and Google Scholar). Next, selected topics in the literature on citation impact indicators are reviewed in detail. The first topic is the selection of publications and citations to be included in the calculation of citation impact indicators. The second topic is the normalization of citation impact indicators, in particular normalization for field differences. Counting methods for dealing with co-authored publications are the third topic, and citation impact indicators for journals are the last topic. The paper concludes by offering some recommendations for future research.