OpenRaiser/Intern-Atlas huggingface dataset:
The scarcity of crossing dependencies: a direct outcome of a specific constraint?
Venue: arXiv cs.CL 2017
Year: 2017

The structure of a sentence can be represented as a network where vertices are words and edges indicate syntactic dependencies. Interestingly, crossing syntactic dependencies have been observed to be infrequent in human languages. This leads to the question of whether the scarcity of crossings in languages arises from an independent and specific constraint on crossings. We provide statistical evidence suggesting that this is not the case, as the proportion of dependency crossings of sentences from a wide range of languages can be accurately estimated by a simple predictor based on a null hypothesis on the local probability that two dependencies cross given their lengths. The relative error of this predictor never exceeds 5% on average, whereas the error of a baseline predictor assuming a random ordering of the words of a sentence is at least 6 times greater. Our results suggest that the low frequency of crossings in natural languages is neither originated by hidden knowledge of language nor by the undesirability of crossings per se, but as a mere side effect of the principle of dependency length minimization.
Analysis of Algorithms and Partial Algorithms
Venue: arXiv cs.AI 2017
Year: 2017

We present an alternative methodology for the analysis of algorithms, based on the concept of expected discounted reward. This methodology naturally handles algorithms that do not always terminate, so it can (theoretically) be used with partial algorithms for undecidable problems, such as those found in artificial general intelligence (AGI) and automated theorem proving. We mention an approach to self-improving AGI enabled by this methodology. Aug 2017 addendum: This article was originally written with multiple audiences in mind. It is really best put in the following terms. Goertzel, Hutter, Legg, and others have developed a definition of an intelligence score for a general abstract agent: expected lifetime reward in a random environment. AIXI is generally the optimal agent according to this score, but there may be reasons to analyze other agents and compare score values. If we want to use this definition of intelligence in practice, perhaps we can start by analyzing some simple agents. Common algorithms can be thought of as simple agents (environment is input, reward is based on running time) so we take the goal of applying the agent intelligence score to algorithms. That is, we want to find, what are the IQ scores of algorithms? We can do some very simple analysis, but the real answer is that even for simple algorithms, the intelligence score is too difficult to work with in practice.
On the complexity of bounded time and precision reachability for piecewise affine systems
Venue: arXiv cs.CC 2017
Year: 2017

Reachability for piecewise affine systems is known to be undecidable, starting from dimension $2$. In this paper we investigate the exact complexity of several decidable variants of reachability and control questions for piecewise affine systems. We show in particular that the region to region bounded time versions leads to $NP$-complete or co-$NP$-complete problems, starting from dimension $2$. We also prove that a bounded precision version leads to $PSPACE$-complete problems.
Adaptive CSMA under the SINR Model: Efficient Approximation Algorithms for Throughput and Utility Maximization
Venue: arXiv cs.IT 2017
Year: 2017

We consider a Carrier Sense Multiple Access (CSMA) based scheduling algorithm for a single-hop wireless network under a realistic Signal-to-interference-plus-noise ratio (SINR) model for the interference. We propose two local optimization based approximation algorithms to efficiently estimate certain attempt rate parameters of CSMA called fugacities. It is known that adaptive CSMA can achieve throughput optimality by sampling feasible schedules from a Gibbs distribution, with appropriate fugacities. Unfortunately, obtaining these optimal fugacities is an NP-hard problem. Further, the existing adaptive CSMA algorithms use a stochastic gradient descent based method, which usually entails an impractically slow (exponential in the size of the network) convergence to the optimal fugacities. To address this issue, we first propose an algorithm to estimate the fugacities, that can support a given set of desired service rates. The convergence rate and the complexity of this algorithm are independent of the network size, and depend only on the neighborhood size of a link. Further, we show that the proposed algorithm corresponds exactly to performing the well-known Bethe approximation to the underlying Gibbs distribution. Then, we propose another local algorithm to estimate the optimal fugacities under a utility maximization framework, and characterize its accuracy. Numerical results indicate that the proposed methods have a good degree of accuracy, and achieve extremely fast convergence to near-optimal fugacities, and often outperform the convergence rate of the stochastic gradient descent by a few orders of magnitude.
Think before you collect: Setting up a data collection approach for social media studies
Venue: arXiv cs.CY 2017
Year: 2017

This chapter discusses important challenges of designing the data collection setup for social media studies. It outlines how it is necessary to carefully think about which data to collect and to use, and to recognize the effects that a specific data collection approach may have on the types of analyses that can be carried out and the results that can be expected in a study. We will highlight important questions one should ask before setting up a data collection framework and relate them to the different options for accessing social media data. The chapter will mainly be illustrated with examples from studying Twitter and Facebook. A case study studying political communication around the 2013 elections in Germany should serve as a practical application scenario. In this case study we constructed several social media datasets based on different collection approaches, using data from Facebook and Twitter.
A Generalised Quantifier Theory of Natural Language in Categorical Compositional Distributional Semantics with Bialgebras
Venue: arXiv cs.CL 2017
Year: 2017

Categorical compositional distributional semantics is a model of natural language; it combines the statistical vector space models of words with the compositional models of grammar. We formalise in this model the generalised quantifier theory of natural language, due to Barwise and Cooper. The underlying setting is a compact closed category with bialgebras. We start from a generative grammar formalisation and develop an abstract categorical compositional semantics for it, then instantiate the abstract setting to sets and relations and to finite dimensional vector spaces and linear maps. We prove the equivalence of the relational instantiation to the truth theoretic semantics of generalised quantifiers. The vector space instantiation formalises the statistical usages of words and enables us to, for the first time, reason about quantified phrases and sentences compositionally in distributional semantics.
Variance-Reduced and Projection-Free Stochastic Optimization
Venue: arXiv cs.LG 2017
Year: 2017

The Frank-Wolfe optimization algorithm has recently regained popularity for machine learning applications due to its projection-free property and its ability to handle structured constraints. However, in the stochastic learning setting, it is still relatively understudied compared to the gradient descent counterpart. In this work, leveraging a recent variance reduction technique, we propose two stochastic Frank-Wolfe variants which substantially improve previous results in terms of the number of stochastic gradient evaluations needed to achieve $1-ε$ accuracy. For example, we improve from $O(\frac{1}ε)$ to $O(\ln\frac{1}ε)$ if the objective function is smooth and strongly convex, and from $O(\frac{1}{ε^2})$ to $O(\frac{1}{ε^{1.5}})$ if the objective function is smooth and Lipschitz. The theoretical improvement is also observed in experiments on real-world datasets for a multiclass classification application.
Requirement verification in simulation-based automation testing
Venue: arXiv cs.SE 2017
Year: 2017

The emergence of the Industrial Internet results in an increasing number of complicated temporal interdependencies between automation systems and the processes to be controlled. There is a need for verification methods that scale better than formal verification methods and which are more exact than testing. Simulation-based runtime verification is proposed as such a method, and an application of Metric temporal logic is presented as a contribution. The practical scalability of the proposed approach is validated against a production process designed by an industrial partner, resulting in the discovery of requirement violations.
Refined Multiscale Fuzzy Entropy based on Standard Deviation for Biomedical Signal Analysis
Venue: arXiv cs.CE 2017
Year: 2017

Multiscale entropy (MSE) has been a prevalent algorithm to quantify the complexity of fluctuations in the local mean value of biomedical time series. Recent developments in the field have tried to improve the MSE by reducing its variability in large scale factors. On the other hand, there has been recent interest in using other statistical moments than the mean, i.e. variance, in the coarse-graining step of the MSE. Building on these trends, here we introduce the so-called refined composite multiscale fuzzy entropy based on the standard deviation (RCMFEσ) to quantify the dynamical properties of spread over multiple time scales. We demonstrate the dependency of the RCMFEσ, in comparison with other multiscale approaches, on several straightforward signal processing concepts using a set of synthetic signals. We also investigate the complementarity of using the standard deviation instead of the mean in the coarse-graining process using magnetoencephalograms in Alzheimer disease and publicly available electroencephalograms recorded from focal and non-focal areas in epilepsy. Our results indicate that RCMFEσ offers complementary information to that revealed by classical coarse-graining approaches and that it has superior performance to distinguish different types of physiological activity.
Time and Frequency Domain Investigation of Selected Memristor based Analog Circuits
Venue: arXiv cs.ET 2017
Year: 2017

In this paper, we investigate few memristor-based analog circuits namely the phase shift oscillator, integrator, and differentiator which have been explored numerously using the traditional lumped components. We use LTspice-IV platform for simulation of the above-said circuits. The investigation resorts to the nonlinear dopant drift model of memristor and the window function portrayed in the literature for nonlinearity realization. The results of our investigations depict good agreement with the conventional lumped component based phase shift oscillator, integrator, and differentiator circuits. The results are evident to showcase the potential of the memristor as a promising candidate for the next generation analog circuits.
High Dimensional Inference with Random Maximum A-Posteriori Perturbations
Venue: arXiv cs.LG 2017
Year: 2017

This paper presents a new approach, called perturb-max, for high-dimensional statistical inference that is based on applying random perturbations followed by optimization. This framework injects randomness to maximum a-posteriori (MAP) predictors by randomly perturbing the potential function for the input. A classic result from extreme value statistics asserts that perturb-max operations generate unbiased samples from the Gibbs distribution using high-dimensional perturbations. Unfortunately, the computational cost of generating so many high-dimensional random variables can be prohibitive. However, when the perturbations are of low dimension, sampling the perturb-max prediction is as efficient as MAP optimization. This paper shows that the expected value of perturb-max inference with low dimensional perturbations can be used sequentially to generate unbiased samples from the Gibbs distribution. Furthermore the expected value of the maximal perturbations is a natural bound on the entropy of such perturb-max models. A measure concentration result for perturb-max values shows that the deviation of their sampled average from its expectation decays exponentially in the number of samples, allowing effective approximation of the expectation.
Network of Bandits insure Privacy of end-users
Venue: arXiv cs.AI 2017
Year: 2017

In order to distribute the best arm identification task as close as possible to the user's devices, on the edge of the Radio Access Network, we propose a new problem setting, where distributed players collaborate to find the best arm. This architecture guarantees privacy to end-users since no events are stored. The only thing that can be observed by an adversary through the core network is aggregated information across users. We provide a first algorithm, Distributed Median Elimination, which is optimal in term of number of transmitted bits and near optimal in term of speed-up factor with respect to an optimal algorithm run independently on each player. In practice, this first algorithm cannot handle the trade-off between the communication cost and the speed-up factor, and requires some knowledge about the distribution of players. Extended Distributed Median Elimination overcomes these limitations, by playing in parallel different instances of Distributed Median Elimination and selecting the best one. Experiments illustrate and complete the analysis. According to the analysis, in comparison to Median Elimination performed on each player, the proposed algorithm shows significant practical improvements.
On Metric Multi-Covering Problems
Venue: arXiv cs.CG 2017
Year: 2017

In the metric multi-cover problem (MMC), we are given two point sets $Y$ (servers) and $X$ (clients) in an arbitrary metric space $(X \cup Y, d)$, a positive integer $k$ that represents the coverage demand of each client, and a constant $α\geq 1$. Each server can have a single ball of arbitrary radius centered on it. Each client $x \in X$ needs to be covered by at least $k$ such balls centered on servers. The objective function that we wish to minimize is the sum of the $α$-th powers of the radii of the balls. In this article, we consider the MMC problem as well as some non-trivial generalizations, such as (a) the non-uniform MMC, where we allow client-specific demands, and (b) the $t$-MMC, where we require the number of open servers to be at most some given integer $t$. For each of these problems, we present an efficient algorithm that reduces the problem to several instances of the corresponding $1$-covering problem, where the coverage demand of each client is $1$. Our reductions preserve optimality up to a multiplicative constant factor. Applying known constant factor approximation algorithms for $1$-covering, we obtain the first constant approximations for the MMC and these generalizations.
Unsupervised Domain Adaptation with Residual Transfer Networks
Venue: arXiv cs.LG 2017
Year: 2017

The recent success of deep neural networks relies on massive amounts of labeled data. For a target task where labeled data is unavailable, domain adaptation can transfer a learner from a different source domain. In this paper, we propose a new approach to domain adaptation in deep networks that can jointly learn adaptive classifiers and transferable features from labeled data in the source domain and unlabeled data in the target domain. We relax a shared-classifier assumption made by previous methods and assume that the source classifier and target classifier differ by a residual function. We enable classifier adaptation by plugging several layers into deep network to explicitly learn the residual function with reference to the target classifier. We fuse features of multiple layers with tensor product and embed them into reproducing kernel Hilbert spaces to match distributions for feature adaptation. The adaptation can be achieved in most feed-forward models by extending them with new residual layers and loss functions, which can be trained efficiently via back-propagation. Empirical evidence shows that the new approach outperforms state of the art methods on standard domain adaptation benchmarks.
The Independence of Markov's Principle in Type Theory
Venue: arXiv cs.LO 2017
Year: 2017

In this paper, we show that Markov's principle is not derivable in dependent type theory with natural numbers and one universe. One way to prove this would be to remark that Markov's principle does not hold in a sheaf model of type theory over Cantor space, since Markov's principle does not hold for the generic point of this model. Instead we design an extension of type theory, which intuitively extends type theory by the addition of a generic point of Cantor space. We then show the consistency of this extension by a normalization argument. Markov's principle does not hold in this extension, and it follows that it cannot be proved in type theory.
Unsupervised Domain Adaptation Using Approximate Label Matching
Venue: arXiv cs.LG 2017
Year: 2017

Domain adaptation addresses the problem created when training data is generated by a so-called source distribution, but test data is generated by a significantly different target distribution. In this work, we present approximate label matching (ALM), a new unsupervised domain adaptation technique that creates and leverages a rough labeling on the test samples, then uses these noisy labels to learn a transformation that aligns the source and target samples. We show that the transformation estimated by ALM has favorable properties compared to transformations estimated by other methods, which do not use any kind of target labeling. Our model is regularized by requiring that a classifier trained to discriminate source from transformed target samples cannot distinguish between the two. We experiment with ALM on simulated and real data, and show that it outperforms techniques commonly used in the field.
Branching Bisimilarity of Normed BPA Processes as a Rational Monoid
Venue: arXiv cs.LO 2017
Year: 2017

The paper presents an elaborated and simplified version of the structural result for branching bisimilarity on normed BPA (Basic Process Algebra) processes that was the crux of a conference paper by Czerwinski and Jancar (arxiv 7/2014 and LiCS 2015). That paper focused on the computational complexity, and a NEXPTIME-upper bound has been derived; the authors built on the ideas by Fu (ICALP 2013), and strengthened his decidability result. Later He and Huang announced the EXPTIME-completeness of this problem (arxiv 1/2015, and LiCS 2015), giving a technical proof for the EXPTIME membership. He and Huang indirectly acknowledge the decomposition ideas by Czerwinski and Jancar on which they also built, but it is difficult to separate their starting point from their new ideas. One aim here is to present the previous decomposition result of Czerwinski and Jancar in a technically new framework, noting that branching bisimulation equivalence on normed BPA processes corresponds to a rational monoid (in the sense of [Sakarovitch, 1987]); in particular it is shown that the mentioned equivalence can be decided by normal-form computing deterministic finite transducers. Another aim is to provide a complete description, including an informal overview, that should also make clear how Fu's ideas were used, and to give all proofs in a form that should be readable and easily verifiable.
FPTAS for Mixed-Strategy Nash Equilibria in Tree Graphical Games and Their Generalizations
Venue: arXiv cs.GT 2017
Year: 2017

We provide the first fully polynomial time approximation scheme (FPTAS) for computing an approximate mixed-strategy Nash equilibrium in tree-structured graphical multi-hypermatrix games (GMhGs). GMhGs are generalizations of normal-form games, graphical games, graphical polymatrix games, and hypergraphical games. Computing an exact mixed-strategy Nash equilibria in graphical polymatrix games is PPAD-complete and thus generally believed to be intractable. In contrast, to the best of our knowledge, we are the first to establish an FPTAS for tree polymatrix games as well as tree graphical games when the number of actions is bounded by a constant. As a corollary, we give a quasi-polynomial time approximation scheme (quasi-PTAS) when the number of actions is bounded by the logarithm of the number of players.
Simple average-case lower bounds for approximate near-neighbor from isoperimetric inequalities
Venue: arXiv cs.DS 2017
Year: 2017

We prove an $Ω(d/\log \frac{sw}{nd})$ lower bound for the average-case cell-probe complexity of deterministic or Las Vegas randomized algorithms solving approximate near-neighbor (ANN) problem in $d$-dimensional Hamming space in the cell-probe model with $w$-bit cells, using a table of size $s$. This lower bound matches the highest known worst-case cell-probe lower bounds for any static data structure problems. This average-case cell-probe lower bound is proved in a general framework which relates the cell-probe complexity of ANN to isoperimetric inequalities in the underlying metric space. A tighter connection between ANN lower bounds and isoperimetric inequalities is established by a stronger richness lemma proved by cell-sampling techniques.
A landmark-based algorithm for automatic pattern recognition and abnormality detection
Venue: arXiv cs.CV 2017
Year: 2017

We study a class of mathematical and statistical algorithms with the aim of establishing a computer-based framework for fast and reliable automatic abnormality detection on landmark represented image templates. Under this framework, we apply a landmark-based algorithm for finding a group average as an estimator that is said to best represent the common features of the group in study. This algorithm extracts information of momentum at each landmark through the process of template matching. If ever converges, the proposed algorithm produces a local coordinate system for each member of the observing group, in terms of the residual momentum. We use a Bayesian approach on the collected residual momentum representations for making inference. For illustration, we apply this framework to a small database of brain images for detecting structure abnormality. The brain structure changes identified by our framework are highly consistent with studies in the literature.
Eilenberg Theorems for Free
Venue: arXiv cs.FL 2017
Year: 2017

Eilenberg-type correspondences, relating varieties of languages (e.g. of finite words, infinite words, or trees) to pseudovarieties of finite algebras, form the backbone of algebraic language theory. Numerous such correspondences are known in the literature. We demonstrate that they all arise from the same recipe: one models languages and the algebras recognizing them by monads on an algebraic category, and applies a Stone-type duality. Our main contribution is a variety theorem that covers e.g. Wilke's and Pin's work on $\infty$-languages, the variety theorem for cost functions of Daviaud, Kuperberg, and Pin, and unifies the two previous categorical approaches of Bojańczyk and of Adámek et al. In addition we derive a number of new results, including an extension of the local variety theorem of Gehrke, Grigorieff, and Pin from finite to infinite words.