OpenRaiser/Intern-Atlas huggingface dataset:
Unsupervised Pretraining Encourages Moderate-Sparseness
Venue: arXiv cs.LG 2014
Year: 2014

It is well known that direct training of deep neural networks will generally lead to poor results. A major progress in recent years is the invention of various pretraining methods to initialize network parameters and it was shown that such methods lead to good prediction performance. However, the reason for the success of pretraining has not been fully understood, although it was argued that regularization and better optimization play certain roles. This paper provides another explanation for the effectiveness of pretraining, where we show pretraining leads to a sparseness of hidden unit activation in the resulting neural networks. The main reason is that the pretraining models can be interpreted as an adaptive sparse coding. Compared to deep neural network with sigmoid function, our experimental results on MNIST and Birdsong further support this sparseness observation.
On the dynamic width of the 3-colorability problem
Venue: arXiv cs.CC 2014
Year: 2014

A graph $G$ is 3-colorable if and only if it maps homomorphically to the complete 3-vertex graph $K_3$. The last condition can be checked by a $k$-consistency algorithm where the parameter $k$ has to be chosen large enough, dependent on $G$. Let $W(G)$ denote the minimum $k$ sufficient for this purpose. For a non-3-colorable graph $G$, $W(G)$ is equal to the minimum $k$ such that $G$ can be distinguished from $K_3$ in the $k$-variable existential-positive first-order logic. We define the dynamic width of the 3-colorability problem as the function $W(n)=\max_G W(G)$, where the maximum is taken over all non-3-colorable $G$ with $n$ vertices. The assumption $\mathrm{NP}\ne\mathrm{P}$ implies that $W(n)$ is unbounded. Indeed, a lower bound $W(n)=Ω(\log\log n/\log\log\log n)$ follows unconditionally from the work of Nesetril and Zhu on bounded treewidth duality. The Exponential Time Hypothesis implies a much stronger bound $W(n)=Ω(n/\log n)$ and indeed we unconditionally prove that $W(n)=Ω(n)$. In fact, an even stronger statement is true: A first-order sentence distinguishing any 3-colorable graph on $n$ vertices from any non-3-colorable graph on $n$ vertices must have $Ω(n)$ variables. On the other hand, we observe that $W(G)\le 3\,α(G)+1$ and $W(G)\le n-α(G)+1$ for every non-3-colorable graph $G$ with $n$ vertices, where $α(G)$ denotes the independence number of $G$. This implies that $W(n)\le\frac34\,n+1$, improving on the trivial upper bound $W(n)\le n$. We also show that $W(G)>\frac1{16}\, g(G)$ for every non-3-colorable graph $G$, where $g(G)$ denotes the girth of $G$. Finally, we consider the function $W(n)$ over planar graphs and prove that $W(n)=Θ(\sqrt n)$ in the case.
Multilingual Distributed Representations without Word Alignment
Venue: arXiv cs.CL 2014
Year: 2014

Distributed representations of meaning are a natural way to encode covariance relationships between words and phrases in NLP. By overcoming data sparsity problems, as well as providing information about semantic relatedness which is not available in discrete representations, distributed representations have proven useful in many NLP tasks. Recent work has shown how compositional semantic representations can successfully be applied to a number of monolingual applications such as sentiment analysis. At the same time, there has been some initial success in work on learning shared word-level representations across languages. We combine these two approaches by proposing a method for learning distributed representations in a multilingual setup. Our model learns to assign similar embeddings to aligned sentences and dissimilar ones to sentence which are not aligned while not requiring word alignments. We show that our representations are semantically informative and apply them to a cross-lingual document classification task where we outperform the previous state of the art. Further, by employing parallel corpora of multiple language pairs we find that our model learns representations that capture semantic relationships across languages for which no parallel data was used.
Sensor management for multi-target tracking via Multi-Bernoulli filtering
Venue: arXiv cs.SY 2014
Year: 2014

In multi-object stochastic systems, the issue of sensor management is a theoretically and computationally challenging problem. In this paper, we present a novel random finite set (RFS) approach to the multi-target sensor management problem within the partially observed Markov decision process (POMDP) framework. The multi-target state is modelled as a multi-Bernoulli RFS, and the multi-Bernoulli filter is used in conjunction with two different control objectives: maximizing the expected Rényi divergence between the predicted and updated densities, and minimizing the expected posterior cardinality variance. Numerical studies are presented in two scenarios where a mobile sensor tracks five moving targets with different levels of observability.
Generating Matrix Identities and Proof Complexity
Venue: arXiv cs.CC 2014
Year: 2014

Motivated by the fundamental lower bounds questions in proof complexity, we initiate the study of matrix identities as hard instances for strong proof systems. A matrix identity of $d \times d$ matrices over a field $\mathbb{F}$, is a non-commutative polynomial $f(x_1,\ldots,x_n)$ over $\mathbb{F}$ such that $f$ vanishes on every $d \times d$ matrix assignment to its variables. We focus on arithmetic proofs, which are proofs of polynomial identities operating with arithmetic circuits and whose axioms are the polynomial-ring axioms (these proofs serve as an algebraic analogue of the Extended Frege propositional proof system; and over $GF(2)$ they constitute formally a sub-system of Extended Frege [HT12]). We introduce a decreasing in strength hierarchy of proof systems within arithmetic proofs, in which the $d$th level is a sound and complete proof system for proving $d \times d$ matrix identities (over a given field). For each level $d>2$ in the hierarchy, we establish a proof-size lower bound in terms of the number of variables in the matrix identity proved: we show the existence of a family of matrix identities $f_n$ with $n$ variables, such that any proof of $f_n=0$ requires $Ω(n^{2d})$ number of lines. The lower bound argument uses fundamental results from the theory of algebras with polynomial identities together with a generalization of the arguments in [Hru11]. We then set out to study matrix identities as hard instances for (full) arithmetic proofs. We present two conjectures, one about non-commutative arithmetic circuit complexity and the other about proof complexity, under which up to exponential-size lower bounds on arithmetic proofs (in terms of the arithmetic circuit size of the identities proved) hold. Finally, we discuss the applicability of our approach to strong propositional proof systems such as Extended Frege.
The Complexity of Fairness through Equilibrium
Venue: arXiv cs.GT 2014
Year: 2014

Competitive equilibrium with equal incomes (CEEI) is a well known fair allocation mechanism; however, for indivisible resources a CEEI may not exist. It was shown in [Budish '11] that in the case of indivisible resources there is always an allocation, called A-CEEI, that is approximately fair, approximately truthful, and approximately efficient, for some favorable approximation parameters. This approximation is used in practice to assign students to classes. In this paper we show that finding the A-CEEI allocation guaranteed to exist by Budish's theorem is PPAD-complete. We further show that finding an approximate equilibrium with better approximation guarantees is even harder: NP-complete.
Global semantic typing for inductive and coinductive computing
Venue: arXiv cs.LO 2014
Year: 2014

Inductive and coinductive types are commonly construed as ontological (Church-style) types, denoting canonical data-sets such as natural numbers, lists, and streams. For various purposes, notably the study of programs in the context of global semantics, it is preferable to think of types as semantical properties (Curry-style). Intrinsic theories were introduced in the late 1990s to provide a purely logical framework for reasoning about programs and their semantic types. We extend them here to data given by any combination of inductive and coinductive definitions. This approach is of interest because it fits tightly with syntactic, semantic, and proof theoretic fundamentals of formal logic, with potential applications in implicit computational complexity as well as extraction of programs from proofs. We prove a Canonicity Theorem, showing that the global definition of program typing, via the usual (Tarskian) semantics of first-order logic, agrees with their operational semantics in the intended model. Finally, we show that every intrinsic theory is interpretable in a conservative extension of first-order arithmetic. This means that quantification over infinite data objects does not lead, on its own, to proof-theoretic strength beyond that of Peano Arithmetic. Intrinsic theories are perfectly amenable to formulas-as-types Curry-Howard morphisms, and were used to characterize major computational complexity classes Their extensions described here have similar potential which has already been applied.
On the family of $r$-regular graphs with Grundy number $r+1$
Venue: arXiv cs.DM 2014
Year: 2014

The Grundy number of a graph $G$, denoted by $Γ(G)$, is the largest $k$ such that there exists a partition of $V(G)$, into $k$ independent sets $V_1,\ldots, V_k$ and every vertex of $V_i$ is adjacent to at least one vertex in $V_j$, for every $j < i$. The objects which are studied in this article are families of $r$-regular graphs such that $Γ(G) = r + 1$. Using the notion of independent module, a characterization of this family is given for $r=3$. Moreover, we determine classes of graphs in this family, in particular the class of $r$-regular graphs without induced $C_4$, for $r \le 4$. Furthermore, our propositions imply results on partial Grundy number.
Explicit linear kernels via dynamic programming
Venue: arXiv cs.DS 2014
Year: 2014

Several algorithmic meta-theorems on kernelization have appeared in the last years, starting with the result of Bodlaender et al. [FOCS 2009] on graphs of bounded genus, then generalized by Fomin et al. [SODA 2010] to graphs excluding a fixed minor, and by Kim et al. [ICALP 2013] to graphs excluding a fixed topological minor. Typically, these results guarantee the existence of linear or polynomial kernels on sparse graph classes for problems satisfying some generic conditions but, mainly due to their generality, it is not clear how to derive from them constructive kernels with explicit constants. In this paper we make a step toward a fully constructive meta-kernelization theory on sparse graphs. Our approach is based on a more explicit protrusion replacement machinery that, instead of expressibility in CMSO logic, uses dynamic programming, which allows us to find an explicit upper bound on the size of the derived kernels. We demonstrate the usefulness of our techniques by providing the first explicit linear kernels for $r$-Dominating Set and $r$-Scattered Set on apex-minor-free graphs, and for Planar-\mathcal{F}-Deletion on graphs excluding a fixed (topological) minor in the case where all the graphs in \mathcal{F} are connected.
Co-Multistage of Multiple Classifiers for Imbalanced Multiclass Learning
Venue: arXiv cs.LG 2014
Year: 2014

In this work, we propose two stochastic architectural models (CMC and CMC-M) with two layers of classifiers applicable to datasets with one and multiple skewed classes. This distinction becomes important when the datasets have a large number of classes. Therefore, we present a novel solution to imbalanced multiclass learning with several skewed majority classes, which improves minority classes identification. This fact is particularly important for text classification tasks, such as event detection. Our models combined with pre-processing sampling techniques improved the classification results on six well-known datasets. Finally, we have also introduced a new metric SG-Mean to overcome the multiplication by zero limitation of G-Mean.
Transparent Checkpoint-Restart for Hardware-Accelerated 3D Graphics
Venue: arXiv cs.OS 2014
Year: 2014

Providing fault-tolerance for long-running GPU-intensive jobs requires application-specific solutions, and often involves saving the state of complex data structures spread among many graphics libraries. This work describes a mechanism for transparent GPU-independent checkpoint-restart of 3D graphics. The approach is based on a record-prune-replay paradigm: all OpenGL calls relevant to the graphics driver state are recorded; calls not relevant to the internal driver state as of the last graphics frame prior to checkpoint are discarded; and the remaining calls are replayed on restart. A previous approach for OpenGL 1.5, based on a shadow device driver, required more than 78,000 lines of OpenGL-specific code. In contrast, the new approach, based on record-prune-replay, is used to implement the same case in just 4,500 lines of code. The speed of this approach varies between 80 per cent and nearly 100 per cent of the speed of the native hardware acceleration for OpenGL 1.5, as measured when running the ioquake3 game under Linux. This approach has also been extended to demonstrate checkpointing of OpenGL 3.0 for the first time, with a demonstration for PyMol, for molecular visualization.
Data Mining on Social Interaction Networks
Venue: arXiv cs.SI 2014
Year: 2014

Social media and social networks have already woven themselves into the very fabric of everyday life. This results in a dramatic increase of social data capturing various relations between the users and their associated artifacts, both in online networks and the real world using ubiquitous devices. In this work, we consider social interaction networks from a data mining perspective - also with a special focus on real-world face-to-face contact networks: We combine data mining and social network analysis techniques for examining the networks in order to improve our understanding of the data, the modeled behavior, and its underlying emergent processes. Furthermore, we adapt, extend and apply known predictive data mining algorithms on social interaction networks. Additionally, we present novel methods for descriptive data mining for uncovering and extracting relations and patterns for hypothesis generation and exploration, in order to provide characteristic information about the data and networks. The presented approaches and methods aim at extracting valuable knowledge for enhancing the understanding of the respective data, and for supporting the users of the respective systems. We consider data from several social systems, like the social bookmarking system BibSonomy, the social resource sharing system flickr, and ubiquitous social systems: Specifically, we focus on data from the social conference guidance system Conferator and the social group interaction system MyGroup. This work first gives a short introduction into social interaction networks, before we describe several analysis results in the context of online social networks and real-world face-to-face contact networks. Next, we present predictive data mining methods, i.e., for localization, recommendation and link prediction. After that, we present novel descriptive data mining methods for mining communities and patterns.
QoS-Aware User Association for Load Balancing in Heterogeneous Cellular Networks
Venue: arXiv cs.IT 2014
Year: 2014

To solve the problem that the low capacity in hot-spots and coverage holes of conventional cellular networks, the base stations (BSs) having lower transmit power are deployed to form heterogeneous cellular networks (HetNets). However, because of these introduced disparate power BSs, the user distributions among them looked fairly unbalanced if an appropriate user association scheme hasn't been provided. For effectively tackling this problem, we jointly consider the load of each BS and user's achievable rate instead of only utilizing the latter when designing an association algorithm, and formulate it as a network-wide weighted utility maximization problem. Note that, the load mentioned above relates to the amount of required subbands decided by actual rate requirements, i.e., QoS, but the number of associated users, thus it can reflect user's actual load level. As for the proposed problem, we give a maximum probability (max-probability) algorithm by relaxing variables as well as a low-complexity distributed algorithm with a near-optimal solution that provides a theoretical performance guarantee. Experimental results show that, compared with the association strategy advocated by Ye, our strategy has a speeder convergence rate, a lower call blocking probability and a higher load balancing level.
A Topologically-informed Hyperstreamline Seeding Method for Alignment Tensor Fields
Venue: arXiv cs.GR 2014
Year: 2014

A topologically-informed method is presented for seeding of hyperstreamlines for visualization of alignment tensor fields. The method is inspired by and applied to visualization of nematic liquid crystal (LC) reorientation dynamics simulations. The method distributes hyperstreamlines along domain boundaries and edges of a nearest-neighbor graph whose vertices are degenerate regions of the alignment tensor field, which correspond to orientational defects in a nematic LC domain. This is accomplished without iteration while conforming to a user-specified spacing between hyperstreamlines and avoids possible failure modes associated with hyperstreamline integration in the vicinity of degeneracies of alignment (orientational defects). It is shown that the presented seeding method enables automated hyperstreamline-based visualization of a broad range of alignment tensor fields which enhances the ability of researchers to interpret these fields and provides an alternative to using glyph-based techniques.
Some remarks on spatial uniformity of solutions of reaction-diffusion PDE's and a related synchronization problem for ODE's
Venue: arXiv cs.SY 2014
Year: 2014

In this note, we present a condition which guarantees spatial uniformity for the asymptotic behavior of the solutions of a reaction-diffusion PDE with Neumann boundary conditions in one dimension, using the Jacobian matrix of the reaction term and the first Dirichlet eigenvalue of the Laplacian operator on the given spatial domain. We also derive an analog of this PDE result for the synchronization of a network of identical ODE models coupled by diffusion terms.
Opportunistic Downlink Interference Alignment
Venue: arXiv cs.IT 2014
Year: 2014

In this paper, we propose an opportunistic downlink interference alignment (ODIA) for interference-limited cellular downlink, which intelligently combines user scheduling and downlink IA techniques. The proposed ODIA not only efficiently reduces the effect of inter-cell interference from other-cell base stations (BSs) but also eliminates intra-cell interference among spatial streams in the same cell. We show that the minimum number of users required to achieve a target degrees-of-freedom (DoF) can be fundamentally reduced, i.e., the fundamental user scaling law can be improved by using the ODIA, compared with the existing downlink IA schemes. In addition, we adopt a limited feedback strategy in the ODIA framework, and then analyze the required number of feedback bits leading to the same performance as that of the ODIA assuming perfect feedback. We also modify the original ODIA in order to further improve sum-rate, which achieves the optimal multiuser diversity gain, i.e., $\log \log N$, per spatial stream even in the presence of downlink inter-cell interference, where $N$ denotes the number of users in a cell. Simulation results show that the ODIA significantly outperforms existing interference management techniques in terms of sum-rate in realistic cellular environments. Note that the ODIA operates in a distributed and decoupled manner, while requiring no information exchange among BSs and no iterative beamformer optimization between BSs and users, thus leading to an easier implementation.
Determining Relevant Relations for Datalog Queries under Access Limitations is Undecidable
Venue: arXiv cs.DB 2014
Year: 2014

Access limitations are restrictions in the way in which the tuples of a relation can be accessed. Under access limitations, query answering becomes more complex than in the traditional case, with no guarantee that the answer tuples that can be extracted (aka maximal answer) are all those that would be found without access limitations (aka complete answer). The field of query answering under access limitations has been broadly investigated in the past. Attention has been devoted to the problem of determining relations that are relevant for a query, i.e., those (possibly off-query) relations that might need to be accessed in order to find all tuples in the maximal answer. In this short paper, we show that relevance is undecidable for Datalog queries.
Robust Hierarchical Clustering
Venue: arXiv cs.LG 2014
Year: 2014

One of the most widely used techniques for data clustering is agglomerative clustering. Such algorithms have been long used across many different fields ranging from computational biology to social sciences to computer vision in part because their output is easy to interpret. Unfortunately, it is well known, however, that many of the classic agglomerative clustering algorithms are not robust to noise. In this paper we propose and analyze a new robust algorithm for bottom-up agglomerative clustering. We show that our algorithm can be used to cluster accurately in cases where the data satisfies a number of natural properties and where the traditional agglomerative algorithms fail. We also show how to adapt our algorithm to the inductive setting where our given data is only a small random sample of the entire data set. Experimental evaluations on synthetic and real world data sets show that our algorithm achieves better performance than other hierarchical algorithms in the presence of noise.
Analysis and Control of Beliefs in Social Networks
Venue: arXiv cs.SI 2014
Year: 2014

In this paper, we investigate the problem of how beliefs diffuse among members of social networks. We propose an information flow model (IFM) of belief that captures how interactions among members affect the diffusion and eventual convergence of a belief. The IFM model includes a generalized Markov Graph (GMG) model as a social network model, which reveals that the diffusion of beliefs depends heavily on two characteristics of the social network characteristics, namely degree centralities and clustering coefficients. We apply the IFM to both converged belief estimation and belief control strategy optimization. The model is compared with an IFM including the Barabasi-Albert model, and is evaluated via experiments with published real social network data.