OpenRaiser/Intern-Atlas huggingface dataset:
On the Computational Complexity of Model Checking for Dynamic Epistemic Logic with S5 Models
Venue: arXiv cs.CC 2020
Year: 2020

Dynamic epistemic logic (DEL) is a logical framework for representing and reasoning about knowledge change for multiple agents. An important computational task in this framework is the model checking problem, which has been shown to be PSPACE-hard even for S5 models and two agents---in the presence of other features, such as multi-pointed models. We answer open questions in the literature about the complexity of this problem in more restricted settings. We provide a detailed complexity analysis of the model checking problem for DEL, where we consider various combinations of restrictions, such as the number of agents, whether the models are single-pointed or multi-pointed, and whether postconditions are allowed in the updates. In particular, we show that the problem is already PSPACE-hard in (1) the case of one agent, multi-pointed S5 models, and no postconditions, and (2) the case of two agents, only single-pointed S5 models, and no postconditions. In addition, we study the setting where only semi-private announcements are allowed as updates. We show that for this case the problem is already PSPACE-hard when restricted to two agents and three propositional variables. The results that we obtain in this paper help outline the exact boundaries of the restricted settings for which the model checking problem for DEL is computationally tractable.
Cookie Synchronization: Everything You Always Wanted to Know But Were Afraid to Ask
Venue: arXiv cs.IR 2020
Year: 2020

User data is the primary input of digital advertising, fueling the free Internet as we know it. As a result, web companies invest a lot in elaborate tracking mechanisms to acquire user data that can sell to data markets and advertisers. However, with same-origin policy, and cookies as a primary identification mechanism on the web, each tracker knows the same user with a different ID. To mitigate this, Cookie Synchronization (CSync) came to the rescue, facilitating an information sharing channel between third parties that may or not have direct access to the website the user visits. In the background, with CSync, they merge user data they own, but also reconstruct a user's browsing history, bypassing the same origin policy. In this paper, we perform a first to our knowledge in-depth study of CSync in the wild, using a year-long weblog from 850 real mobile users. Through our study, we aim to understand the characteristics of the CSync protocol and the impact it has on web users' privacy. For this, we design and implement CONRAD, a holistic mechanism to detect CSync events at real time, and the privacy loss on the user side, even when the synced IDs are obfuscated. Using CONRAD, we find that 97% of the regular web users are exposed to CSync: most of them within the first week of their browsing, and the median userID gets leaked, on average, to 3.5 different domains. Finally, we see that CSync increases the number of domains that track the user by a factor of 6.75.
Dynamically Sacrificing Accuracy for Reduced Computation: Cascaded Inference Based on Softmax Confidence
Venue: arXiv cs.LG 2020
Year: 2020

We study the tradeoff between computational effort and classification accuracy in a cascade of deep neural networks. During inference, the user sets the acceptable accuracy degradation which then automatically determines confidence thresholds for the intermediate classifiers. As soon as the confidence threshold is met, inference terminates immediately without having to compute the output of the complete network. Confidence levels are derived directly from the softmax outputs of intermediate classifiers, as we do not train special decision functions. We show that using a softmax output as a confidence measure in a cascade of deep neural networks leads to a reduction of 15%-50% in the number of MAC operations while degrading the classification accuracy by roughly 1%. Our method can be easily incorporated into pre-trained non-cascaded architectures, as we exemplify on ResNet. Our main contribution is a method that dynamically adjusts the tradeoff between accuracy and computation without retraining the model.
Convolutional Embedded Networks for Population Scale Clustering and Bio-ancestry Inferencing
Venue: arXiv cs.LG 2020
Year: 2020

The study of genetic variants can help find correlating population groups to identify cohorts that are predisposed to common diseases and explain differences in disease susceptibility and how patients react to drugs. Machine learning algorithms are increasingly being applied to identify interacting GVs to understand their complex phenotypic traits. Since the performance of a learning algorithm not only depends on the size and nature of the data but also on the quality of underlying representation, deep neural networks can learn non-linear mappings that allow transforming GVs data into more clustering and classification friendly representations than manual feature selection. In this paper, we proposed convolutional embedded networks in which we combine two DNN architectures called convolutional embedded clustering and convolutional autoencoder classifier for clustering individuals and predicting geographic ethnicity based on GVs, respectively. We employed CAE-based representation learning on 95 million GVs from the 1000 genomes and Simons genome diversity projects. Quantitative and qualitative analyses with a focus on accuracy and scalability show that our approach outperforms state-of-the-art approaches such as VariantSpark and ADMIXTURE. In particular, CEC can cluster targeted population groups in 22 hours with an adjusted rand index of 0.915, the normalized mutual information of 0.92, and the clustering accuracy of 89%. Contrarily, the CAE classifier can predict the geographic ethnicity of unknown samples with an F1 and Mathews correlation coefficient(MCC) score of 0.9004 and 0.8245, respectively. To provide interpretations of the predictions, we identify significant biomarkers using gradient boosted trees(GBT) and SHAP. Overall, our approach is transparent and faster than the baseline methods, and scalable for 5% to 100% of the full human genome.
Structural Learning of Multivariate Regression Chain Graphs via Decomposition
Venue: arXiv cs.AI 2020
Year: 2020

We extend the decomposition approach for learning Bayesian networks (BNs) proposed by (Xie et. al.) to learning multivariate regression chain graphs (MVR CGs), which include BNs as a special case. The same advantages of this decomposition approach hold in the more general setting: reduced complexity and increased power of computational independence tests. Moreover, latent (hidden) variables can be represented in MVR CGs by using bidirected edges, and our algorithm correctly recovers any independence structure that is faithful to an MVR CG, thus greatly extending the range of applications of decomposition-based model selection techniques. Simulations under a variety of settings demonstrate the competitive performance of our method in comparison with the PC-like algorithm (Sonntag and Pena). In fact, the decomposition-based algorithm usually outperforms the PC-like algorithm except in running time. The performance of both algorithms is much better when the underlying graph is sparse.
Application of information-percolation method to reconstruction problems on graphs
Venue: arXiv cs.IT 2020
Year: 2020

In this paper we propose a method of proving impossibility results based on applying strong data-processing inequalities to estimate mutual information between sets of variables forming certain Markov random fields. The end result is that mutual information between two "far away" (as measured by the graph distance) variables is bounded by the probability of the existence of an open path in a bond-percolation problem on the same graph. Furthermore, stronger bounds can be obtained by establishing mutual information comparison results with an erasure model on the same graph, with erasure probabilities given by the contraction coefficients. As applications, we show that our method gives sharp threshold for partially recovering a rank-one perturbation of a random Gaussian matrix (spiked Wigner model), yields the best known upper bound on the noise level for group synchronization (obtained concurrently by Abbe and Boix), and establishes new impossibility result for community detection on the stochastic block model with $k$ communities.
Event-triggered controllers based on the supremum norm of sampling-induced error
Venue: arXiv cs.SY 2020
Year: 2020

The paper proposes a novel event-triggered control scheme for nonlinear systems based on the input-delay method. Specifically, the closed-loop system is associated with a pair of auxiliary input and output. The auxiliary output is defined as the derivative of the continuous-time input function, while the auxiliary input is defined as the input disturbance caused by the sampling or equivalently the integral of the auxiliary output over the sampling period. As a result, a cyclic mapping forms from the input to the output via the system dynamics and back from the output to the input via the integral. The event-triggering law is constructed to make the mapping contractive such that the stabilization is achieved and an easy-to-check Zeno-free condition is provided. With this idea, we develop a theorem for the event-triggered control of interconnected nonlinear systems which is employed to solve the event-triggered control for lower-triangular systems with dynamic uncertainties.
On Adaptive Linear-Quadratic Regulators
Venue: arXiv cs.SY 2020
Year: 2020

Performance of adaptive control policies is assessed through the regret with respect to the optimal regulator, which reflects the increase in the operating cost due to uncertainty about the dynamics parameters. However, available results in the literature do not provide a quantitative characterization of the effect of the unknown parameters on the regret. Further, there are problems regarding the efficient implementation of some of the existing adaptive policies. Finally, results regarding the accuracy with which the system's parameters are identified are scarce and rather incomplete. This study aims to comprehensively address these three issues. First, by introducing a novel decomposition of adaptive policies, we establish a sharp expression for the regret of an arbitrary policy in terms of the deviations from the optimal regulator. Second, we show that adaptive policies based on slight modifications of the Certainty Equivalence scheme are efficient. Specifically, we establish a regret of (nearly) square-root rate for two families of randomized adaptive policies. The presented regret bounds are obtained by using anti-concentration results on the random matrices employed for randomizing the estimates of the unknown parameters. Moreover, we study the minimal additional information on dynamics matrices that using them the regret will become of logarithmic order. Finally, the rates at which the unknown parameters of the system are being identified are presented.
Active query-driven visual search using probabilistic bisection and convolutional neural networks
Venue: arXiv cs.CV 2020
Year: 2020

We present a novel efficient object detection and localization framework based on the probabilistic bisection algorithm. A Convolutional Neural Network (CNN) is trained and used as a noisy oracle that provides answers to input query images. The responses along with error probability estimates obtained from the CNN are used to update beliefs on the object location along each dimension. We show that querying along each dimension achieves the same lower bound on localization error as the joint query design. Finally, we compare our approach to the traditional sliding window technique on a real world face localization task and show speed improvements by at least an order of magnitude while maintaining accurate localization.
Multi-Source Multi-Sink Nash Flows Over Time
Venue: arXiv cs.GT 2020
Year: 2020

Nash flows over time describe the behavior of selfish users eager to reach their destination as early as possible while traveling along the arcs of a network with capacities and transit times. Throughout the past decade, they have been thoroughly studied in single-source single-sink networks for the deterministic queuing model, which is of particular relevance and frequently used in the context of traffic and transport networks. In this setting there exist Nash flows over time that can be described by a sequence of static flows featuring special properties, so-called `thin flows with resetting'. This insight can also be used algorithmically to compute Nash flows over time. We present an extension of these result to networks with multiple sources and sinks which are much more relevant in practical applications. In particular, we come up with a subtle generalization of thin flows with resetting which yields a compact description as well as an algorithmic approach for computing multi-terminal Nash flows over time.
Functional Object-Oriented Network: Construction & Expansion
Venue: arXiv cs.RO 2020
Year: 2020

We build upon the functional object-oriented network (FOON), a structured knowledge representation which is constructed from observations of human activities and manipulations. A FOON can be used for representing object-motion affordances. Knowledge retrieval through graph search allows us to obtain novel manipulation sequences using knowledge spanning across many video sources, hence the novelty in our approach. However, we are limited to the sources collected. To further improve the performance of knowledge retrieval as a follow up to our previous work, we discuss generalizing knowledge to be applied to objects which are similar to what we have in FOON without manually annotating new sources of knowledge. We discuss two means of generalization: 1) expanding our network through the use of object similarity to create new functional units from those we already have, and 2) compressing the functional units by object categories rather than specific objects. We discuss experiments which compare the performance of our knowledge retrieval algorithm with both expansion and compression by categories.
CSI Learning Based Active Secure Coding Scheme For Detectable Wiretap Channel
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we consider the problem of secure and reliable communication with uncertain channel state information (CSI) and present a new solution named active secure coding which combines the machine learning methods with the traditional physical layer secure coding scheme. First, we build a detectable wiretap channel model by combining the hidden Markov model with the compound wiretap channel model, in which the varying of channel block CSI is a Markov process and the detected information is a stochastic emission from the current CSI. Next, we present a CSI learning scheme to learn the CSI from the detected information by the Baum-Welch and Viterbi algorithms. Then we construct explicit secure polar codes based on the learned CSI, and combine it with the CSI learning scheme to form the active secure polar coding scheme. Simulation results show that an acceptable level of reliability and security can be achieved by the proposed active secure polar coding scheme.
Polynomial bounds for centered colorings on proper minor-closed graph classes
Venue: arXiv cs.DM 2020
Year: 2020

For $p\in \mathbb{N}$, a coloring $λ$ of the vertices of a graph $G$ is {\em{$p$-centered}} if for every connected subgraph~$H$ of $G$, either $H$ receives more than $p$ colors under $λ$ or there is a color that appears exactly once in $H$. In this paper, we prove that every $K_t$-minor-free graph admits a $p$-centered coloring with $\mathcal{O}(p^{g(t)})$ colors for some function $g$. In the special case that the graph is embeddable in a fixed surface $Σ$ we show that it admits a $p$-centered coloring with $\mathcal{O}(p^{19})$ colors, with the degree of the polynomial independent of the genus of $Σ$. This provides the first polynomial upper bounds on the number of colors needed in $p$-centered colorings of graphs drawn from proper minor-closed classes, which answers an open problem posed by Dvoř{á}k. As an algorithmic application, we use our main result to prove that if $\mathcal{C}$ is a fixed proper minor-closed class of graphs, then given graphs $H$ and $G$, on $p$ and $n$ vertices, respectively, where $G\in \mathcal{C}$, it can be decided whether $H$ is a subgraph of $G$ in time $2^{\mathcal{O}(p\log p)}\cdot n^{\mathcal{O}(1)}$ and space $n^{\mathcal{O}(1)}$.
Learning Implicit Generative Models by Teaching Explicit Ones
Venue: arXiv cs.LG 2020
Year: 2020

Implicit generative models are difficult to train as no explicit density functions are defined. Generative adversarial nets (GANs) present a minimax framework to train such models, which however can suffer from mode collapse due to the nature of the JS-divergence. This paper presents a learning by teaching (LBT) approach to learning implicit models, which intrinsically avoids the mode collapse problem by optimizing a KL-divergence rather than the JS-divergence in GANs. In LBT, an auxiliary density estimator is introduced to fit the implicit model's distribution while the implicit model teaches the density estimator to match the data distribution. LBT is formulated as a bilevel optimization problem, whose optimal generator matches the true data distribution. LBT can be naturally integrated with GANs to derive a hybrid LBT-GAN that enjoys complimentary benefits. Finally, we present a stochastic gradient ascent algorithm with unrolling to solve the challenging learning problems. Experimental results demonstrate the effectiveness of our method.
Negative Momentum for Improved Game Dynamics
Venue: arXiv cs.LG 2020
Year: 2020

Games generalize the single-objective optimization paradigm by introducing different objective functions for different players. Differentiable games often proceed by simultaneous or alternating gradient updates. In machine learning, games are gaining new importance through formulations like generative adversarial networks (GANs) and actor-critic systems. However, compared to single-objective optimization, game dynamics are more complex and less understood. In this paper, we analyze gradient-based methods with momentum on simple games. We prove that alternating updates are more stable than simultaneous updates. Next, we show both theoretically and empirically that alternating gradient updates with a negative momentum term achieves convergence in a difficult toy adversarial problem, but also on the notoriously difficult to train saturating GANs.
Computer Analysis of Architecture Using Automatic Image Understanding
Venue: arXiv cs.CV 2020
Year: 2020

In the past few years, computer vision and pattern recognition systems have been becoming increasingly more powerful, expanding the range of automatic tasks enabled by machine vision. Here we show that computer analysis of building images can perform quantitative analysis of architecture, and quantify similarities between city architectural styles in a quantitative fashion. Images of buildings from 18 cities and three countries were acquired using Google StreetView, and were used to train a machine vision system to automatically identify the location of the imaged building based on the image visual content. Experimental results show that the automatic computer analysis can automatically identify the geographical location of the StreetView image. More importantly, the algorithm was able to group the cities and countries and provide a phylogeny of the similarities between architectural styles as captured by StreetView images. These results demonstrate that computer vision and pattern recognition algorithms can perform the complex cognitive task of analyzing images of buildings, and can be used to measure and quantify visual similarities and differences between different styles of architectures. This experiment provides a new paradigm for studying architecture, based on a quantitative approach that can enhance the traditional manual observation and analysis. The source code used for the analysis is open and publicly available.
Newton-ADMM: A Distributed GPU-Accelerated Optimizer for Multiclass Classification Problems
Venue: arXiv cs.LG 2020
Year: 2020

First-order optimization methods, such as stochastic gradient descent (SGD) and its variants, are widely used in machine learning applications due to their simplicity and low per-iteration costs. However, they often require larger numbers of iterations, with associated communication costs in distributed environments. In contrast, Newton-type methods, while having higher per-iteration costs, typically require a significantly smaller number of iterations, which directly translates to reduced communication costs. In this paper, we present a novel distributed optimizer for classification problems, which integrates a GPU-accelerated Newton-type solver with the global consensus formulation of Alternating Direction of Method Multipliers (ADMM). By leveraging the communication efficiency of ADMM, GPU-accelerated inexact-Newton solver, and an effective spectral penalty parameter selection strategy, we show that our proposed method (i) yields better generalization performance on several classification problems; (ii) significantly outperforms state-of-the-art methods in distributed time to solution; and (iii) offers better scaling on large distributed platforms.
An argument in favor of strong scaling for deep neural networks with small datasets
Venue: arXiv cs.DC 2020
Year: 2020

In recent years, with the popularization of deep learning frameworks and large datasets, researchers have started parallelizing their models in order to train faster. This is crucially important, because they typically explore many hyperparameters in order to find the best ones for their applications. This process is time consuming and, consequently, speeding up training improves productivity. One approach to parallelize deep learning models followed by many researchers is based on weak scaling. The minibatches increase in size as new GPUs are added to the system. In addition, new learning rates schedules have been proposed to fix optimization issues that occur with large minibatch sizes. In this paper, however, we show that the recommendations provided by recent work do not apply to models that lack large datasets. In fact, we argument in favor of using strong scaling for achieving reliable performance in such cases. We evaluated our approach with up to 32 GPUs and show that weak scaling not only does not have the same accuracy as the sequential model, it also fails to converge most of time. Meanwhile, strong scaling has good scalability while having exactly the same accuracy of a sequential implementation.