OpenRaiser/Intern-Atlas huggingface dataset:
Approximate MMAP by Marginal Search
Venue: arXiv cs.AI 2020
Year: 2020

We present a heuristic strategy for marginal MAP (MMAP) queries in graphical models. The algorithm is based on a reduction of the task to a polynomial number of marginal inference computations. Given an input evidence, the marginals mass functions of the variables to be explained are computed. Marginal information gain is used to decide the variables to be explained first, and their most probable marginal states are consequently moved to the evidence. The sequential iteration of this procedure leads to a MMAP explanation and the minimum information gain obtained during the process can be regarded as a confidence measure for the explanation. Preliminary experiments show that the proposed confidence measure is properly detecting instances for which the algorithm is accurate and, for sufficiently high confidence levels, the algorithm gives the exact solution or an approximation whose Hamming distance from the exact one is small.
On the Performance Analysis of Epidemic Routing in Non-Sparse Delay Tolerant Networks
Venue: arXiv cs.NI 2020
Year: 2020

We study the behavior of epidemic routing in a delay tolerant network as a function of node density. Focusing on the probability of successful delivery to a destination within a deadline (PS), we show that PS experiences a phase transition as node density increases. Specifically, we prove that PS exhibits a phase transition when nodes are placed according to a Poisson process and allowed to move according to independent and identical processes with limited speed. We then propose four fluid models to evaluate the performance of epidemic routing in non-sparse networks. A model is proposed for supercritical networks based on approximation of the infection rate as a function of time. Other models are based on the approximation of the pairwise infection rate. Two of them, one for subcritical networks and another for supercritical networks, use the pairwise infection rate as a function of the number of infected nodes. The other model uses pairwise infection rate as a function of time, and can be applied for both subcritical and supercritical networks achieving good accuracy. The model for subcritical networks is accurate when density is not close to the percolation critical density. Moreover, the models that target only supercritical regime are accurate.
On the I/O complexity of the k-nearest neighbor problem
Venue: arXiv cs.DS 2020
Year: 2020

We consider static, external memory indexes for exact and approximate versions of the $k$-nearest neighbor ($k$-NN) problem, and show new lower bounds under a standard indivisibility assumption: - Polynomial space indexing schemes for high-dimensional $k$-NN in Hamming space cannot take advantage of block transfers: $Ω(k)$ block reads are needed to to answer a query. - For the $\ell_\infty$ metric the lower bound holds even if we allow $c$-appoximate nearest neighbors to be returned, for $c \in (1, 3)$. - The restriction to $c < 3$ is necessary: For every metric there exists an indexing scheme in the indexability model of Hellerstein et al.~using space $O(kn)$, where $n$ is the number of points, that can retrieve $k$ 3-approximate nearest neighbors using $\lceil k/B\rceil$ I/Os, which is optimal. - For specific metrics, data structures with better approximation factors are possible. For $k$-NN in Hamming space and every approximation factor $c>1$ there exists a polynomial space data structure that returns $k$ $c$-approximate nearest neighbors in $\lceil k/B\rceil$ I/Os. To show these lower bounds we develop two new techniques: First, to handle that approximation algorithms have more freedom in deciding which result set to return we develop a relaxed version of the $λ$-set workload technique of Hellerstein et al. This technique allows us to show lower bounds that hold in $d\geq n$ dimensions. To extend the lower bounds down to $d = O(k \log(n/k))$ dimensions, we develop a new deterministic dimension reduction technique that may be of independent interest.
Solving Some Affine Equations over Finite Fields
Venue: arXiv cs.IT 2020
Year: 2020

Let $l$ and $k$ be two integers such that $l|k$. Define $T_l^k(X):=X+X^{p^l}+\cdots+X^{p^{l(k/l-2)}}+X^{p^{l(k/l-1)}}$ and $S_l^k(X):=X-X^{p^l}+\cdots+(-1)^{(k/l-1)}X^{p^{l(k/l-1)}}$, where $p$ is any prime. This paper gives explicit representations of all solutions in $\GF{p^n}$ to the affine equations $T_l^{k}(X)=a$ and $S_l^{k}(X)=a$, $a\in \GF{p^n}$. For the case $p=2$ that was solved very recently in \cite{MKCL2019}, the result of this paper reveals another solution.
Federated Matrix Factorization: Algorithm Design and Application to Data Clustering
Venue: arXiv cs.LG 2020
Year: 2020

Recent demands on data privacy have called for federated learning (FL) as a new distributed learning paradigm in massive and heterogeneous networks. Although many FL algorithms have been proposed, few of them have considered the matrix factorization (MF) model, which is known to have a vast number of signal processing and machine learning applications. Different from the existing FL algorithms that are designed for smooth problems with single block of variables, in federated MF (FedMF), one has to deal with challenging non-convex and non-smooth problems (due to constraints or regularization) with two blocks of variables. In this paper, we address the challenge by proposing two new FedMF algorithms, namely, FedMAvg and FedMGS, based on the model averaging and gradient sharing principles, respectively. Both FedMAvg and FedMGS adopt multiple steps of local updates per communication round to speed up convergence, and allow only a randomly sampled subset of clients to communicate with the server for reducing the communication cost. Convergence analyses for the two algorithms are respectively presented, which delineate the impacts of data distribution, local update number, and partial client communication on the algorithm performance. By focusing on a data clustering task, extensive experiment results are presented to examine the practical performance of both algorithms, as well as demonstrating their efficacy over the existing distributed clustering algorithms.
GRASPA 1.0: GRASPA is a Robot Arm graSping Performance benchmArk
Venue: arXiv cs.RO 2020
Year: 2020

The use of benchmarks is a widespread and scientifically meaningful practice to validate performance of different approaches to the same task. In the context of robot grasping the use of common object sets has emerged in recent years, however no dominant protocols and metrics to test grasping pipelines have taken root yet. In this paper, we present version 1.0 of GRASPA, a benchmark to test effectiveness of grasping pipelines on physical robot setups. This approach tackles the complexity of such pipelines by proposing different metrics that account for the features and limits of the test platform. As an example application, we deploy GRASPA on the iCub humanoid robot and use it to benchmark our grasping pipeline. As closing remarks, we discuss how the GRASPA indicators we obtained as outcome can provide insight into how different steps of the pipeline affect the overall grasping performance.
Learning light field synthesis with Multi-Plane Images: scene encoding as a recurrent segmentation task
Venue: arXiv cs.CV 2020
Year: 2020

In this paper we address the problem of view synthesis from large baseline light fields, by turning a sparse set of input views into a Multi-plane Image (MPI). Because available datasets are scarce, we propose a lightweight network that does not require extensive training. Unlike latest approaches, our model does not learn to estimate RGB layers but only encodes the scene geometry within MPI alpha layers, which comes down to a segmentation task. A Learned Gradient Descent (LGD) framework is used to cascade the same convolutional network in a recurrent fashion in order to refine the volumetric representation obtained. Thanks to its low number of parameters, our model trains successfully on a small light field video dataset and provides visually appealing results. It also exhibits convenient generalization properties regarding both the number of input views, the number of depth planes in the MPI, and the number of refinement iterations.
Detect and Correct Bias in Multi-Site Neuroimaging Datasets
Venue: arXiv cs.CV 2020
Year: 2020

The desire to train complex machine learning algorithms and to increase the statistical power in association studies drives neuroimaging research to use ever-larger datasets. The most obvious way to increase sample size is by pooling scans from independent studies. However, simple pooling is often ill-advised as selection, measurement, and confounding biases may creep in and yield spurious correlations. In this work, we combine 35,320 magnetic resonance images of the brain from 17 studies to examine bias in neuroimaging. In the first experiment, Name That Dataset, we provide empirical evidence for the presence of bias by showing that scans can be correctly assigned to their respective dataset with 71.5% accuracy. Given such evidence, we take a closer look at confounding bias, which is often viewed as the main shortcoming in observational studies. In practice, we neither know all potential confounders nor do we have data on them. Hence, we model confounders as unknown, latent variables. Kolmogorov complexity is then used to decide whether the confounded or the causal model provides the simplest factorization of the graphical model. Finally, we present methods for dataset harmonization and study their ability to remove bias in imaging features. In particular, we propose an extension of the recently introduced ComBat algorithm to control for global variation across image features, inspired by adjusting for population stratification in genetics. Our results demonstrate that harmonization can reduce dataset-specific information in image features. Further, confounding bias can be reduced and even turned into a causal relationship. However, harmonziation also requires caution as it can easily remove relevant subject-specific information. Code is available at https://github.com/ai-med/Dataset-Bias.
Recreating Bat Behavior on Quad-rotor UAVs-A Simulation Approach
Venue: arXiv cs.RO 2020
Year: 2020

We develop an effective computer model to simulate sensing environments that consist of natural trees. The simulated environments are random and contain full geometry of the tree foliage. While this simulated model can be used as a general platform for studying the sensing mechanism of different flying species, our ultimate goal is to build bat-inspired Quad-rotor UAVs- UAVs that can recreate bat's flying behavior (e.g., obstacle avoidance, path planning) in dense vegetation. To this end, we also introduce an foliage echo simulator that can produce simulated echoes by mimicking bat's biosonar. In our current model, a few realistic model choices or assumptions are made. First, in order to create natural looking trees, the branching structures of trees are modeled by L-systems, whereas the detailed geometry of branches, sub-branches and leaves is created by randomizing a reference tree in a CAD object file. Additionally, the foliage echo simulator is simplified so that no shading effect is considered. We demonstrate our developed model by simulating real-world scenarios with multiple trees and compute the corresponding impulse responses along a Quad-rotor trajectory.
EncDBDB: Searchable Encrypted, Fast, Compressed, In-Memory Database using Enclaves
Venue: arXiv cs.CR 2020
Year: 2020

Data confidentiality is an important requirement for clients when outsourcing databases to the cloud. Trusted execution environments, such as Intel SGX, offer an efficient, hardware-based solution to this cryptographic problem. Existing solutions are not optimized for column-oriented, in-memory databases and pose impractical memory requirements on the enclave. We present EncDBDB, a novel approach for client-controlled encryption of a column-oriented, in-memory databases allowing range searches using an enclave. EncDBDB offers nine encrypted dictionaries, which provide different security, performance and storage efficiency tradeoffs for the data. It is especially suited for complex, read-oriented, analytic queries, e.g., as present in data warehouses. The computational overhead compared to plaintext processing is within a millisecond even for databases with millions of entries and the leakage is limited. Compressed encrypted data requires less space than a corresponding plaintext column. Furthermore, the resulting code - and data - in the enclave is very small reducing the potential for security-relevant implementation errors and side-channel leakages.
Component Analysis for Visual Question Answering Architectures
Venue: arXiv cs.CV 2020
Year: 2020

Recent research advances in Computer Vision and Natural Language Processing have introduced novel tasks that are paving the way for solving AI-complete problems. One of those tasks is called Visual Question Answering (VQA). A VQA system must take an image and a free-form, open-ended natural language question about the image, and produce a natural language answer as the output. Such a task has drawn great attention from the scientific community, which generated a plethora of approaches that aim to improve the VQA predictive accuracy. Most of them comprise three major components: (i) independent representation learning of images and questions; (ii) feature fusion so the model can use information from both sources to answer visual questions; and (iii) the generation of the correct answer in natural language. With so many approaches being recently introduced, it became unclear the real contribution of each component for the ultimate performance of the model. The main goal of this paper is to provide a comprehensive analysis regarding the impact of each component in VQA models. Our extensive set of experiments cover both visual and textual elements, as well as the combination of these representations in form of fusion and attention mechanisms. Our major contribution is to identify core components for training VQA models so as to maximize their predictive performance.
A Receding-Horizon MDP Approach for Performance Evaluation of Moving Target Defense in Networks
Venue: arXiv cs.CR 2020
Year: 2020

In this paper, we study the problem of assessing the effectiveness of a proactive defense-by-detection policy with a network-based moving target defense. We model the network system using a probabilistic attack graph--a graphical security model. Given a network system with a proactive defense strategy, an intelligent attacker needs to perform reconnaissance repeatedly to learn about the locations of intrusion detection systems and re-plan optimally to reach the target while avoiding detection. To compute the attacker's strategy for security evaluation, we develop a receding-horizon planning algorithm using a risk-sensitive Markov decision process with a time-varying reward function. Finally, we implement both defense and attack strategies in a synthetic network and analyze how the frequency of network randomization and the number of detection systems can influence the success rate of the attacker. This study provides insights for designing proactive defense strategies against online and multi-stage attacks by a resourceful attacker.
Multi-Agent Reinforcement Learning and Human Social Factors in Climate Change Mitigation
Venue: arXiv cs.MA 2020
Year: 2020

Many complex real-world problems, such as climate change mitigation, are intertwined with human social factors. Climate change mitigation, a social dilemma made difficult by the inherent complexities of human behavior, has an impact at a global scale. We propose applying multi-agent reinforcement learning (MARL) in this setting to develop intelligent agents that can influence the social factors at play in climate change mitigation. There are ethical, practical, and technical challenges that must be addressed when deploying MARL in this way. In this paper, we present these challenges and outline an approach to address them. Understanding how intelligent agents can be used to impact human social factors is important to prevent their abuse and can be beneficial in furthering our knowledge of these complex problems as a whole. The challenges we present are not limited to our specific application but are applicable to broader MARL. Thus, developing MARL for social factors in climate change mitigation helps address general problems hindering MARL's applicability to other real-world problems while also motivating discussion on the social implications of MARL deployment.
Learnable Bernoulli Dropout for Bayesian Deep Learning
Venue: arXiv cs.LG 2020
Year: 2020

In this work, we propose learnable Bernoulli dropout (LBD), a new model-agnostic dropout scheme that considers the dropout rates as parameters jointly optimized with other model parameters. By probabilistic modeling of Bernoulli dropout, our method enables more robust prediction and uncertainty quantification in deep models. Especially, when combined with variational auto-encoders (VAEs), LBD enables flexible semi-implicit posterior representations, leading to new semi-implicit VAE~(SIVAE) models. We solve the optimization for training with respect to the dropout parameters using Augment-REINFORCE-Merge (ARM), an unbiased and low-variance gradient estimator. Our experiments on a range of tasks show the superior performance of our approach compared with other commonly used dropout schemes. Overall, LBD leads to improved accuracy and uncertainty estimates in image classification and semantic segmentation. Moreover, using SIVAE, we can achieve state-of-the-art performance on collaborative filtering for implicit feedback on several public datasets.
GLU Variants Improve Transformer
Venue: arXiv cs.LG 2020
Year: 2020

Gated Linear Units (arXiv:1612.08083) consist of the component-wise product of two linear projections, one of which is first passed through a sigmoid function. Variations on GLU are possible, using different nonlinear (or even linear) functions in place of sigmoid. We test these variants in the feed-forward sublayers of the Transformer (arXiv:1706.03762) sequence-to-sequence model, and find that some of them yield quality improvements over the typically-used ReLU or GELU activations.
Data Efficient Training for Reinforcement Learning with Adaptive Behavior Policy Sharing
Venue: arXiv cs.LG 2020
Year: 2020

Deep Reinforcement Learning (RL) is proven powerful for decision making in simulated environments. However, training deep RL model is challenging in real world applications such as production-scale health-care or recommender systems because of the expensiveness of interaction and limitation of budget at deployment. One aspect of the data inefficiency comes from the expensive hyper-parameter tuning when optimizing deep neural networks. We propose Adaptive Behavior Policy Sharing (ABPS), a data-efficient training algorithm that allows sharing of experience collected by behavior policy that is adaptively selected from a pool of agents trained with an ensemble of hyper-parameters. We further extend ABPS to evolve hyper-parameters during training by hybridizing ABPS with an adapted version of Population Based Training (ABPS-PBT). We conduct experiments with multiple Atari games with up to 16 hyper-parameter/architecture setups. ABPS achieves superior overall performance, reduced variance on top 25% agents, and equivalent performance on the best agent compared to conventional hyper-parameter tuning with independent training, even though ABPS only requires the same number of environmental interactions as training a single agent. We also show that ABPS-PBT further improves the convergence speed and reduces the variance.
Complexity Analysis of Generalized and Fractional Hypertree Decompositions
Venue: arXiv cs.CC 2020
Year: 2020

Hypertree decompositions (HDs), as well as the more powerful generalized hypertree decompositions (GHDs), and the yet more general fractional hypertree decompositions (FHDs) are hypergraph decomposition methods successfully used for answering conjunctive queries and for solving constraint satisfaction problems. Every hypergraph $H$ has a width relative to each of these methods: its hypertree width $hw(H)$, its generalized hypertree width $ghw(H)$, and its fractional hypertree width $fhw(H)$, respectively. It is known that $hw(H)\leq k$ can be checked in polynomial time for fixed $k$, while checking $ghw(H)\leq k$ is NP-complete for $k \geq 3$. The complexity of checking $fhw(H)\leq k$ for a fixed $k$ has been open for over a decade. We settle this open problem by showing that checking $fhw(H)\leq k$ is NP-complete, even for $k=2$. The same construction allows us to prove also the NP-completeness of checking $ghw(H)\leq k$ for $k=2$. After that, we identify meaningful restrictions which make checking for bounded $ghw$ or $fhw$ tractable or allow for an efficient approximation of the $fhw$.
Change actions: from incremental computation to discrete derivatives
Venue: arXiv cs.LO 2020
Year: 2020

The goal of this thesis is threefold: first, to provide a general semantic setting for reasoning about incremental computation. Second, to establish and clarify the connection between derivatives in the incremental sense and derivatives in the analytic sense, that is to say, to provide a common definition of derivative of which the previous two are particular instances. Third, to give a theoretically sound calculus for this general setting. To this end we define and explore the notions of change actions and differential maps between change actions and show how these notions relate to incremental computation through the concrete example of the semi-naive evaluation of Datalog queries. We also introduce the notion of a change action model as a setting for higher-order differentiation, and exhibit some interesting examples. Finally, we show how Cartesian difference categories, a family of particularly well-behaved change action models, generalise Cartesian differential categories and give rise to a calculus in the spirit of Ehrhard and Regnier's differential lambda-calculus.
A Bounded Measure for Estimating the Benefit of Visualization
Venue: arXiv cs.AI 2020
Year: 2020

Information theory can be used to analyze the cost-benefit of visualization processes. However, the current measure of benefit contains an unbounded term that is neither easy to estimate nor intuitive to interpret. In this work, we propose to revise the existing cost-benefit measure by replacing the unbounded term with a bounded one. We examine a number of bounded measures that include the Jenson-Shannon divergence and a new divergence measure formulated as part of this work. We use visual analysis to support the multi-criteria comparison, narrowing the search down to those options with better mathematical properties. We apply those remaining options to two visualization case studies to instantiate their uses in practical scenarios, while the collected real world data further informs the selection of a bounded measure, which can be used to estimate the benefit of visualization.