OpenRaiser/Intern-Atlas huggingface dataset:
Online Gaussian Process State-Space Model: Learning and Planning for Partially Observable Dynamical Systems
Venue: arXiv cs.RO 2020
Year: 2020

This paper proposes an online learning method of Gaussian process state-space model (GP-SSM). GP-SSM is a probabilistic representation learning scheme that represents unknown state transition and/or measurement models as Gaussian processes (GPs). While the majority of prior literature on learning of GP-SSM are focused on processing a given set of time series data, data may arrive and accumulate sequentially over time in most dynamical systems. Storing all such sequential data and updating the model over entire data incur large amount of computational resources in space and time. To overcome this difficulty, we propose a practical method, termed \textit{onlineGPSSM}, that incorporates stochastic variational inference (VI) and online VI with novel formulation. The proposed method mitigates the computational complexity without catastrophic forgetting and also support adaptation to changes in a system and/or a real environments. Furthermore, we present application of onlineGPSSM into the reinforcement learning (RL) of partially observable dynamical systems by integrating onlineGPSSM with Bayesian filtering and trajectory optimization algorithms. Numerical examples are presented to demonstrate applicability of the proposed method.
OverSketched Newton: Fast Convex Optimization for Serverless Systems
Venue: arXiv cs.DC 2020
Year: 2020

Motivated by recent developments in serverless systems for large-scale computation as well as improvements in scalable randomized matrix algorithms, we develop OverSketched Newton, a randomized Hessian-based optimization algorithm to solve large-scale convex optimization problems in serverless systems. OverSketched Newton leverages matrix sketching ideas from Randomized Numerical Linear Algebra to compute the Hessian approximately. These sketching methods lead to inbuilt resiliency against stragglers that are a characteristic of serverless architectures. Depending on whether the problem is strongly convex or not, we propose different iteration updates using the approximate Hessian. For both cases, we establish convergence guarantees for OverSketched Newton and empirically validate our results by solving large-scale supervised learning problems on real-world datasets. Experiments demonstrate a reduction of ~50% in total running time on AWS Lambda, compared to state-of-the-art distributed optimization schemes.
Comparison of State-of-the-Art Deep Learning APIs for Image Multi-Label Classification using Semantic Metrics
Venue: arXiv cs.CV 2020
Year: 2020

Image understanding heavily relies on accurate multi-label classification. In recent years, deep learning algorithms have become very successful for such tasks, and various commercial and open-source APIs have been released for public use. However, these APIs are often trained on different datasets, which, besides affecting their performance, might pose a challenge to their performance evaluation. This challenge concerns the different object-class dictionaries of the APIs' training dataset and the benchmark dataset, in which the predicted labels are semantically similar to the benchmark labels but considered different simply because they have different wording in the dictionaries. To face this challenge, we propose semantic similarity metrics to obtain richer understating of the APIs predicted labels and thus their performance. In this study, we evaluate and compare the performance of 13 of the most prominent commercial and open-source APIs in a best-of-breed challenge on the Visual Genome and Open Images benchmark datasets. Our findings demonstrate that, while using traditional metrics, the Microsoft Computer Vision, Imagga, and IBM APIs performed better than others. However, applying semantic metrics also unveil the InceptionResNet-v2, Inception-v3, and ResNet50 APIs, which are trained only with the simple ImageNet dataset, as challengers for top semantic performers.
Ensemble Observability of Bloch Equations with Unknown Population Density
Venue: arXiv cs.SY 2020
Year: 2020

We introduce in the paper a novel observability problem for a continuum ensemble of nonholonomic control systems with unknown population density. We address the problem by focussing on a prototype of such ensemble system, namely, the ensemble of Bloch equations. The dynamics of the equations are structurally identical, but show variations in Larmor dispersion and radio frequency (rf) inhomogeneity. We assume that the initial state of every individual system is unknown and, moreover, the population density of these individual systems is also unknown. Furthermore, we assume that at any time, there is only one scalar measurement output at our disposal. The measurement output integrates a certain observation function, common to all individual systems, over the continuum ensemble. The observability problem we pose in the paper is thus the following: Whether one is able to use the common control input (i.e., the rf field) and the single measurement output to estimate the initial states of the individual systems and, moreover, to identify the population density? Amongst other things, we establish a sufficient condition for the ensemble system to be observable: We show that if the common observation function is any harmonic homogeneous polynomial of positive degree, then the ensemble system is observable. The main focus of the paper is to demonstrate how to leverage tools from representation theory of Lie algebras to address the observability problem. Although the results we establish in the paper are for the specific ensemble of Bloch equations, the approach we develop along the analysis can be generalized to investigate observability of other general ensembles of nonholonomic control systems with a single, integrated measurement output.
Recovering the Lowest Layer of Deep Networks with High Threshold Activations
Venue: arXiv cs.LG 2020
Year: 2020

Giving provable guarantees for learning neural networks is a core challenge of machine learning theory. Most prior work gives parameter recovery guarantees for one hidden layer networks, however, the networks used in practice have multiple non-linear layers. In this work, we show how we can strengthen such results to deeper networks -- we address the problem of uncovering the lowest layer in a deep neural network under the assumption that the lowest layer uses a high threshold before applying the activation, the upper network can be modeled as a well-behaved polynomial and the input distribution is Gaussian.
Overcoming Small Minirhizotron Datasets Using Transfer Learning
Venue: arXiv cs.CV 2020
Year: 2020

Minirhizotron technology is widely used for studying the development of roots. Such systems collect visible-wavelength color imagery of plant roots in-situ by scanning an imaging system within a clear tube driven into the soil. Automated analysis of root systems could facilitate new scientific discoveries that would be critical to address the world's pressing food, resource, and climate issues. A key component of automated analysis of plant roots from imagery is the automated pixel-level segmentation of roots from their surrounding soil. Supervised learning techniques appear to be an appropriate tool for the challenge due to varying local soil and root conditions, however, lack of enough annotated training data is a major limitation due to the error-prone and time-consuming manually labeling process. In this paper, we investigate the use of deep neural networks based on the U-net architecture for automated, precise pixel-wise root segmentation in minirhizotron imagery. We compiled two minirhizotron image datasets to accomplish this study: one with 17,550 peanut root images and another with 28 switchgrass root images. Both datasets were paired with manually labeled ground truth masks. We trained three neural networks with different architectures on the larger peanut root dataset to explore the effect of the neural network depth on segmentation performance. To tackle the more limited switchgrass root dataset, we showed that models initialized with features pre-trained on the peanut dataset and then fine-tuned on the switchgrass dataset can improve segmentation performance significantly. We obtained 99\% segmentation accuracy in switchgrass imagery using only 21 training images. We also observed that features pre-trained on a closely related but relatively moderate size dataset like our peanut dataset are more effective than features pre-trained on the large but unrelated ImageNet dataset.
Managing Recurrent Virtual Network Updates in Multi-Tenant Datacenters: A System Perspective
Venue: arXiv cs.CR 2020
Year: 2020

With the advent of software-defined networking, network configuration through programmable interfaces becomes practical, leading to various on-demand opportunities for network routing update in multi-tenant datacenters, where tenants have diverse requirements on network routings such as short latency, low path inflation, large bandwidth, high reliability, etc. Conventional solutions that rely on topology search coupled with an objective function https:// www.overleaf.com/project/5beb742041ab9c0e3caec84f to find desired routings have at least two shortcomings: (i) they run into scalability issues when handling consistent and frequent routing updates and (ii) they restrict the flexibility and capability to satisfy various routing requirements. To address these issues, this paper proposes a novel search and optimization decoupled design, which not only saves considerable topology search costs via search result reuse, but also avoids possible sub-optimality in greedy routing search algorithms by making decisions based on the global view of all possible routings. We implement a prototype of our proposed system, OpReduce, and perform extensive evaluations to validate its design goals.
Winning Isn't Everything: Enhancing Game Development with Intelligent Agents
Venue: arXiv cs.AI 2020
Year: 2020

Recently, there have been several high-profile achievements of agents learning to play games against humans and beat them. In this paper, we study the problem of training intelligent agents in service of game development. Unlike the agents built to "beat the game", our agents aim to produce human-like behavior to help with game evaluation and balancing. We discuss two fundamental metrics based on which we measure the human-likeness of agents, namely skill and style, which are multi-faceted concepts with practical implications outlined in this paper. We report four case studies in which the style and skill requirements inform the choice of algorithms and metrics used to train agents; ranging from A* search to state-of-the-art deep reinforcement learning. We, further, show that the learning potential of state-of-the-art deep RL models does not seamlessly transfer from the benchmark environments to target ones without heavily tuning their hyperparameters, leading to linear scaling of the engineering efforts and computational cost with the number of target domains.
Combination of Multiple Global Descriptors for Image Retrieval
Venue: arXiv cs.CV 2020
Year: 2020

Recent studies in image retrieval task have shown that ensembling different models and combining multiple global descriptors lead to performance improvement. However, training different models for the ensemble is not only difficult but also inefficient with respect to time and memory. In this paper, we propose a novel framework that exploits multiple global descriptors to get an ensemble effect while it can be trained in an end-to-end manner. The proposed framework is flexible and expandable by the global descriptor, CNN backbone, loss, and dataset. Moreover, we investigate the effectiveness of combining multiple global descriptors with quantitative and qualitative analysis. Our extensive experiments show that the combined descriptor outperforms a single global descriptor, as it can utilize different types of feature properties. In the benchmark evaluation, the proposed framework achieves the state-of-the-art performance on the CARS196, CUB200-2011, In-shop Clothes, and Stanford Online Products on image retrieval tasks. Our model implementations and pretrained models are publicly available.
A trajectory-based framework for data-driven system analysis and control
Venue: arXiv cs.SY 2020
Year: 2020

The vector space of all input-output trajectories of a discrete-time linear time-invariant (LTI) system is spanned by time-shifts of a single measured trajectory, given that the respective input signal is persistently exciting. This fact, which was proven in the behavioral control framework, shows that a single measured trajectory can capture the full behavior of an LTI system and might therefore be used directly for system analysis and controller design, without explicitly identifying a model. In this paper, we translate the result from the behavioral context to the classical state-space control framework and we extend it to certain classes of nonlinear systems, which are linear in suitable input-output coordinates. Moreover, we show how this extension can be applied to the data-driven simulation problem, where we introduce kernel-methods to obtain a rich set of basis functions.
Lexical convergence and collective identities on Facebook
Venue: arXiv cs.SI 2020
Year: 2020

Recent studies, targeting Facebook, showed the tendency of users to interact with information adhering to their preferred narrative and to ignore dissenting information. Primarily driven by confirmation bias, users tend to join polarized clusters where they cooperate to reinforce a like-minded system of beliefs, thus facilitating fake news and misinformation cascades. To gain a deeper understanding of these phenomena, in this work we analyze the lexicons used by the communities of users emerging on Facebook around verified and unverified contents. We show how the lexical approach provides important insights about the kind of information processed by the two communities of users and about their overall sentiment. Furthermore, by focusing on comment threads, we observe a strong positive correlation between the lexical convergence of co-commenters and their number of interactions, which in turns suggests that such a trend could be a proxy for the emergence of collective identities and polarization in opinion dynamics.
Tree Search vs Optimization Approaches for Map Generation
Venue: arXiv cs.AI 2020
Year: 2020

Search-based procedural content generation uses stochastic global optimization algorithms to search for game content. However, standard tree search algorithms can be competitive with evolution on some optimization problems. We investigate the applicability of several tree search methods to level generation and compare them systematically with several optimization algorithms, including evolutionary algorithms. We compare them on three different game level generation problems: Binary, Zelda, and Sokoban. We introduce two new representations that can help tree search algorithms deal with the large branching factor of the generation problem. We find that in general, optimization algorithms clearly outperform tree search algorithms, but given the right problem representation certain tree search algorithms perform similarly to optimization algorithms, and in one particular problem, we see surprisingly strong results from MCTS.
Inverse Kinematics, Identification, RIC-based Control, and implementation of an Aerial Manipulator
Venue: arXiv cs.RO 2020
Year: 2020

This paper presents the inverse kinematic analysis and parameters identification of a novel aerial manipulation system. This system consists of 2-link manipulator attached to the bottom of a quadrotor. This new system presents a solution for the limitations found in the current quadrotor manipulation system. By deriving the inverse kinematics, one can design the controller such that the desired end effector position and orientation can be tracked. To study the feasibility of the proposed system, a quadrotor with high enough payload to add the 2-link manipulator is designed and constructed. Experimental setup of the system is introduced with an experiment to estimate the rotors parameters. Its parameters are identified to be used in the simulation and controller design of the proposed system. System dynamics are derived briefly based on Newton Euler Method. The controller of the proposed system is designed based on Robust Internal-loop Compensator (RIC) and compared to Fuzzy Model Reference Learning Control (FMRLC) technique which was previously designed and tested for the proposed system. These controllers are tested for provide system stability and trajectory tracking under the effect of picking as well as placing a payload and under the effect of changing the operating region. Simulation framework is implemented in MATLAB/SIMULINK environment. The simulation results indicate the effectiveness of the inverse kinematic analysis and the proposed control technique.
A Deep Dive into Understanding Tumor Foci Classification using Multiparametric MRI Based on Convolutional Neural Network
Venue: arXiv cs.CV 2020
Year: 2020

Deep learning models have had a great success in disease classifications using large data pools of skin cancer images or lung X-rays. However, data scarcity has been the roadblock of applying deep learning models directly on prostate multiparametric MRI (mpMRI). Although model interpretation has been heavily studied for natural images for the past few years, there has been a lack of interpretation of deep learning models trained on medical images. This work designs a customized workflow for the small and imbalanced data set of prostate mpMRI where features were extracted from a deep learning model and then analyzed by a traditional machine learning classifier. In addition, this work contributes to revealing how deep learning models interpret mpMRI for prostate cancer patients stratification.
Online Multi-target regression trees with stacked leaf models
Venue: arXiv cs.LG 2020
Year: 2020

One of the current challenges in machine learning is how to deal with data coming at increasing rates in data streams. New predictive learning strategies are needed to cope with the high throughput data and concept drift. One of the data stream mining tasks where new learning strategies are needed is multi-target regression, due to its applicability in a high number of real world problems. While reliable and effective learning strategies have been proposed for batch multi-target regression, few have been proposed for multi-target online learning in data streams. Besides, most of the existing solutions do not consider the occurrence of inter-target correlations when making predictions. In this work, we propose a novel online learning strategy for multi-target regression in data streams. The proposed strategy extends existing online decision tree learning algorithm to explore inter-target dependencies while making predictions. For such, the proposed strategy, called Stacked Single-target Hoeffding Tree (SST-HT), uses the inter-target dependencies as an additional information source to enhance predictive accuracy. Throughout an extensive experimental setup, we evaluate our proposal against state-of-the-art decision tree-based algorithms for online multi-target regression. According to the experimental results, SST-HT presents superior predictive accuracy, with a small increase in the processing time and memory requirements.
Learning Relational Representations with Auto-encoding Logic Programs
Venue: arXiv cs.LG 2020
Year: 2020

Deep learning methods capable of handling relational data have proliferated over the last years. In contrast to traditional relational learning methods that leverage first-order logic for representing such data, these deep learning methods aim at re-representing symbolic relational data in Euclidean spaces. They offer better scalability, but can only numerically approximate relational structures and are less flexible in terms of reasoning tasks supported. This paper introduces a novel framework for relational representation learning that combines the best of both worlds. This framework, inspired by the auto-encoding principle, uses first-order logic as a data representation language, and the mapping between the original and latent representation is done by means of logic programs instead of neural networks. We show how learning can be cast as a constraint optimisation problem for which existing solvers can be used. The use of logic as a representation language makes the proposed framework more accurate (as the representation is exact, rather than approximate), more flexible, and more interpretable than deep learning methods. We experimentally show that these latent representations are indeed beneficial in relational learning tasks.
LightChain: A DHT-based Blockchain for Resource Constrained Environments
Venue: arXiv cs.DC 2020
Year: 2020

As an append-only distributed database, blockchain is utilized in a vast variety of applications including the cryptocurrency and Internet-of-Things (IoT). The existing blockchain solutions have downsides in communication and storage efficiency, convergence to centralization, and consistency problems. In this paper, we propose LightChain, which is the first blockchain architecture that operates over a Distributed Hash Table (DHT) of participating peers. LightChain is a permissionless blockchain that provides addressable blocks and transactions within the network, which makes them efficiently accessible by all the peers. Each block and transaction is replicated within the DHT of peers and is retrieved in an on-demand manner. Hence, peers in LightChain are not required to retrieve or keep the entire blockchain. LightChain is fair as all of the participating peers have a uniform chance of being involved in the consensus regardless of their influence such as hashing power or stake. LightChain provides a deterministic fork-resolving strategy as well as a blacklisting mechanism, and it is secure against colluding adversarial peers attacking the availability and integrity of the system. We provide mathematical analysis and experimental results on scenarios involving 10K nodes to demonstrate the security and fairness of LightChain. As we experimentally show in this paper, compared to the mainstream blockchains like Bitcoin and Ethereum, LightChain requires around 66 times less per node storage, and is around 380 times faster on bootstrapping a new node to the system, while each LightChain node is rewarded equally likely for participating in the protocol.
Unsupervised Multi-modal Hashing for Cross-modal retrieval
Venue: arXiv cs.CV 2020
Year: 2020

With the advantage of low storage cost and high efficiency, hashing learning has received much attention in the domain of Big Data. In this paper, we propose a novel unsupervised hashing learning method to cope with this open problem to directly preserve the manifold structure by hashing. To address this problem, both the semantic correlation in textual space and the locally geometric structure in the visual space are explored simultaneously in our framework. Besides, the `2;1-norm constraint is imposed on the projection matrices to learn the discriminative hash function for each modality. Extensive experiments are performed to evaluate the proposed method on the three publicly available datasets and the experimental results show that our method can achieve superior performance over the state-of-the-art methods.