OpenRaiser/Intern-Atlas huggingface dataset:
MobiVSR: A Visual Speech Recognition Solution for Mobile Devices
Venue: arXiv cs.CL 2019
Year: 2019

Visual speech recognition (VSR) is the task of recognizing spoken language from video input only, without any audio. VSR has many applications as an assistive technology, especially if it could be deployed in mobile devices and embedded systems. The need of intensive computational resources and large memory footprint are two of the major obstacles in developing neural network models for VSR in a resource constrained environment. We propose a novel end-to-end deep neural network architecture for word level VSR called MobiVSR with a design parameter that aids in balancing the model's accuracy and parameter count. We use depthwise-separable 3D convolution for the first time in the domain of VSR and show how it makes our model efficient. MobiVSR achieves an accuracy of 73\% on a challenging Lip Reading in the Wild dataset with 6 times fewer parameters and 20 times lesser memory footprint than the current state of the art. MobiVSR can also be compressed to 6 MB by applying post training quantization.
Bayesian Optimized Continual Learning with Attention Mechanism
Venue: arXiv cs.LG 2019
Year: 2019

Though neural networks have achieved much progress in various applications, it is still highly challenging for them to learn from a continuous stream of tasks without forgetting. Continual learning, a new learning paradigm, aims to solve this issue. In this work, we propose a new model for continual learning, called Bayesian Optimized Continual Learning with Attention Mechanism (BOCL) that dynamically expands the network capacity upon the arrival of new tasks by Bayesian optimization and selectively utilizes previous knowledge (e.g. feature maps of previous tasks) via attention mechanism. Our experiments on variants of MNIST and CIFAR-100 demonstrate that our methods outperform the state-of-the-art in preventing catastrophic forgetting and fitting new tasks better.
An Incremental Dimensionality Reduction Method for Visualizing Streaming Multidimensional Data
Venue: arXiv cs.GR 2019
Year: 2019

Dimensionality reduction (DR) methods are commonly used for analyzing and visualizing multidimensional data. However, when data is a live streaming feed, conventional DR methods cannot be directly used because of their computational complexity and inability to preserve the projected data positions at previous time points. In addition, the problem becomes even more challenging when the dynamic data records have a varying number of dimensions as often found in real-world applications. This paper presents an incremental DR solution. We enhance an existing incremental PCA method in several ways to ensure its usability for visualizing streaming multidimensional data. First, we use geometric transformation and animation methods to help preserve a viewer's mental map when visualizing the incremental results. Second, to handle data dimension variants, we use an optimization method to estimate the projected data positions, and also convey the resulting uncertainty in the visualization. We demonstrate the effectiveness of our design with two case studies using real-world datasets.
Supervized Segmentation with Graph-Structured Deep Metric Learning
Venue: arXiv cs.LG 2019
Year: 2019

We present a fully-supervized method for learning to segment data structured by an adjacency graph. We introduce the graph-structured contrastive loss, a loss function structured by a ground truth segmentation. It promotes learning vertex embeddings which are homogeneous within desired segments, and have high contrast at their interface. Thus, computing a piecewise-constant approximation of such embeddings produces a graph-partition close to the objective segmentation. This loss is fully backpropagable, which allows us to learn vertex embeddings with deep learning algorithms. We evaluate our methods on a 3D point cloud oversegmentation task, defining a new state-of-the-art by a large margin. These results are based on the published work of Landrieu and Boussaha 2019.
UniquID: A Quest to Reconcile Identity Access Management and the Internet of Things
Venue: arXiv cs.CR 2019
Year: 2019

The Internet of Things (IoT) has caused a revolutionary paradigm shift in computer networking. After decades of human-centered routines, where devices were merely tools that enabled human beings to authenticate themselves and perform activities, we are now dealing with a device-centered paradigm: the devices themselves are actors, not just tools for people. Conventional identity access management (IAM) frameworks were not designed to handle the challenges of IoT. Trying to use traditional IAM systems to reconcile heterogeneous devices and complex federations of online services (e.g., IoT sensors and cloud computing solutions) adds a cumbersome architectural layer that can become hard to maintain and act as a single point of failure. In this paper, we propose UniquID, a blockchain-based solution that overcomes the need for centralized IAM architectures while providing scalability and robustness. We also present the experimental results of a proof-of-concept UniquID enrolment network, and we discuss two different use-cases that show the considerable value of a blockchain-based IAM.
Densifying Assumed-sparse Tensors: Improving Memory Efficiency and MPI Collective Performance during Tensor Accumulation for Parallelized Training of Neural Machine Translation Models
Venue: arXiv cs.LG 2019
Year: 2019

Neural machine translation - using neural networks to translate human language - is an area of active research exploring new neuron types and network topologies with the goal of dramatically improving machine translation performance. Current state-of-the-art approaches, such as the multi-head attention-based transformer, require very large translation corpuses and many epochs to produce models of reasonable quality. Recent attempts to parallelize the official TensorFlow "Transformer" model across multiple nodes have hit roadblocks due to excessive memory use and resulting out of memory errors when performing MPI collectives. This paper describes modifications made to the Horovod MPI-based distributed training framework to reduce memory usage for transformer models by converting assumed-sparse tensors to dense tensors, and subsequently replacing sparse gradient gather with dense gradient reduction. The result is a dramatic increase in scale-out capability, with CPU-only scaling tests achieving 91% weak scaling efficiency up to 1200 MPI processes (300 nodes), and up to 65% strong scaling efficiency up to 400 MPI processes (200 nodes) using the Stampede2 supercomputer.
Biologically plausible deep learning -- but how far can we go with shallow networks?
Venue: arXiv cs.NE 2019
Year: 2019

Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate models for biologically plausible variants of deep learning, typically defining success as reaching around 98% test accuracy on the MNIST data set. Here, we investigate how far we can go on digit (MNIST) and object (CIFAR10) classification with biologically plausible, local learning rules in a network with one hidden layer and a single readout layer. The hidden layer weights are either fixed (random or random Gabor filters) or trained with unsupervised methods (PCA, ICA or Sparse Coding) that can be implemented by local learning rules. The readout layer is trained with a supervised, local learning rule. We first implement these models with rate neurons. This comparison reveals, first, that unsupervised learning does not lead to better performance than fixed random projections or Gabor filters for large hidden layers. Second, networks with localized receptive fields perform significantly better than networks with all-to-all connectivity and can reach backpropagation performance on MNIST. We then implement two of the networks - fixed, localized, random & random Gabor filters in the hidden layer - with spiking leaky integrate-and-fire neurons and spike timing dependent plasticity to train the readout layer. These spiking models achieve > 98.2% test accuracy on MNIST, which is close to the performance of rate networks with one hidden layer trained with backpropagation. The performance of our shallow network models is comparable to most current biologically plausible models of deep learning. Furthermore, our results with a shallow spiking network provide an important reference and suggest the use of datasets other than MNIST for testing the performance of future models of biologically plausible deep learning.
Massive Autonomous UAV Path Planning: A Neural Network Based Mean-Field Game Theoretic Approach
Venue: arXiv cs.SY 2019
Year: 2019

This paper investigates the autonomous control of massive unmanned aerial vehicles (UAVs) for mission-critical applications (e.g., dispatching many UAVs from a source to a destination for firefighting). Achieving their fast travel and low motion energy without inter-UAV collision under wind perturbation is a daunting control task, which incurs huge communication energy for exchanging UAV states in real time. We tackle this problem by exploiting a mean-field game (MFG) theoretic control method that requires the UAV state exchanges only once at the initial source. Afterwards, each UAV can control its acceleration by locally solving two partial differential equations (PDEs), known as the Hamilton-Jacobi-Bellman (HJB) and Fokker-Planck-Kolmogorov (FPK) equations. This approach, however, brings about huge computation energy for solving the PDEs, particularly under multi-dimensional UAV states. We address this issue by utilizing a machine learning (ML) method where two separate ML models approximate the solutions of the HJB and FPK equations. These ML models are trained and exploited using an online gradient descent method with low computational complexity. Numerical evaluations validate that the proposed ML aided MFG theoretic algorithm, referred to as MFG learning control, is effective in collision avoidance with low communication energy and acceptable computation energy.
DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registration
Venue: arXiv cs.CV 2019
Year: 2019

We present DeepICP - a novel end-to-end learning-based 3D point cloud registration framework that achieves comparable registration accuracy to prior state-of-the-art geometric methods. Different from other keypoint based methods where a RANSAC procedure is usually needed, we implement the use of various deep neural network structures to establish an end-to-end trainable network. Our keypoint detector is trained through this end-to-end structure and enables the system to avoid the inference of dynamic objects, leverages the help of sufficiently salient features on stationary objects, and as a result, achieves high robustness. Rather than searching the corresponding points among existing points, the key contribution is that we innovatively generate them based on learned matching probabilities among a group of candidates, which can boost the registration accuracy. Our loss function incorporates both the local similarity and the global geometric constraints to ensure all above network designs can converge towards the right direction. We comprehensively validate the effectiveness of our approach using both the KITTI dataset and the Apollo-SouthBay dataset. Results demonstrate that our method achieves comparable or better performance than the state-of-the-art geometry-based methods. Detailed ablation and visualization analysis are included to further illustrate the behavior and insights of our network. The low registration error and high robustness of our method makes it attractive for substantial applications relying on the point cloud registration task.
Online Multistage Subset Maximization Problems
Venue: arXiv cs.DS 2019
Year: 2019

Numerous combinatorial optimization problems (knapsack, maximum-weight matching, etc.) can be expressed as \emph{subset maximization problems}: One is given a ground set $N=\{1,\dots,n\}$, a collection $\mathcal{F}\subseteq 2^N$ of subsets thereof such that $\emptyset\in\mathcal{F}$, and an objective (profit) function $p:\mathcal{F}\rightarrow\mathbb{R}_+$. The task is to choose a set $S\in\mathcal{F}$ that maximizes $p(S)$. We consider the \emph{multistage} version (Eisenstat et al., Gupta et al., both ICALP 2014) of such problems: The profit function $p_t$ (and possibly the set of feasible solutions $\mathcal{F}_t$) may change over time. Since in many applications changing the solution is costly, the task becomes to find a sequence of solutions that optimizes the trade-off between good per-time solutions and stable solutions taking into account an additional similarity bonus. As similarity measure for two consecutive solutions, we consider either the size of the intersection of the two solutions or the difference of $n$ and the Hamming distance between the two characteristic vectors. We study multistage subset maximization problems in the \emph{online} setting, that is, $p_t$ (along with possibly $\mathcal{F}_t$) only arrive one by one and, upon such an arrival, the online algorithm has to output the corresponding solution without knowledge of the future. We develop general techniques for online multistage subset maximization and thereby characterize those models (given by the type of data evolution and the type of similarity measure) that admit a constant-competitive online algorithm. When no constant competitive ratio is possible, we employ lookahead to circumvent this issue. When a constant competitive ratio is possible, we provide almost matching lower and upper bounds on the best achievable one.
Quantifying Teaching Behaviour in Robot Learning from Demonstration
Venue: arXiv cs.RO 2019
Year: 2019

Learning from demonstration allows for rapid deployment of robot manipulators to a great many tasks, by relying on a person showing the robot what to do rather than programming it. While this approach provides many opportunities, measuring, evaluating and improving the person's teaching ability has remained largely unexplored in robot manipulation research. To this end, a model for learning from demonstration is presented here which incorporates the teacher's understanding of, and influence on, the learner. The proposed model is used to clarify the teacher's objectives during learning from demonstration, providing new views on how teaching failures and efficiency can be defined. The benefit of this approach is shown in two experiments (N=30 and N=36, respectively), which highlight the difficulty teachers have in providing effective demonstrations, and show how ~169-180% improvement in teaching efficiency can be achieved through evaluation and feedback shaped by the proposed framework, relative to unguided teaching.
Breast Tumor Classification and Segmentation using Convolutional Neural Networks
Venue: arXiv cs.CV 2019
Year: 2019

Breast cancer is considered as the most fatal type of cancer among women worldwide and it is crucially important to be diagnosed at its early stages. In the current study, we aim to represent a fast and efficient framework which consists of two main parts:1- image classification, and 2- tumor region segmentation. At the initial stage, the images are classified into the two categories of normal and abnormal. Since the Deep Neural Networks have performed successfully in machine vision task, we would employ the convolutional neural networks for the classification of images. In the second stage, the suggested framework is to diagnose and segment the tumor in the mammography images. First, the mammography images are pre-processed by removing noise and artifacts, and then, segment the image using the level-set algorithm based on the spatial fuzzy c-means clustering. The proper initialization and optimal configuration have strong effects on the performance of the level-set segmentation. Thus, in our suggested framework, we have improved the level-set algorithm by utilizing the spatial fuzzy c-means clustering which ultimately results in a more precise segmentation. In order to evaluate the proposed approach, we conducted experiments using the Mammographic Image Analysis (MIAS) dataset. The tests have shown that the convolutional neural networks could achieve high accuracy in classification of images. Moreover, the improved level-set segmentation method, along with the fuzzy c-means clustering, could perfectly do the segmentation on the tumor area. The suggested method has classified the images with the accuracy of 78% and the AUC of 69%, which, as compared to the previous methods, is 2% more accurate and 6% better AUC; and has been able to extract the tumor area in a more precise way.
Check-It: A Plugin for Detecting and Reducing the Spread of Fake News and Misinformation on the Web
Venue: arXiv cs.SI 2019
Year: 2019

Over the past few years, we have been witnessing the rise of misinformation on the Web. People fall victims of fake news during their daily lives and assist their further propagation knowingly and inadvertently. There have been many initiatives that are trying to mitigate the damage caused by fake news, focusing on signals from either domain flag-lists, online social networks or artificial intelligence. In this work, we present Check-It, a system that combines, in an intelligent way, a variety of signals into a pipeline for fake news identification. Check-It is developed as a web browser plugin with the objective of efficient and timely fake news detection, respecting the user's privacy. Experimental results show that Check-It is able to outperform the state-of-the-art methods. On a dataset, consisting of 9 millions of articles labeled as fake and real, Check-It obtains classification accuracies that exceed 99%.
Primary User Localization and Online Radio Cartography via Structured Tensor Decomposition
Venue: arXiv cs.IT 2019
Year: 2019

Source localization and radio cartography using multi-way representation of spectrum is the subject of study in this paper. A joint matrix factorization and tensor decomposition problem is proposed and solved using an iterative algorithm. The multi-way measured spectrum is organized in a tensor and it is modeled by multiplication of a propagation tensor and a channel gain matrix. The tensor indicates the propagating power from each location and each frequency over time and the channel matrix links the propagating tensor to the sensed spectrum. We utilize sparsity and other intrinsic characteristics of spectrum to identify the solution of the proposed problem. Moreover, The online implementation of the proposed framework results in online radio cartography which is a powerful tool for efficient spectrum awareness and utilization. The simulation results show that our algorithm is a promising technique for dynamic primary user localization and online radio cartography.
Differential Recurrent Neural Network and its Application for Human Activity Recognition
Venue: arXiv cs.CV 2019
Year: 2019

The Long Short-Term Memory (LSTM) recurrent neural network is capable of processing complex sequential information since it utilizes special gating schemes for learning representations from long input sequences. It has the potential to model any sequential time-series data, where the current hidden state has to be considered in the context of the past hidden states. This property makes LSTM an ideal choice to learn the complex dynamics present in long sequences. Unfortunately, the conventional LSTMs do not consider the impact of spatio-temporal dynamics corresponding to the given salient motion patterns, when they gate the information that ought to be memorized through time. To address this problem, we propose a differential gating scheme for the LSTM neural network, which emphasizes on the change in information gain caused by the salient motions between the successive video frames. This change in information gain is quantified by Derivative of States (DoS), and thus the proposed LSTM model is termed as differential Recurrent Neural Network (dRNN). In addition, the original work used the hidden state at the last time-step to model the entire video sequence. Based on the energy profiling of DoS, we further propose to employ the State Energy Profile (SEP) to search for salient dRNN states and construct more informative representations. The effectiveness of the proposed model was demonstrated by automatically recognizing human actions from the real-world 2D and 3D single-person action datasets. We point out that LSTM is a special form of dRNN. As a result, we have introduced a new family of LSTMs. Our study is one of the first works towards demonstrating the potential of learning complex time-series representations via high-order derivatives of states.
Kesten-McKay law for random subensembles of Paley equiangular tight frames
Venue: arXiv cs.IT 2019
Year: 2019

We apply the method of moments to prove a recent conjecture of Haikin, Zamir and Gavish (2017) concerning the distribution of the singular values of random subensembles of Paley equiangular tight frames. Our analysis applies more generally to real equiangular tight frames of redundancy 2, and we suspect similar ideas will eventually produce more general results for arbitrary choices of redundancy.
Enabling Explainable Fusion in Deep Learning with Fuzzy Integral Neural Networks
Venue: arXiv cs.NE 2019
Year: 2019

Information fusion is an essential part of numerous engineering systems and biological functions, e.g., human cognition. Fusion occurs at many levels, ranging from the low-level combination of signals to the high-level aggregation of heterogeneous decision-making processes. While the last decade has witnessed an explosion of research in deep learning, fusion in neural networks has not observed the same revolution. Specifically, most neural fusion approaches are ad hoc, are not understood, are distributed versus localized, and/or explainability is low (if present at all). Herein, we prove that the fuzzy Choquet integral (ChI), a powerful nonlinear aggregation function, can be represented as a multi-layer network, referred to hereafter as ChIMP. We also put forth an improved ChIMP (iChIMP) that leads to a stochastic gradient descent-based optimization in light of the exponential number of ChI inequality constraints. An additional benefit of ChIMP/iChIMP is that it enables eXplainable AI (XAI). Synthetic validation experiments are provided and iChIMP is applied to the fusion of a set of heterogeneous architecture deep models in remote sensing. We show an improvement in model accuracy and our previously established XAI indices shed light on the quality of our data, model, and its decisions.
Structured Discriminative Tensor Dictionary Learning for Unsupervised Domain Adaptation
Venue: arXiv cs.CV 2019
Year: 2019

Unsupervised Domain Adaptation (UDA) addresses the problem of performance degradation due to domain shift between training and testing sets, which is common in computer vision applications. Most existing UDA approaches are based on vector-form data although the typical format of data or features in visual applications is multi-dimensional tensor. Besides, current methods, including the deep network approaches, assume that abundant labeled source samples are provided for training. However, the number of labeled source samples are always limited due to expensive annotation cost in practice, making sub-optimal performance been observed. In this paper, we propose to seek discriminative representation for multi-dimensional data by learning a structured dictionary in tensor space. The dictionary separates domain-specific information and class-specific information to guarantee the representation robust to domains. In addition, a pseudo-label estimation scheme is developed to combine with discriminant analysis in the algorithm iteration for avoiding the external classifier design. We perform extensive results on different datasets with limited source samples. Experimental results demonstrates that the proposed method outperforms the state-of-the-art approaches.