OpenRaiser/Intern-Atlas huggingface dataset:
Uncertain Trees: Dealing with Uncertain Inputs in Regression Trees
Venue: arXiv cs.LG 2018
Year: 2018

Tree-based ensemble methods, as Random Forests and Gradient Boosted Trees, have been successfully used for regression in many applications and research studies. Furthermore, these methods have been extended in order to deal with uncertainty in the output variable, using for example a quantile loss in Random Forests (Meinshausen, 2006). To the best of our knowledge, no extension has been provided yet for dealing with uncertainties in the input variables, even though such uncertainties are common in practical situations. We propose here such an extension by showing how standard regression trees optimizing a quadratic loss can be adapted and learned while taking into account the uncertainties in the inputs. By doing so, one no longer assumes that an observation lies into a single region of the regression tree, but rather that it belongs to each region with a certain probability. Experiments conducted on several data sets illustrate the good behavior of the proposed extension.
Real-time Action Recognition with Dissimilarity-based Training of Specialized Module Networks
Venue: arXiv cs.CV 2018
Year: 2018

This paper addresses the problem of real-time action recognition in trimmed videos, for which deep neural networks have defined the state-of-the-art performance in the recent literature. For attaining higher recognition accuracies with efficient computations, researchers have addressed the various aspects of limitations in the recognition pipeline. This includes network architecture, the number of input streams (where additional streams augment the color information), the cost function to be optimized, in addition to others. The literature has always aimed, though, at assigning the adopted network (or networks, in case of multiple streams) the task of recognizing the whole number of action classes in the dataset at hand. We propose to train multiple specialized module networks instead. Each module is trained to recognize a subset of the action classes. Towards this goal, we present a dissimilarity-based optimized procedure for distributing the action classes over the modules, which can be trained simultaneously offline. On two standard datasets--UCF-101 and HMDB-51--the proposed method demonstrates a comparable performance, that is superior in some aspects, to the state-of-the-art, and that satisfies the real-time constraint. We achieved 72.5\% accuracy on the challenging HMDB-51 dataset. By assigning fewer and unalike classes to each module network, this research paves the way to benefit from light-weight architectures without compromising recognition accuracy.
Asymptotic Gilbert-Varshamov bound on Frequency Hopping Sequences
Venue: arXiv cs.IT 2018
Year: 2018

Given a $q$-ary frequency hopping sequence set of length $n$ and size $M$ with Hamming correlation $H$, one can obtain a $q$-ary (nonlinear) cyclic code of length $n$ and size $nM$ with Hamming distance $n-H$. Thus, every upper bound on the size of a code from coding theory gives an upper bound on the size of a frequency hopping sequence set. Indeed, all upper bounds from coding theory have been converted to upper bounds on frequency hopping sequence sets (\cite{Ding09}). On the other hand, a lower bound from coding theory does not automatically produce a lower bound for frequency hopping sequence sets. In particular, the most important lower bound--the Gilbert-Varshamov bound in coding theory has not been transformed to frequency hopping sequence sets. The purpose of this paper is to convert the Gilbert-Varshamov bound in coding theory to frequency hopping sequence sets by establishing a connection between a special family of cyclic codes (which are called hopping cyclic codes in this paper) and frequency hopping sequence sets. We provide two proofs of the Gilbert-Varshamov bound. One is based on probabilistic method that requires advanced tool--martingale. This proof covers the whole rate region. The other proof is purely elementary but only covers part of the rate region.
Machine Learning in Network Centrality Measures: Tutorial and Outlook
Venue: arXiv cs.LG 2018
Year: 2018

Complex networks are ubiquitous to several Computer Science domains. Centrality measures are an important analysis mechanism to uncover vital elements of complex networks. However, these metrics have high computational costs and requirements that hinder their applications in large real-world networks. In this tutorial, we explain how the use of neural network learning algorithms can render the application of the metrics in complex networks of arbitrary size. Moreover, the tutorial describes how to identify the best configuration for neural network training and learning such for tasks, besides presenting an easy way to generate and acquire training data. We do so by means of a general methodology, using complex network models adaptable to any application. We show that a regression model generated by the neural network successfully approximates the metric values and therefore are a robust, effective alternative in real-world applications. The methodology and proposed machine learning model use only a fraction of time with respect to other approximation algorithms, which is crucial in complex network applications.
A Hitchhiker's Guide On Distributed Training of Deep Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

Deep learning has led to tremendous advancements in the field of Artificial Intelligence. One caveat however is the substantial amount of compute needed to train these deep learning models. Training a benchmark dataset like ImageNet on a single machine with a modern GPU can take upto a week, distributing training on multiple machines has been observed to drastically bring this time down. Recent work has brought down ImageNet training time to a time as low as 4 minutes by using a cluster of 2048 GPUs. This paper surveys the various algorithms and techniques used to distribute training and presents the current state of the art for a modern distributed training framework. More specifically, we explore the synchronous and asynchronous variants of distributed Stochastic Gradient Descent, various All Reduce gradient aggregation strategies and best practices for obtaining higher throughout and lower latency over a cluster such as mixed precision training, large batch training and gradient compression.
A Sweet Pepper Harvesting Robot for Protected Cropping Environments
Venue: arXiv cs.RO 2018
Year: 2018

Using robots to harvest sweet peppers in protected cropping environments has remained unsolved despite considerable effort by the research community over several decades. In this paper, we present the robotic harvester, Harvey, designed for sweet peppers in protected cropping environments that achieved a 76.5% success rate (within a modified scenario) which improves upon our prior work which achieved 58% and related sweet pepper harvesting work which achieved 33\%. This improvement was primarily achieved through the introduction of a novel peduncle segmentation system using an efficient deep convolutional neural network, in conjunction with 3D post-filtering to detect the critical cutting location. We benchmark the peduncle segmentation against prior art demonstrating a considerable improvement in performance with an F_1 score of 0.564 compared to 0.302. The robotic harvester uses a perception pipeline to detect a target sweet pepper and an appropriate grasp and cutting pose used to determine the trajectory of a multi-modal harvesting tool to grasp the sweet pepper and cut it from the plant. A novel decoupling mechanism enables the gripping and cutting operations to be performed independently. We perform an in-depth analysis of the full robotic harvesting system to highlight bottlenecks and failure points that future work could address.
A Distributed Epigenetic Shape Formation and Regeneration Algorithm for a Swarm of Robots
Venue: arXiv cs.RO 2018
Year: 2018

Living cells exhibit both growth and regeneration of body tissues. Epigenetic Tracking (ET), models this growth and regenerative qualities of living cells and has been used to generate complex 2D and 3D shapes. In this paper, we present an ET based algorithm that aids a swarm of identically-programmed robots to form arbitrary shapes and regenerate them when cut. The algorithm works in a distributed manner using only local interactions and computations without any central control and aids the robots to form the shape in a triangular lattice structure. In case of damage or splitting of the shape, it helps each set of the remaining robots to regenerate and position themselves to build scaled down versions of the original shape. The paper presents the shapes formed and regenerated by the algorithm using the Kilombo simulator.
Evolutionary Self-Expressive Models for Subspace Clustering
Venue: arXiv cs.CV 2018
Year: 2018

The problem of organizing data that evolves over time into clusters is encountered in a number of practical settings. We introduce evolutionary subspace clustering, a method whose objective is to cluster a collection of evolving data points that lie on a union of low-dimensional evolving subspaces. To learn the parsimonious representation of the data points at each time step, we propose a non-convex optimization framework that exploits the self-expressiveness property of the evolving data while taking into account representation from the preceding time step. To find an approximate solution to the aforementioned non-convex optimization problem, we develop a scheme based on alternating minimization that both learns the parsimonious representation as well as adaptively tunes and infers a smoothing parameter reflective of the rate of data evolution. The latter addresses a fundamental challenge in evolutionary clustering -- determining if and to what extent one should consider previous clustering solutions when analyzing an evolving data collection. Our experiments on both synthetic and real-world datasets demonstrate that the proposed framework outperforms state-of-the-art static subspace clustering algorithms and existing evolutionary clustering schemes in terms of both accuracy and running time, in a range of scenarios.
Parametric model order reduction and its application to inverse analysis of large nonlinear coupled cardiac problems
Venue: arXiv cs.CE 2018
Year: 2018

Predictive high-fidelity finite element simulations of human cardiac mechanics co\-mmon\-ly require a large number of structural degrees of freedom. Additionally, these models are often coupled with lumped-parameter models of hemodynamics. High computational demands, however, slow down model calibration and therefore limit the use of cardiac simulations in clinical practice. As cardiac models rely on several patient-specific parameters, just one solution corresponding to one specific parameter set does not at all meet clinical demands. Moreover, while solving the nonlinear problem, 90\% of the computation time is spent solving linear systems of equations. We propose a novel approach to reduce only the structural dimension of the monolithically coupled structure-windkessel system by projection onto a lower-dimensional subspace. We obtain a good approximation of the displacement field as well as of key scalar cardiac outputs even with very few reduced degrees of freedom while achieving considerable speedups. For subspace generation, we use proper orthogonal decomposition of displacement snapshots. To incorporate changes in the parameter set into our reduced order model, we provide a comparison of subspace interpolation methods. We further show how projection-based model order reduction can be easily integrated into a gradient-based optimization and demonstrate its performance in a real-world multivariate inverse analysis scenario. Using the presented projection-based model order reduction approach can significantly speed up model personalization and could be used for many-query tasks in a clinical setting.
PartsNet: A Unified Deep Network for Automotive Engine Precision Parts Defect Detection
Venue: arXiv cs.CV 2018
Year: 2018

Defect detection is a basic and essential task in automatic parts production, especially for automotive engine precision parts. In this paper, we propose a new idea to construct a deep convolutional network combining related knowledge of feature processing and the representation ability of deep learning. Our algorithm consists of a pixel-wise segmentation Deep Neural Network (DNN) and a feature refining network. The fully convolutional DNN is presented to learn basic features of parts defects. After that, several typical traditional methods which are used to refine the segmentation results are transformed into convolutional manners and integrated. We assemble these methods as a shallow network with fixed weights and empirical thresholds. These thresholds are then released to enhance its adaptation ability and realize end-to-end training. Testing results on different datasets show that the proposed method has good portability and outperforms the state-of-the-art algorithms.
Ruuh: A Deep Learning Based Conversational Social Agent
Venue: arXiv cs.CL 2018
Year: 2018

Dialogue systems and conversational agents are becoming increasingly popular in the modern society but building an agent capable of holding intelligent conversation with its users is a challenging problem for artificial intelligence. In this demo, we demonstrate a deep learning based conversational social agent called "Ruuh" (facebook.com/Ruuh) designed by a team at Microsoft India to converse on a wide range of topics. Ruuh needs to think beyond the utilitarian notion of merely generating "relevant" responses and meet a wider range of user social needs, like expressing happiness when user's favorite team wins, sharing a cute comment on showing the pictures of the user's pet and so on. The agent also needs to detect and respond to abusive language, sensitive topics and trolling behavior of the users. Many of these problems pose significant research challenges which will be demonstrated in our demo. Our agent has interacted with over 2 million real world users till date which has generated over 150 million user conversations.
Symmetry and Stability of Homogenuous Flocks. A Position Paper
Venue: arXiv cs.SY 2018
Year: 2018

The study of the movement of flocks, whether biological or technological is motivated by the desire to understand the capability of coherent motion of a large number of agents that only receive very limited information. In a biological flock a large group of animals seek their course while moving in a more or less fixed formation. It seems reasonable that the immediate course is determined by leaders at the boundary of the flock. The others follow: what is their algorithm? The most popular technological application consists of cars on a one-lane road. The light turns green and the lead car accelerates. What is the efficient algorithm for the others to closely follow without accidents? In this position paper we present some general questions from a more fundamental point of view. We believe that the time is right to solve many of these questions: they are within our reach.
A Convergence indicator for Multi-Objective Optimisation Algorithms
Venue: arXiv cs.NE 2018
Year: 2018

The algorithms of multi-objective optimisation had a relative growth in the last years. Thereby, it's requires some way of comparing the results of these. In this sense, performance measures play a key role. In general, it's considered some properties of these algorithms such as capacity, convergence, diversity or convergence-diversity. There are some known measures such as generational distance (GD), inverted generational distance (IGD), hypervolume (HV), Spread($Δ$), Averaged Hausdorff distance ($Δ_p$), R2-indicator, among others. In this paper, we focuses on proposing a new indicator to measure convergence based on the traditional formula for Shannon entropy. The main features about this measure are: 1) It does not require tho know the true Pareto set and 2) Medium computational cost when compared with Hypervolume.
A Learning Approach To Wireless Information and Power Transfer Signal and System Design
Venue: arXiv cs.IT 2018
Year: 2018

The end-to-end learning of Simultaneous Wireless Information and Power Transfer (SWIPT) over a noisy channel is studied. Adopting a nonlinear model for the energy harvester (EH) at the receiver, a joint optimization of the transmitter and the receiver is implemented using Deep Neural Network (DNN)-based autoencoders. Modulation constellations for different levels of "power" and "information rate" demand at the receiver are obtained. The numerically optimized signal constellations are inline with the previous theoretical results. In particular, it is observed that as the receiver energy demand increases, all but one of the modulation symbols are concentrated around the origin and the other symbol is shot away from the origin along either the real or imaginary subchannel.
Dreaming neural networks: forgetting spurious memories and reinforcing pure ones
Venue: arXiv cs.NE 2018
Year: 2018

The standard Hopfield model for associative neural networks accounts for biological Hebbian learning and acts as the harmonic oscillator for pattern recognition, however its maximal storage capacity is $α\sim 0.14$, far from the theoretical bound for symmetric networks, i.e. $α=1$. Inspired by sleeping and dreaming mechanisms in mammal brains, we propose an extension of this model displaying the standard on-line (awake) learning mechanism (that allows the storage of external information in terms of patterns) and an off-line (sleep) unlearning$\&$consolidating mechanism (that allows spurious-pattern removal and pure-pattern reinforcement): this obtained daily prescription is able to saturate the theoretical bound $α=1$, remaining also extremely robust against thermal noise. Both neural and synaptic features are analyzed both analytically and numerically. In particular, beyond obtaining a phase diagram for neural dynamics, we focus on synaptic plasticity and we give explicit prescriptions on the temporal evolution of the synaptic matrix. We analytically prove that our algorithm makes the Hebbian kernel converge with high probability to the projection matrix built over the pure stored patterns. Furthermore, we obtain a sharp and explicit estimate for the "sleep rate" in order to ensure such a convergence. Finally, we run extensive numerical simulations (mainly Monte Carlo sampling) to check the approximations underlying the analytical investigations (e.g., we developed the whole theory at the so called replica-symmetric level, as standard in the Amit-Gutfreund-Sompolinsky reference framework) and possible finite-size effects, finding overall full agreement with the theory.
Leveraging Gaussian Process and Voting-Empowered Many-Objective Evaluation for Fault Identification
Venue: arXiv cs.CE 2018
Year: 2018

Using piezoelectric impedance/admittance sensing for structural health monitoring is promising, owing to the simplicity in circuitry design as well as the high-frequency interrogation capability. The actual identification of fault location and severity using impedance/admittance measurements, nevertheless, remains to be an extremely challenging task. A first-principle based structural model using finite element discretization requires high dimensionality to characterize the high-frequency response. As such, direct inversion using the sensitivity matrix usually yields an under-determined problem. Alternatively, the identification problem may be cast into an optimization framework in which fault parameters are identified through repeated forward finite element analysis which however is oftentimes computationally prohibitive. This paper presents an efficient data-assisted optimization approach for fault identification without using finite element model iteratively. We formulate a many-objective optimization problem to identify fault parameters, where response surfaces of impedance measurements are constructed through Gaussian process-based calibration. To balance between solution diversity and convergence, an -dominance enabled many-objective simulated annealing algorithm is established. As multiple solutions are expected, a voting score calculation procedure is developed to further identify those solutions that yield better implications regarding structural health condition. The effectiveness of the proposed approach is demonstrated by systematic numerical and experimental case studies.
Compressive Sampling Approach for Image Acquisition with Lensless Endoscope
Venue: arXiv cs.CV 2018
Year: 2018

The lensless endoscope is a promising device designed to image tissues in vivo at the cellular scale. The traditional acquisition setup consists in raster scanning during which the focused light beam from the optical fiber illuminates sequentially each pixel of the field of view (FOV). The calibration step to focus the beam and the sampling scheme both take time. In this preliminary work, we propose a scanning method based on compressive sampling theory. The method does not rely on a focused beam but rather on the random illumination patterns generated by the single-mode fibers. Experiments are performed on synthetic data for different compression rates (from 10 to 100% of the FOV).
Estimating Traffic Conditions At Metropolitan Scale Using Traffic Flow Theory
Venue: arXiv cs.OH 2018
Year: 2018

The rapid urbanization and increasing traffic have serious social, economic, and environmental impact on metropolitan areas worldwide. It is of a great importance to understand the complex interplay of road networks and traffic conditions. The authors propose a novel framework to estimate traffic conditions at the metropolitan scale using GPS traces. Their approach begins with an initial estimation of network travel times by solving a convex optimization program based on traffic flow theory. Then, they iteratively refine the estimated network travel times and vehicle traversed paths. Last, the authors perform a bilevel optimization process to estimate traffic conditions on road segments that are not covered by GPS data. The evaluation and comparison of the authors' approach over two state-of-the-art methods show up to 96.57% relative improvements. The authors have further conducted field tests by coupling road networks of San Francisco and Beijing with real-world GIS data, which involve 128,701 nodes, 148,899 road segments, and over 26 million GPS traces.
Multi-label Multi-task Deep Learning for Behavioral Coding
Venue: arXiv cs.CL 2018
Year: 2018

We propose a methodology for estimating human behaviors in psychotherapy sessions using mutli-label and multi-task learning paradigms. We discuss the problem of behavioral coding in which data of human interactions is the annotated with labels to describe relevant human behaviors of interest. We describe two related, yet distinct, corpora consisting of therapist client interactions in psychotherapy sessions. We experimentally compare the proposed learning approaches for estimating behaviors of interest in these datasets. Specifically, we compare single and multiple label learning approaches, single and multiple task learning approaches, and evaluate the performance of these approaches when incorporating turn context. We demonstrate the prediction performance gains which can be achieved by using the proposed paradigms and discuss the insights these models provide into these complex interactions.