OpenRaiser/Intern-Atlas huggingface dataset:
MADRaS : Multi Agent Driving Simulator
Venue: arXiv cs.RO 2020
Year: 2020

In this work, we present MADRaS, an open-source multi-agent driving simulator for use in the design and evaluation of motion planning algorithms for autonomous driving. MADRaS provides a platform for constructing a wide variety of highway and track driving scenarios where multiple driving agents can train for motion planning tasks using reinforcement learning and other machine learning algorithms. MADRaS is built on TORCS, an open-source car-racing simulator. TORCS offers a variety of cars with different dynamic properties and driving tracks with different geometries and surface properties. MADRaS inherits these functionalities from TORCS and introduces support for multi-agent training, inter-vehicular communication, noisy observations, stochastic actions, and custom traffic cars whose behaviours can be programmed to simulate challenging traffic conditions encountered in the real world. MADRaS can be used to create driving tasks whose complexities can be tuned along eight axes in well-defined steps. This makes it particularly suited for curriculum and continual learning. MADRaS is lightweight and it provides a convenient OpenAI Gym interface for independent control of each car. Apart from the primitive steering-acceleration-brake control mode of TORCS, MADRaS offers a hierarchical track-position -- speed control that can potentially be used to achieve better generalization. MADRaS uses multiprocessing to run each agent as a parallel process for efficiency and integrates well with popular reinforcement learning libraries like RLLib.
Understanding the Predictability of Gesture Parameters from Speech and their Perceptual Importance
Venue: arXiv cs.HC 2020
Year: 2020

Gesture behavior is a natural part of human conversation. Much work has focused on removing the need for tedious hand-animation to create embodied conversational agents by designing speech-driven gesture generators. However, these generators often work in a black-box manner, assuming a general relationship between input speech and output motion. As their success remains limited, we investigate in more detail how speech may relate to different aspects of gesture motion. We determine a number of parameters characterizing gesture, such as speed and gesture size, and explore their relationship to the speech signal in a two-fold manner. First, we train multiple recurrent networks to predict the gesture parameters from speech to understand how well gesture attributes can be modeled from speech alone. We find that gesture parameters can be partially predicted from speech, and some parameters, such as path length, being predicted more accurately than others, like velocity. Second, we design a perceptual study to assess the importance of each gesture parameter for producing motion that people perceive as appropriate for the speech. Results show that a degradation in any parameter was viewed negatively, but some changes, such as hand shape, are more impactful than others. A video summarization can be found at https://youtu.be/aw6-_5kmLjY.
Multiobjectivization of Local Search: Single-Objective Optimization Benefits From Multi-Objective Gradient Descent
Venue: arXiv cs.NE 2020
Year: 2020

Multimodality is one of the biggest difficulties for optimization as local optima are often preventing algorithms from making progress. This does not only challenge local strategies that can get stuck. It also hinders meta-heuristics like evolutionary algorithms in convergence to the global optimum. In this paper we present a new concept of gradient descent, which is able to escape local traps. It relies on multiobjectivization of the original problem and applies the recently proposed and here slightly modified multi-objective local search mechanism MOGSA. We use a sophisticated visualization technique for multi-objective problems to prove the working principle of our idea. As such, this work highlights the transfer of new insights from the multi-objective to the single-objective domain and provides first visual evidence that multiobjectivization can link single-objective local optima in multimodal landscapes.
Generalized Centrality Aggregation and Exclusive Centrality
Venue: arXiv cs.SI 2020
Year: 2020

There are several applications that benefit from a definition of centrality which is applicable to sets of vertices, rather than individual vertices. However, existing definitions might not be able to help us in answering several network analysis questions. In this paper, we study generalizing aggregation of centralities of individual vertices, to the centrality of the set consisting of these vertices. In particular, we propose exclusive betweenness centrality, defined as the number of shortest paths passing over exactly one of the vertices in the set, and discuss how this can be useful in determining the proper center of a network. We mathematically formulate the relationship between exclusive betweenness centrality and the existing notions of set centrality, and use this relation to present an exact algorithm for computing exclusive betweenness centrality. Since it is usually practically intractable to compute exact centrality scores for large real-world networks, we also present approximate algorithms for estimating exclusive betweenness centrality. In the end, we evaluate the empirical efficiency of exclusive betweenness centrality computation over several real-world networks. Moreover, we empirically study the correlations between exclusive betweenness centrality and the existing set centrality notions.
Semi-Supervised Learning for Multi-Task Scene Understanding by Neural Graph Consensus
Venue: arXiv cs.CV 2020
Year: 2020

We address the challenging problem of semi-supervised learning in the context of multiple visual interpretations of the world by finding consensus in a graph of neural networks. Each graph node is a scene interpretation layer, while each edge is a deep net that transforms one layer at one node into another from a different node. During the supervised phase edge networks are trained independently. During the next unsupervised stage edge nets are trained on the pseudo-ground truth provided by consensus among multiple paths that reach the nets' start and end nodes. These paths act as ensemble teachers for any given edge and strong consensus is used for high-confidence supervisory signal. The unsupervised learning process is repeated over several generations, in which each edge becomes a "student" and also part of different ensemble "teachers" for training other students. By optimizing such consensus between different paths, the graph reaches consistency and robustness over multiple interpretations and generations, in the face of unknown labels. We give theoretical justifications of the proposed idea and validate it on a large dataset. We show how prediction of different representations such as depth, semantic segmentation, surface normals and pose from RGB input could be effectively learned through self-supervised consensus in our graph. We also compare to state-of-the-art methods for multi-task and semi-supervised learning and show superior performance.
Semantics-Guided Clustering with Deep Progressive Learning for Semi-Supervised Person Re-identification
Venue: arXiv cs.CV 2020
Year: 2020

Person re-identification (re-ID) requires one to match images of the same person across camera views. As a more challenging task, semi-supervised re-ID tackles the problem that only a number of identities in training data are fully labeled, while the remaining are unlabeled. Assuming that such labeled and unlabeled training data share disjoint identity labels, we propose a novel framework of Semantics-Guided Clustering with Deep Progressive Learning (SGC-DPL) to jointly exploit the above data. By advancing the proposed Semantics-Guided Affinity Propagation (SG-AP), we are able to assign pseudo-labels to selected unlabeled data in a progressive fashion, under the semantics guidance from the labeled ones. As a result, our approach is able to augment the labeled training data in the semi-supervised setting. Our experiments on two large-scale person re-ID benchmarks demonstrate the superiority of our SGC-DPL over state-of-the-art methods across different degrees of supervision. In extension, the generalization ability of our SGC-DPL is also verified in other tasks like vehicle re-ID or image retrieval with the semi-supervised setting.
MM-Hand: 3D-Aware Multi-Modal Guided Hand Generative Network for 3D Hand Pose Synthesis
Venue: arXiv cs.CV 2020
Year: 2020

Estimating the 3D hand pose from a monocular RGB image is important but challenging. A solution is training on large-scale RGB hand images with accurate 3D hand keypoint annotations. However, it is too expensive in practice. Instead, we have developed a learning-based approach to synthesize realistic, diverse, and 3D pose-preserving hand images under the guidance of 3D pose information. We propose a 3D-aware multi-modal guided hand generative network (MM-Hand), together with a novel geometry-based curriculum learning strategy. Our extensive experimental results demonstrate that the 3D-annotated images generated by MM-Hand qualitatively and quantitatively outperform existing options. Moreover, the augmented data can consistently improve the quantitative performance of the state-of-the-art 3D hand pose estimators on two benchmark datasets. The code will be available at https://github.com/ScottHoang/mm-hand.
Deep FPF: Gain function approximation in high-dimensional setting
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we present a novel approach to approximate the gain function of the feedback particle filter (FPF). The exact gain function is the solution of a Poisson equation involving a probability-weighted Laplacian. The numerical problem is to approximate the exact gain function using only finitely many particles sampled from the probability distribution. Inspired by the recent success of the deep learning methods, we represent the gain function as a gradient of the output of a neural network. Thereupon considering a certain variational formulation of the Poisson equation, an optimization problem is posed for learning the weights of the neural network. A stochastic gradient algorithm is described for this purpose. The proposed approach has two significant properties/advantages: (i) The stochastic optimization algorithm allows one to process, in parallel, only a batch of samples (particles) ensuring good scaling properties with the number of particles; (ii) The remarkable representation power of neural networks means that the algorithm is potentially applicable and useful to solve high-dimensional problems. We numerically establish these two properties and provide extensive comparison to the existing approaches.
Random Access with Layered Preambles based on NOMA for Two Different Types of Devices in MTC
Venue: arXiv cs.IT 2020
Year: 2020

In machine-type communication (MTC), random access has been employed for a number of devices and sensors to access uplink channels using a pool of preambles. To support different priorities due to various quality-of-service (QoS) requirements, random access can be generalized with multiple pools, which may result in low spectral efficiency. In this paper, for high spectral efficiency, random access with layered preambles (RALP) is proposed to support devices with two different priorities based on the notion of power-domain non-orthogonal multiple access (NOMA). In RALP, two groups of devices, namely type-1 and type-2 devices, are supported with different priorities, where type-1 devices have higher priority than type-2 devices. Closed-form expressions are derived for the detection performance of preambles transmitted by type-1 devices, which can be used for a certain performance guarantee of type-1 devices of high priority. Low-complexity preamble detection methods are also discussed.
3D-Aided Data Augmentation for Robust Face Understanding
Venue: arXiv cs.CV 2020
Year: 2020

Data augmentation has been highly effective in narrowing the data gap and reducing the cost for human annotation, especially for tasks where ground truth labels are difficult and expensive to acquire. In face recognition, large pose and illumination variation of face images has been a key factor for performance degradation. However, human annotation for the various face understanding tasks including face landmark localization, face attributes classification and face recognition under these challenging scenarios are highly costly to acquire. Therefore, it would be desirable to perform data augmentation for these cases. But simple 2D data augmentation techniques on the image domain are not able to satisfy the requirement of these challenging cases. As such, 3D face modeling, in particular, single image 3D face modeling, stands a feasible solution for these challenging conditions beyond 2D based data augmentation. To this end, we propose a method that produces realistic 3D augmented images from multiple viewpoints with different illumination conditions through 3D face modeling, each associated with geometrically accurate face landmarks, attributes and identity information. Experiments demonstrate that the proposed 3D data augmentation method significantly improves the performance and robustness of various face understanding tasks while achieving state-of-arts on multiple benchmarks.
Bounding Boxes Are All We Need: Street View Image Classification via Context Encoding of Detected Buildings
Venue: arXiv cs.CV 2020
Year: 2020

Street view images classification aiming at urban land use analysis is difficult because the class labels (e.g., commercial area), are concepts with higher abstract level compared to the ones of general visual tasks (e.g., persons and cars). Therefore, classification models using only visual features often fail to achieve satisfactory performance. In this paper, a novel approach based on a "Detector-Encoder-Classifier" framework is proposed. Instead of using visual features of the whole image directly as common image-level models based on convolutional neural networks (CNNs) do, the proposed framework firstly obtains the bounding boxes of buildings in street view images from a detector. Their contextual information such as the co-occurrence patterns of building classes and their layout are then encoded into metadata by the proposed algorithm "CODING" (Context encOding of Detected buildINGs). Finally, these bounding box metadata are classified by a recurrent neural network (RNN). In addition, we made a dual-labeled dataset named "BEAUTY" (Building dEtection And Urban funcTional-zone portraYing) of 19,070 street view images and 38,857 buildings based on the existing BIC GSV [1]. The dataset can be used not only for street view image classification, but also for multi-class building detection. Experiments on "BEAUTY" show that the proposed approach achieves a 12.65% performance improvement on macro-precision and 12% on macro-recall over image-level CNN based models. Our code and dataset are available at https://github.com/kyle-one/Context-Encoding-of-Detected-Buildings/
Personality Trait Detection Using Bagged SVM over BERT Word Embedding Ensembles
Venue: arXiv cs.CL 2020
Year: 2020

Recently, the automatic prediction of personality traits has received increasing attention and has emerged as a hot topic within the field of affective computing. In this work, we present a novel deep learning-based approach for automated personality detection from text. We leverage state of the art advances in natural language understanding, namely the BERT language model to extract contextualized word embeddings from textual data for automated author personality detection. Our primary goal is to develop a computationally efficient, high-performance personality prediction model which can be easily used by a large number of people without access to huge computation resources. Our extensive experiments with this ideology in mind, led us to develop a novel model which feeds contextualized embeddings along with psycholinguistic features toa Bagged-SVM classifier for personality trait prediction. Our model outperforms the previous state of the art by 1.04% and, at the same time is significantly more computationally efficient to train. We report our results on the famous gold standard Essays dataset for personality detection.
HCI Models for Digital Musical Instruments: Methodologies for Rigorous Testing of Digital Musical Instruments
Venue: arXiv cs.HC 2020
Year: 2020

Here we present an analysis of literature relating to the evaluation methodologies of Digital Musical Instruments (DMIs) derived from the field of Human-Computer Interaction (HCI). We then apply choice aspects from these existing evaluation models and apply them to an optimized evaluation for assessing new DMIs.
Perplexity-free Parametric t-SNE
Venue: arXiv cs.LG 2020
Year: 2020

The t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm is a ubiquitously employed dimensionality reduction (DR) method. Its non-parametric nature and impressive efficacy motivated its parametric extension. It is however bounded to a user-defined perplexity parameter, restricting its DR quality compared to recently developed multi-scale perplexity-free approaches. This paper hence proposes a multi-scale parametric t-SNE scheme, relieved from the perplexity tuning and with a deep neural network implementing the mapping. It produces reliable embeddings with out-of-sample extensions, competitive with the best perplexity adjustments in terms of neighborhood preservation on multiple data sets.
Computational Separation Between Convolutional and Fully-Connected Networks
Venue: arXiv cs.LG 2020
Year: 2020

Convolutional neural networks (CNN) exhibit unmatched performance in a multitude of computer vision tasks. However, the advantage of using convolutional networks over fully-connected networks is not understood from a theoretical perspective. In this work, we show how convolutional networks can leverage locality in the data, and thus achieve a computational advantage over fully-connected networks. Specifically, we show a class of problems that can be efficiently solved using convolutional networks trained with gradient-descent, but at the same time is hard to learn using a polynomial-size fully-connected network.
TRIX: Low-Skew Pulse Propagation for Fault-Tolerant Hardware
Venue: arXiv cs.DC 2020
Year: 2020

The vast majority of hardware architectures use a carefully timed reference signal to clock their computational logic. However, standard distribution solutions are not fault-tolerant. In this work, we present a simple grid structure as a more reliable clock propagation method and study it by means of simulation experiments. Fault-tolerance is achieved by forwarding clock pulses on arrival of the second of three incoming signals from the previous layer. A key question is how well neighboring grid nodes are synchronized, even without faults. Analyzing the clock skew under typical-case conditions is highly challenging. Because the forwarding mechanism involves taking the median, standard probabilistic tools fail, even when modeling link delays just by unbiased coin flips. Our statistical approach provides substantial evidence that this system performs surprisingly well. Specifically, in an "infinitely wide" grid of height~$H$, the delay at a pre-selected node exhibits a standard deviation of $O(H^{1/4})$ ($\approx 2.7$ link delay uncertainties for $H=2000$) and skew between adjacent nodes of $o(\log \log H)$ ($\approx 0.77$ link delay uncertainties for $H=2000$). We conclude that the proposed system is a very promising clock distribution method. This leads to the open problem of a stochastic explanation of the tight concentration of delays and skews. More generally, we believe that understanding our very simple abstraction of the system is of mathematical interest in its own right.
Spiking Neural Networks Through the Lens of Streaming Algorithms
Venue: arXiv cs.DC 2020
Year: 2020

We initiate the study of biological neural networks from the perspective of streaming algorithms. Like computers, human brains suffer from memory limitations which pose a significant obstacle when processing large scale and dynamically changing data. In computer science, these challenges are captured by the well-known streaming model, which can be traced back to Munro and Paterson `78 and has had significant impact in theory and beyond. In the classical streaming setting, one must compute some function $f$ of a stream of updates $\mathcal{S} = \{u_1,\ldots,u_m\}$, given restricted single-pass access to the stream. The primary complexity measure is the space used by the algorithm. We take the first steps towards understanding the connection between streaming and neural algorithms. On the upper bound side, we design neural algorithms based on known streaming algorithms for fundamental tasks, including distinct elements, approximate median, heavy hitters, and more. The number of neurons in our neural solutions almost matches the space bounds of the corresponding streaming algorithms. As a general algorithmic primitive, we show how to implement the important streaming technique of linear sketching efficient in spiking neural networks. On the lower bound side, we give a generic reduction, showing that any space-efficient spiking neural network can be simulated by a space-efficiently streaming algorithm. This reduction lets us translate streaming-space lower bounds into nearly matching neural-space lower bounds, establishing a close connection between these two models.
Privately Answering Counting Queries with Generalized Gaussian Mechanisms
Venue: arXiv cs.DS 2020
Year: 2020

We consider the problem of answering $k$ counting (i.e. sensitivity-1) queries about a database with $(ε, δ)$-differential privacy. We give a mechanism such that if the true answers to the queries are the vector $d$, the mechanism outputs answers $\tilde{d}$ with the $\ell_\infty$-error guarantee: $$\mathcal{E}\left[||\tilde{d} - d||_\infty\right] = O\left(\frac{\sqrt{k \log \log \log k \log(1/δ)}}ε\right).$$ This reduces the multiplicative gap between the best known upper and lower bounds on $\ell_\infty$-error from $O(\sqrt{\log \log k})$ to $O(\sqrt{\log \log \log k})$. Our main technical contribution is an analysis of the family of mechanisms of the following form for answering counting queries: Sample $x$ from a \textit{Generalized Gaussian}, i.e. with probability proportional to $\exp(-(||x||_p/σ)^p)$, and output $\tilde{d} = d + x$. This family of mechanisms offers a tradeoff between $\ell_1$ and $\ell_\infty$-error guarantees and may be of independent interest. For $p = O(\log \log k)$, this mechanism already matches the previous best known $\ell_\infty$-error bound. We arrive at our main result by composing this mechanism for $p = O(\log \log \log k)$ with the sparse vector mechanism, generalizing a technique of Steinke and Ullman.
A New Mask R-CNN Based Method for Improved Landslide Detection
Venue: arXiv cs.CV 2020
Year: 2020

This paper presents a novel method of landslide detection by exploiting the Mask R-CNN capability of identifying an object layout by using a pixel-based segmentation, along with transfer learning used to train the proposed model. A data set of 160 elements is created containing landslide and non-landslide images. The proposed method consists of three steps: (i) augmenting training image samples to increase the volume of the training data, (ii) fine tuning with limited image samples, and (iii) performance evaluation of the algorithm in terms of precision, recall and F1 measure, on the considered landslide images, by adopting ResNet-50 and 101 as backbone models. The experimental results are quite encouraging as the proposed method achieves Precision equals to 1.00, Recall 0.93 and F1 measure 0.97, when ResNet-101 is used as backbone model, and with a low number of landslide photographs used as training samples. The proposed algorithm can be potentially useful for land use planners and policy makers of hilly areas where intermittent slope deformations necessitate landslide detection as prerequisite before planning.