OpenRaiser/Intern-Atlas huggingface dataset:
Towards Incorporating Contextual Knowledge into the Prediction of Driving Behavior
Venue: arXiv cs.CV 2020
Year: 2020

Predicting the behavior of surrounding traffic participants is crucial for advanced driver assistance systems and autonomous driving. Most researchers however do not consider contextual knowledge when predicting vehicle motion. Extending former studies, we investigate how predictions are affected by external conditions. To do so, we categorize different kinds of contextual information and provide a carefully chosen definition as well as examples for external conditions. More precisely, we investigate how a state-of-the-art approach for lateral motion prediction is influenced by one selected external condition, namely the traffic density. Our investigations demonstrate that this kind of information is highly relevant in order to improve the performance of prediction algorithms. Therefore, this study constitutes the first step towards the integration of such information into automated vehicles. Moreover, our motion prediction approach is evaluated based on the public highD data set showing a maneuver prediction performance with areas under the ROC curve above 97% and a median lateral prediction error of only 0.18m on a prediction horizon of 5s.
Flood & Loot: A Systemic Attack On The Lightning Network
Venue: arXiv cs.CR 2020
Year: 2020

The Lightning Network promises to alleviate Bitcoin's known scalability problems. The operation of such second layer approaches relies on the ability of participants to turn to the blockchain to claim funds at any time, which is assumed to happen rarely. One of the risks that was identified early on is that of a wide systemic attack on the protocol, in which an attacker triggers the closure of many Lightning channels at once. The resulting high volume of transactions in the blockchain will not allow for the proper settlement of all debts, and attackers may get away with stealing some funds. This paper explores the details of such an attack and evaluates its cost and overall impact on Bitcoin and the Lightning Network. Specifically, we show that an attacker is able to simultaneously cause victim nodes to overload the Bitcoin blockchain with requests and to steal funds that were locked in channels. We go on to examine the interaction of Lightning nodes with the fee estimation mechanism and show that the attacker can continuously lower the fee of transactions that will later be used by the victim in its attempts to recover funds - eventually reaching a state in which only low fractions of the block are available for lightning transactions. Our attack is made easier even further as the Lightning protocol allows the attacker to increase the fee offered by his own transactions. We continue to empirically show that the vast majority of nodes agree to channel opening requests from unknown sources and are therefore susceptible to this attack. We highlight differences between various implementations of the Lightning Network protocol and review the susceptibility of each one to the attack. Finally, we propose mitigation strategies to lower the systemic attack risk of the network.
"Notic My Speech" -- Blending Speech Patterns With Multimedia
Venue: arXiv cs.CL 2020
Year: 2020

Speech as a natural signal is composed of three parts - visemes (visual part of speech), phonemes (spoken part of speech), and language (the imposed structure). However, video as a medium for the delivery of speech and a multimedia construct has mostly ignored the cognitive aspects of speech delivery. For example, video applications like transcoding and compression have till now ignored the fact how speech is delivered and heard. To close the gap between speech understanding and multimedia video applications, in this paper, we show the initial experiments by modelling the perception on visual speech and showing its use case on video compression. On the other hand, in the visual speech recognition domain, existing studies have mostly modeled it as a classification problem, while ignoring the correlations between views, phonemes, visemes, and speech perception. This results in solutions which are further away from how human perception works. To bridge this gap, we propose a view-temporal attention mechanism to model both the view dependence and the visemic importance in speech recognition and understanding. We conduct experiments on three public visual speech recognition datasets. The experimental results show that our proposed method outperformed the existing work by 4.99% in terms of the viseme error rate. Moreover, we show that there is a strong correlation between our model's understanding of multi-view speech and the human perception. This characteristic benefits downstream applications such as video compression and streaming where a significant number of less important frames can be compressed or eliminated while being able to maximally preserve human speech understanding with good user experience.
Multi-Image Summarization: Textual Summary from a Set of Cohesive Images
Venue: arXiv cs.CV 2020
Year: 2020

Multi-sentence summarization is a well studied problem in NLP, while generating image descriptions for a single image is a well studied problem in Computer Vision. However, for applications such as image cluster labeling or web page summarization, summarizing a set of images is also a useful and challenging task. This paper proposes the new task of multi-image summarization, which aims to generate a concise and descriptive textual summary given a coherent set of input images. We propose a model that extends the image-captioning Transformer-based architecture for single image to multi-image. A dense average image feature aggregation network allows the model to focus on a coherent subset of attributes across the input images. We explore various input representations to the Transformer network and empirically show that aggregated image features are superior to individual image embeddings. We additionally show that the performance of the model is further improved by pretraining the model parameters on a single-image captioning task, which appears to be particularly effective in eliminating hallucinations in the output.
DeshuffleGAN: A Self-Supervised GAN to Improve Structure Learning
Venue: arXiv cs.CV 2020
Year: 2020

Generative Adversarial Networks (GANs) triggered an increased interest in problem of image generation due to their improved output image quality and versatility for expansion towards new methods. Numerous GAN-based works attempt to improve generation by architectural and loss-based extensions. We argue that one of the crucial points to improve the GAN performance in terms of realism and similarity to the original data distribution is to be able to provide the model with a capability to learn the spatial structure in data. To that end, we propose the DeshuffleGAN to enhance the learning of the discriminator and the generator, via a self-supervision approach. Specifically, we introduce a deshuffling task that solves a puzzle of randomly shuffled image tiles, which in turn helps the DeshuffleGAN learn to increase its expressive capacity for spatial structure and realistic appearance. We provide experimental evidence for the performance improvement in generated images, compared to the baseline methods, which is consistently observed over two different datasets.
Analytic Manifold Learning: Unifying and Evaluating Representations for Continuous Control
Venue: arXiv cs.LG 2020
Year: 2020

We address the problem of learning reusable state representations from streaming high-dimensional observations. This is important for areas like Reinforcement Learning (RL), which yields non-stationary data distributions during training. We make two key contributions. First, we propose an evaluation suite that measures alignment between latent and true low-dimensional states. We benchmark several widely used unsupervised learning approaches. This uncovers the strengths and limitations of existing approaches that impose additional constraints/objectives on the latent space. Our second contribution is a unifying mathematical formulation for learning latent relations. We learn analytic relations on source domains, then use these relations to help structure the latent space when learning on target domains. This formulation enables a more general, flexible and principled way of shaping the latent space. It formalizes the notion of learning independent relations, without imposing restrictive simplifying assumptions or requiring domain-specific information. We present mathematical properties, concrete algorithms for implementation and experimental validation of successful learning and transfer of latent relations.
DynE: Dynamic Ensemble Decoding for Multi-Document Summarization
Venue: arXiv cs.CL 2020
Year: 2020

Sequence-to-sequence (s2s) models are the basis for extensive work in natural language processing. However, some applications, such as multi-document summarization, multi-modal machine translation, and the automatic post-editing of machine translation, require mapping a set of multiple distinct inputs into a single output sequence. Recent work has introduced bespoke architectures for these multi-input settings, and developed models which can handle increasingly longer inputs; however, the performance of special model architectures is limited by the available in-domain training data. In this work we propose a simple decoding methodology which ensembles the output of multiple instances of the same model on different inputs. Our proposed approach allows models trained for vanilla s2s tasks to be directly used in multi-input settings. This works particularly well when each of the inputs has significant overlap with the others, as when compressing a cluster of news articles about the same event into a single coherent summary, and we obtain state-of-the-art results on several multi-document summarization datasets.
Man, machine and work in a digital twin setup: a case study
Venue: arXiv cs.CY 2020
Year: 2020

This paper explores the opportunities of using a digital twin to address the complexities of collaborative production systems through an industrial case and a demonstrator. A digital twin, as a virtual counterpart of a physical human-robot assembly system, is built as a front-runner for validation and control through design, build, and operation. The forms of digital twins along the system life cycle, the building blocks, and potential advantages are presented. Recommendations for future research and practice in the use of digital twins in the field of collaborative robots are given.
Micro-controllers: Promoting Structurally Flexible Controllers in Self-Adaptive Software Systems
Venue: arXiv cs.SE 2020
Year: 2020

To promote structurally flexible controllers in self-adaptive software systems, this paper proposes the use of micro-controllers. Instead of generic monolithic controllers, like Rainbow, we advocate the use of service-specific micro-controllers which can be based on microservices. Although traditional generic controllers can be configured parametrically according to system needs, their use and reuse are nevertheless restrictive because of the wide range of services expected from the different stages of the feedback control loop.The solution being advocated is to have structurally flexible controllers that can be composed from micro-controllers. Controlling the architectural configuration of these micro-controllers is a meta-controller that is able to configure the controller according to the services required for controlling the target system. The feasibility of the proposed approach of using micro-controllers at the level of the controller is demonstrated in the context of the PhoneAdapter case study in which micro-controllers are configured at run-time depending on changes affecting the system or its environment.
Interference and Coverage Analysis in Coexisting RF and Dense TeraHertz Wireless Networks
Venue: arXiv cs.NI 2020
Year: 2020

This paper develops a stochastic geometry framework to characterize the statistics of the downlink interference and coverage probability of a typical user in a coexisting terahertz (THz) and radio frequency (RF) network. We first characterize the exact Laplace Transform (LT) of the aggregate interference and coverage probability of a user in a THz-only network. Then, for a coexisting RF/THz network, we derive the coverage probability of a typical user considering biased received signal power association (BRSP). The framework can be customized to capture the performance of a typical user in various network configurations such as THz-only, opportunistic RF/THz, and hybrid RF/THz. In addition, asymptotic approximations are presented for scenarios where the intensity of THz BSs becomes large or molecular absorption coefficient in THz approaches to zero. Numerical results demonstrate the accuracy of the derived expressions and extract insights related to the significance of the BRSP association compared to the conventional reference signal received power (RSRP) association in the coexisting network.
Fast Graph Attention Networks Using Effective Resistance Based Graph Sparsification
Venue: arXiv cs.LG 2020
Year: 2020

The attention mechanism has demonstrated superior performance for inference over nodes in graph neural networks (GNNs), however, they result in a high computational burden during both training and inference. We propose FastGAT, a method to make attention based GNNs lightweight by using spectral sparsification to generate an optimal pruning of the input graph. This results in a per-epoch time that is almost linear in the number of graph nodes as opposed to quadratic. We theoretically prove that spectral sparsification preserves the features computed by the GAT model, thereby justifying our algorithm. We experimentally evaluate FastGAT on several large real world graph datasets for node classification tasks under both inductive and transductive settings. FastGAT can dramatically reduce (up to \textbf{10x}) the computational time and memory requirements, allowing the usage of attention based GNNs on large graphs.
Deep Autoencoding Topic Model with Scalable Hybrid Bayesian Inference
Venue: arXiv cs.LG 2020
Year: 2020

To build a flexible and interpretable model for document analysis, we develop deep autoencoding topic model (DATM) that uses a hierarchy of gamma distributions to construct its multi-stochastic-layer generative network. In order to provide scalable posterior inference for the parameters of the generative network, we develop topic-layer-adaptive stochastic gradient Riemannian MCMC that jointly learns simplex-constrained global parameters across all layers and topics, with topic and layer specific learning rates. Given a posterior sample of the global parameters, in order to efficiently infer the local latent representations of a document under DATM across all stochastic layers, we propose a Weibull upward-downward variational encoder that deterministically propagates information upward via a deep neural network, followed by a Weibull distribution based stochastic downward generative model. To jointly model documents and their associated labels, we further propose supervised DATM that enhances the discriminative power of its latent representations. The efficacy and scalability of our models are demonstrated on both unsupervised and supervised learning tasks on big corpora.
User Profiling from Reviews for Accurate Time-Based Recommendations
Venue: arXiv cs.IR 2020
Year: 2020

Recommender systems are a valuable way to engage users in a system, increase participation and show them resources they may not have found otherwise. One significant challenge is that user interests may change over time and certain items have an inherently temporal aspect. As a result, a recommender system should try and take into account the time-dependant user-item relationships. However, temporal aspects of a user profile may not always be explicitly available and so we may need to infer this information from available resources. Product reviews on sites, such as Amazon, represent a valuable data source to understand why someone bought an item and potentially who the item is for. This information can then be used to construct a dynamic user profile. In this paper, we demonstrate utilising reviews to extract temporal information to infer the \textit{age category preference} of users, and leverage this feature to generate time-dependent recommendations. Given the predictable and yet shifting nature of age and time, we show that, recommendations generated using this dynamic aspect lead to higher accuracy compared with techniques from state of art. Mining temporally related content in reviews can enable the recommender to go beyond finding similar items or users to potentially predict a future need of a user.
Hash Cracking Benchmarking of Replacement Patterns
Venue: arXiv cs.CR 2020
Year: 2020

In this paper, we explain our methodology to identify replacement patterns. The main purpose of this article is to show that with replacement methods on plain texts, it is possible to have more success rates when trying to recovering hashed passwords.
Mining Personalized Climate Preferences for Assistant Driving
Venue: arXiv cs.LG 2020
Year: 2020

Both assistant driving and self-driving have attracted a great amount of attention in the last few years. However, the majority of research efforts focus on safe driving; few research has been conducted on in-vehicle climate control, or assistant driving based on travellers' personal habits or preferences. In this paper, we propose a novel approach for climate control, driver behavior recognition and driving recommendation for better fitting drivers' preferences in their daily driving. The algorithm consists three components: (1) A in-vehicle sensing and context feature enriching compnent with a Internet of Things (IoT) platform for collecting related environment, vehicle-running, and traffic parameters that affect drivers' behaviors. (2) A non-intrusive intelligent driver behaviour and vehicle status detection component, which can automatically label vehicle's status (open windows, turn on air condition, etc.), based on results of applying further feature extraction and machine learning algorithms. (3) A personalized driver habits learning and preference recommendation component for more healthy and comfortable experiences. A prototype using a client-server architecture with an iOS app and an air-quality monitoring sensor has been developed for collecting heterogeneous data and testing our algorithms. Real-world experiments on driving data of 11,370 km (320 hours) by different drivers in multiple cities worldwide have been conducted, which demonstrate the effective and accuracy of our approach.
Minimum Width for Universal Approximation
Venue: arXiv cs.LG 2020
Year: 2020

The universal approximation property of width-bounded networks has been studied as a dual of classical universal approximation results on depth-bounded networks. However, the critical width enabling the universal approximation has not been exactly characterized in terms of the input dimension $d_x$ and the output dimension $d_y$. In this work, we provide the first definitive result in this direction for networks using the ReLU activation functions: The minimum width required for the universal approximation of the $L^p$ functions is exactly $\max\{d_x+1,d_y\}$. We also prove that the same conclusion does not hold for the uniform approximation with ReLU, but does hold with an additional threshold activation function. Our proof technique can be also used to derive a tighter upper bound on the minimum width required for the universal approximation using networks with general activation functions.
DeepCapture: Image Spam Detection Using Deep Learning and Data Augmentation
Venue: arXiv cs.LG 2020
Year: 2020

Image spam emails are often used to evade text-based spam filters that detect spam emails with their frequently used keywords. In this paper, we propose a new image spam email detection tool called DeepCapture using a convolutional neural network (CNN) model. There have been many efforts to detect image spam emails, but there is a significant performance degrade against entirely new and unseen image spam emails due to overfitting during the training phase. To address this challenging issue, we mainly focus on developing a more robust model to address the overfitting problem. Our key idea is to build a CNN-XGBoost framework consisting of eight layers only with a large number of training samples using data augmentation techniques tailored towards the image spam detection task. To show the feasibility of DeepCapture, we evaluate its performance with publicly available datasets consisting of 6,000 spam and 2,313 non-spam image samples. The experimental results show that DeepCapture is capable of achieving an F1-score of 88%, which has a 6% improvement over the best existing spam detection model CNN-SVM with an F1-score of 82%. Moreover, DeepCapture outperformed existing image spam detection solutions against new and unseen image datasets.
Calibrating Deep Neural Network Classifiers on Out-of-Distribution Datasets
Venue: arXiv cs.LG 2020
Year: 2020

To increase the trustworthiness of deep neural network (DNN) classifiers, an accurate prediction confidence that represents the true likelihood of correctness is crucial. Towards this end, many post-hoc calibration methods have been proposed to leverage a lightweight model to map the target DNN's output layer into a calibrated confidence. Nonetheless, on an out-of-distribution (OOD) dataset in practice, the target DNN can often mis-classify samples with a high confidence, creating significant challenges for the existing calibration methods to produce an accurate confidence. In this paper, we propose a new post-hoc confidence calibration method, called CCAC (Confidence Calibration with an Auxiliary Class), for DNN classifiers on OOD datasets. The key novelty of CCAC is an auxiliary class in the calibration model which separates mis-classified samples from correctly classified ones, thus effectively mitigating the target DNN's being confidently wrong. We also propose a simplified version of CCAC to reduce free parameters and facilitate transfer to a new unseen dataset. Our experiments on different DNN models, datasets and applications show that CCAC can consistently outperform the prior post-hoc calibration methods.
SPLASH: Learnable Activation Functions for Improving Accuracy and Adversarial Robustness
Venue: arXiv cs.LG 2020
Year: 2020

We introduce SPLASH units, a class of learnable activation functions shown to simultaneously improve the accuracy of deep neural networks while also improving their robustness to adversarial attacks. SPLASH units have both a simple parameterization and maintain the ability to approximate a wide range of non-linear functions. SPLASH units are: 1) continuous; 2) grounded (f(0) = 0); 3) use symmetric hinges; and 4) the locations of the hinges are derived directly from the data (i.e. no learning required). Compared to nine other learned and fixed activation functions, including ReLU and its variants, SPLASH units show superior performance across three datasets (MNIST, CIFAR-10, and CIFAR-100) and four architectures (LeNet5, All-CNN, ResNet-20, and Network-in-Network). Furthermore, we show that SPLASH units significantly increase the robustness of deep neural networks to adversarial attacks. Our experiments on both black-box and open-box adversarial attacks show that commonly-used architectures, namely LeNet5, All-CNN, ResNet-20, and Network-in-Network, can be up to 31% more robust to adversarial attacks by simply using SPLASH units instead of ReLUs.