OpenRaiser/Intern-Atlas huggingface dataset:
KeepAugment: A Simple Information-Preserving Data Augmentation Approach
Venue: arXiv cs.CV 2020
Year: 2020

Data augmentation (DA) is an essential technique for training state-of-the-art deep learning systems. In this paper, we empirically show data augmentation might introduce noisy augmented examples and consequently hurt the performance on unaugmented data during inference. To alleviate this issue, we propose a simple yet highly effective approach, dubbed \emph{KeepAugment}, to increase augmented images fidelity. The idea is first to use the saliency map to detect important regions on the original images and then preserve these informative regions during augmentation. This information-preserving strategy allows us to generate more faithful training examples. Empirically, we demonstrate our method significantly improves on a number of prior art data augmentation schemes, e.g. AutoAugment, Cutout, random erasing, achieving promising results on image classification, semi-supervised image classification, multi-view multi-camera tracking and object detection.
The Interpretable Dictionary in Sparse Coding
Venue: arXiv cs.LG 2020
Year: 2020

Artificial neural networks (ANNs), specifically deep learning networks, have often been labeled as black boxes due to the fact that the internal representation of the data is not easily interpretable. In our work, we illustrate that an ANN, trained using sparse coding under specific sparsity constraints, yields a more interpretable model than the standard deep learning model. The dictionary learned by sparse coding can be more easily understood and the activations of these elements creates a selective feature output. We compare and contrast our sparse coding model with an equivalent feed forward convolutional autoencoder trained on the same data. Our results show both qualitative and quantitative benefits in the interpretation of the learned sparse coding dictionary as well as the internal activation representations.
Acoustic span embeddings for multilingual query-by-example search
Venue: arXiv cs.CL 2020
Year: 2020

Query-by-example (QbE) speech search is the task of matching spoken queries to utterances within a search collection. In low- or zero-resource settings, QbE search is often addressed with approaches based on dynamic time warping (DTW). Recent work has found that methods based on acoustic word embeddings (AWEs) can improve both performance and search speed. However, prior work on AWE-based QbE has primarily focused on English data and with single-word queries. In this work, we generalize AWE training to spans of words, producing acoustic span embeddings (ASE), and explore the application of ASE to QbE with arbitrary-length queries in multiple unseen languages. We consider the commonly used setting where we have access to labeled data in other languages (in our case, several low-resource languages) distinct from the unseen test languages. We evaluate our approach on the QUESST 2015 QbE tasks, finding that multilingual ASE-based search is much faster than DTW-based search and outperforms the best previously published results on this task.
Delayed Feedback Modeling for the Entire Space Conversion Rate Prediction
Venue: arXiv cs.LG 2020
Year: 2020

Estimating post-click conversion rate (CVR) accurately is crucial in E-commerce. However, CVR prediction usually suffers from three major challenges in practice: i) data sparsity: compared with impressions, conversion samples are often extremely scarce; ii) sample selection bias: conventional CVR models are trained with clicked impressions while making inference on the entire space of all impressions; iii) delayed feedback: many conversions can only be observed after a relatively long and random delay since clicks happened, resulting in many false negative labels during training. Previous studies mainly focus on one or two issues while ignoring the others. In this paper, we propose a novel neural network framework ESDF to tackle the above three challenges simultaneously. Unlike existing methods, ESDF models the CVR prediction from a perspective of entire space, and combines the advantage of user sequential behavior pattern and the time delay factor. Specifically, ESDF utilizes sequential behavior of user actions on the entire space with all impressions to alleviate the sample selection bias problem. By sharing the embedding parameters between CTR and CVR networks, data sparsity problem is greatly relieved. Different from conventional delayed feedback methods, ESDF does not make any special assumption about the delay distribution. We discretize the delay time by day slot and model the probability based on survival analysis with deep neural network, which is more practical and suitable for industrial situations. Extensive experiments are conducted to evaluate the effectiveness of our method. To the best of our knowledge, ESDF is the first attempt to unitedly solve the above three challenges in CVR prediction area.
Benchmarking Inference Performance of Deep Learning Models on Analog Devices
Venue: arXiv cs.LG 2020
Year: 2020

Analog hardware implemented deep learning models are promising for computation and energy constrained systems such as edge computing devices. However, the analog nature of the device and the associated many noise sources will cause changes to the value of the weights in the trained deep learning models deployed on such devices. In this study, systematic evaluation of the inference performance of trained popular deep learning models for image classification deployed on analog devices has been carried out, where additive white Gaussian noise has been added to the weights of the trained models during inference. It is observed that deeper models and models with more redundancy in design such as VGG are more robust to the noise in general. However, the performance is also affected by the design philosophy of the model, the detailed structure of the model, the exact machine learning task, as well as the datasets.
AutoWeka4MCPS-AVATAR: Accelerating Automated Machine Learning Pipeline Composition and Optimisation
Venue: arXiv cs.LG 2020
Year: 2020

Automated machine learning pipeline (ML) composition and optimisation aim at automating the process of finding the most promising ML pipelines within allocated resources (i.e., time, CPU and memory). Existing methods, such as Bayesian-based and genetic-based optimisation, which are implemented in Auto-Weka, Auto-sklearn and TPOT, evaluate pipelines by executing them. Therefore, the pipeline composition and optimisation of these methods frequently require a tremendous amount of time that prevents them from exploring complex pipelines to find better predictive models. To further explore this research challenge, we have conducted experiments showing that many of the generated pipelines are invalid in the first place, and attempting to execute them is a waste of time and resources. To address this issue, we propose a novel method to evaluate the validity of ML pipelines, without their execution, using a surrogate model (AVATAR). The AVATAR generates a knowledge base by automatically learning the capabilities and effects of ML algorithms on datasets' characteristics. This knowledge base is used for a simplified mapping from an original ML pipeline to a surrogate model which is a Petri net based pipeline. Instead of executing the original ML pipeline to evaluate its validity, the AVATAR evaluates its surrogate model constructed by capabilities and effects of the ML pipeline components and input/output simplified mappings. Evaluating this surrogate model is less resource-intensive than the execution of the original pipeline. As a result, the AVATAR enables the pipeline composition and optimisation methods to evaluate more pipelines by quickly rejecting invalid pipelines. We integrate the AVATAR into the sequential model-based algorithm configuration (SMAC). Our experiments show that when SMAC employs AVATAR, it finds better solutions than on its own.
Counterfactual Fairness with Disentangled Causal Effect Variational Autoencoder
Venue: arXiv cs.LG 2020
Year: 2020

The problem of fair classification can be mollified if we develop a method to remove the embedded sensitive information from the classification features. This line of separating the sensitive information is developed through the causal inference, and the causal inference enables the counterfactual generations to contrast the what-if case of the opposite sensitive attribute. Along with this separation with the causality, a frequent assumption in the deep latent causal model defines a single latent variable to absorb the entire exogenous uncertainty of the causal graph. However, we claim that such structure cannot distinguish the 1) information caused by the intervention (i.e., sensitive variable) and 2) information correlated with the intervention from the data. Therefore, this paper proposes Disentangled Causal Effect Variational Autoencoder (DCEVAE) to resolve this limitation by disentangling the exogenous uncertainty into two latent variables: either 1) independent to interventions or 2) correlated to interventions without causality. Particularly, our disentangling approach preserves the latent variable correlated to interventions in generating counterfactual examples. We show that our method estimates the total effect and the counterfactual effect without a complete causal graph. By adding a fairness regularization, DCEVAE generates a counterfactual fair dataset while losing less original information. Also, DCEVAE generates natural counterfactual images by only flipping sensitive information. Additionally, we theoretically show the differences in the covariance structures of DCEVAE and prior works from the perspective of the latent disentanglement.
Rotational Error Metrics for Quadrotor Control
Venue: arXiv cs.RO 2020
Year: 2020

We analyze and experimentally compare various rotational error metrics for use in quadrotor controllers. Traditional quadrotor attitude controllers have used Euler angles or the full rotation to compute an attitude error and scale that to compute a control response. Recently, several works have shown that prioritizing quadrotor tilt, or thrust vector error, in the attitude controller leads to improved position control, especially in situations with large yaw error. We provide a catalog of proposed rotational metrics, place them into the same framework, and show that we can independently reason about and design the magnitude of the response and the direction of the response. Existing approaches mainly fall into two categories: (1) metrics that induce a response in the shortest direction to correct the full rotation error and (2) metrics that combine a response in the shortest direction to correct tilt error with the shortest direction to correct yaw error. We show experimental results to highlight the salient differences between the rotational error metrics. See https://alspitz.github.io/roterrormetrics.html for an interactive simulation visualizing the experiments performed.
Acceleration of Cooperative Least Mean Square via Chebyshev Periodical Successive Over-Relaxation
Venue: arXiv cs.IT 2020
Year: 2020

A distributed algorithm for least mean square (LMS) can be used in distributed signal estimation and in distributed training for multivariate regression models. The convergence speed of an algorithm is a critical factor because a faster algorithm requires less communications overhead and it results in a narrower network bandwidth. The goal of this paper is to present that use of Chebyshev periodical successive over-relaxation (PSOR) can accelerate distributed LMS algorithms in a naturally manner. The basic idea of Chbyshev PSOR is to introduce index-dependent PSOR factors that control the spectral radius of a matrix governing the convergence behavior of the modified fixed-point iteration. Accelerations of convergence speed are empirically confirmed in a wide range of networks, such as known small graphs (e.g., Karate graph), and random graphs, such as Erdos-Renyi (ER) random graphs and Barabasi-Albert random graphs.
Path Planning with Automatic Seam Extraction over Point Cloud Models for Robotic Arc Welding
Venue: arXiv cs.RO 2020
Year: 2020

This paper presents a point cloud based robotic system for arc welding. Using hand gesture controls, the system scans partial point cloud views of workpiece and reconstructs them into a complete 3D model by a linear iterative closest point algorithm. Then, a bilateral filter is extended to denoise the workpiece model and preserve important geometrical information. To extract the welding seam from the model, a novel intensity-based algorithm is proposed that detects edge points and generates a smooth 6-DOF welding path. The methods are tested on multiple workpieces with different joint types and poses. Experimental results prove the robustness and efficiency of this robotic system on automatic path planning for welding applications.
Towards Imperceptible Universal Attacks on Texture Recognition
Venue: arXiv cs.CV 2020
Year: 2020

Although deep neural networks (DNNs) have been shown to be susceptible to image-agnostic adversarial attacks on natural image classification problems, the effects of such attacks on DNN-based texture recognition have yet to be explored. As part of our work, we find that limiting the perturbation's $l_p$ norm in the spatial domain may not be a suitable way to restrict the perceptibility of universal adversarial perturbations for texture images. Based on the fact that human perception is affected by local visual frequency characteristics, we propose a frequency-tuned universal attack method to compute universal perturbations in the frequency domain. Our experiments indicate that our proposed method can produce less perceptible perturbations yet with a similar or higher white-box fooling rates on various DNN texture classifiers and texture datasets as compared to existing universal attack techniques. We also demonstrate that our approach can improve the attack robustness against defended models as well as the cross-dataset transferability for texture recognition problems.
LiDAR-based Panoptic Segmentation via Dynamic Shifting Network
Venue: arXiv cs.CV 2020
Year: 2020

With the rapid advances of autonomous driving, it becomes critical to equip its sensing system with more holistic 3D perception. However, existing works focus on parsing either the objects (e.g. cars and pedestrians) or scenes (e.g. trees and buildings) from the LiDAR sensor. In this work, we address the task of LiDAR-based panoptic segmentation, which aims to parse both objects and scenes in a unified manner. As one of the first endeavors towards this new challenging task, we propose the Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm. In particular, DS-Net has three appealing properties: 1) strong backbone design. DS-Net adopts the cylinder convolution that is specifically designed for LiDAR point clouds. The extracted features are shared by the semantic branch and the instance branch which operates in a bottom-up clustering style. 2) Dynamic Shifting for complex point distributions. We observe that commonly-used clustering algorithms like BFS or DBSCAN are incapable of handling complex autonomous driving scenes with non-uniform point cloud distributions and varying instance sizes. Thus, we present an efficient learnable clustering module, dynamic shifting, which adapts kernel functions on-the-fly for different instances. 3) Consensus-driven Fusion. Finally, consensus-driven fusion is used to deal with the disagreement between semantic and instance predictions. To comprehensively evaluate the performance of LiDAR-based panoptic segmentation, we construct and curate benchmarks from two large-scale autonomous driving LiDAR datasets, SemanticKITTI and nuScenes. Extensive experiments demonstrate that our proposed DS-Net achieves superior accuracies over current state-of-the-art methods. Notably, we achieve 1st place on the public leaderboard of SemanticKITTI, outperforming 2nd place by 2.6% in terms of the PQ metric.
Low-Power IoT Communication Security: On the Performance of DTLS and TLS 1.3
Venue: arXiv cs.CR 2020
Year: 2020

Similarly to elsewhere on the Internet, practical security in the Internet of Things (IoT) is achieved by combining an array of mechanisms, at work at all layers of the protocol stack, in system software, and in hardware. Standard protocols such as Datagram Transport Layer Security (DTLS 1.2) and Transport Layer Security (TLS 1.2) are often recommended to secure communications to/from IoT devices. Recently, the TLS 1.3 standard was released and DTLS 1.3 is in the final stages of standardization. In this paper, we give an overview of version 1.3 of these protocols, and we provide the first experimental comparative performance analysis of different implementations and various configurations of these protocols, on real IoT devices based on low-power microcontrollers. We show how different implementations lead to different compromises. We measure and compare bytes-over-the-air, memory footprint, and energy consumption. We show that, when DTLS/TLS 1.3 requires more resources than DTLS/TLS 1.2, this additional overhead is quite reasonable. We also observe that, in some configurations, DTLS/TLS 1.3 actually decreases overhead and resource consumption. All in all, our study indicates that there is still room to optimize the existing implementations of these protocols.
Non-Invertible-Element Constacyclic Codes over Finite PIRs
Venue: arXiv cs.IT 2020
Year: 2020

In this paper we introduce the notion of $λ$-constacyclic codes over finite rings $R$ for arbitary element $λ$ of $R$. We study the non-invertible-element constacyclic codes (NIE-constacyclic codes) over finite principal ideal rings (PIRs). We determine the algebraic structures of all NIE-constacyclic codes over finite chain rings, give the unique form of the sets of the defining polynomials and obtain their minimum Hamming distances. A general form of the duals of NIE-constacyclic codes over finite chain rings is also provided. In particular, we give a necessary and sufficient condition for the dual of an NIE-constacyclic code to be an NIE-constacyclic code. Using the Chinese Remainder Theorem, we study the NIE-constacyclic codes over finite PIRs. Furthermore, we construct some optimal NIE-constacyclic codes over finite PIRs in the sense that they achieve the maximum possible minimum Hamming distances for some given lengths and cardinalities.
Fuzzy Stochastic Timed Petri Nets for Causal properties representation
Venue: arXiv cs.AI 2020
Year: 2020

Imagery is frequently used to model, represent and communicate knowledge. In particular, graphs are one of the most powerful tools, being able to represent relations between objects. Causal relations are frequently represented by directed graphs, with nodes denoting causes and links denoting causal influence. A causal graph is a skeletal picture, showing causal associations and impact between entities. Common methods used for graphically representing causal scenarios are neurons, truth tables, causal Bayesian networks, cognitive maps and Petri Nets. Causality is often defined in terms of precedence (the cause precedes the effect), concurrency (often, an effect is provoked simultaneously by two or more causes), circularity (a cause provokes the effect and the effect reinforces the cause) and imprecision (the presence of the cause favors the effect, but not necessarily causes it). We will show that, even though the traditional graphical models are able to represent separately some of the properties aforementioned, they fail trying to illustrate indistinctly all of them. To approach that gap, we will introduce Fuzzy Stochastic Timed Petri Nets as a graphical tool able to represent time, co-occurrence, looping and imprecision in causal flow.
Computational efficient deep neural network with difference attention maps for facial action unit detection
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we propose a computational efficient end-to-end training deep neural network (CEDNN) model and spatial attention maps based on difference images. Firstly, the difference image is generated by image processing. Then five binary images of difference images are obtained using different thresholds, which are used as spatial attention maps. We use group convolution to reduce model complexity. Skip connection and $\text{1}\times \text{1}$ convolution are used to ensure good performance even if the network model is not deep. As an input, spatial attention map can be selectively fed into the input of each block. The feature maps tend to focus on the parts that are related to the target task better. In addition, we only need to adjust the parameters of classifier to train different numbers of AU. It can be easily extended to varying datasets without increasing too much computation. A large number of experimental results show that the proposed CEDNN is obviously better than the traditional deep learning method on DISFA+ and CK+ datasets. After adding spatial attention maps, the result is better than the most advanced AU detection method. At the same time, the scale of the network is small, the running speed is fast, and the requirement for experimental equipment is low.
RanStop: A Hardware-assisted Runtime Crypto-Ransomware Detection Technique
Venue: arXiv cs.CR 2020
Year: 2020

Among many prevailing malware, crypto-ransomware poses a significant threat as it financially extorts affected users by creating denial of access via unauthorized encryption of their documents as well as holding their documents hostage and financially extorting them. This results in millions of dollars of annual losses worldwide. Multiple variants of ransomware are growing in number with capabilities of evasion from many anti-viruses and software-only malware detection schemes that rely on static execution signatures. In this paper, we propose a hardware-assisted scheme, called RanStop, for early detection of crypto-ransomware infection in commodity processors. RanStop leverages the information of hardware performance counters embedded in the performance monitoring unit in modern processors to observe micro-architectural event sets and detects known and unknown crypto-ransomware variants. In this paper, we train a recurrent neural network-based machine learning architecture using long short-term memory (LSTM) model for analyzing micro-architectural events in the hardware domain when executing multiple variants of ransomware as well as benign programs. We create timeseries to develop intrinsic statistical features using the information of related HPCs and improve the detection accuracy of RanStop and reduce noise by via LSTM and global average pooling. As an early detection scheme, RanStop can accurately and quickly identify ransomware within 2ms from the start of the program execution by analyzing HPC information collected for 20 timestamps each 100us apart. This detection time is too early for a ransomware to make any significant damage, if none. Moreover, validation against benign programs with behavioral (sub-routine-centric) similarity with that of a crypto-ransomware shows that RanStop can detect ransomware with an average of 97% accuracy for fifty random trials.
Model Elicitation through Direct Questioning
Venue: arXiv cs.AI 2020
Year: 2020

The future will be replete with scenarios where humans are robots will be working together in complex environments. Teammates interact, and the robot's interaction has to be about getting useful information about the human's (teammate's) model. There are many challenges before a robot can interact, such as incorporating the structural differences in the human's model, ensuring simpler responses, etc. In this paper, we investigate how a robot can interact to localize the human model from a set of models. We show how to generate questions to refine the robot's understanding of the teammate's model. We evaluate the method in various planning domains. The evaluation shows that these questions can be generated offline, and can help refine the model through simple answers.