OpenRaiser/Intern-Atlas huggingface dataset:
Assessing the Adversarial Robustness of Monte Carlo and Distillation Methods for Deep Bayesian Neural Network Classification
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we consider the problem of assessing the adversarial robustness of deep neural network models under both Markov chain Monte Carlo (MCMC) and Bayesian Dark Knowledge (BDK) inference approximations. We characterize the robustness of each method to two types of adversarial attacks: the fast gradient sign method (FGSM) and projected gradient descent (PGD). We show that full MCMC-based inference has excellent robustness, significantly outperforming standard point estimation-based learning. On the other hand, BDK provides marginal improvements. As an additional contribution, we present a storage-efficient approach to computing adversarial examples for large Monte Carlo ensembles using both the FGSM and PGD attacks.
Stable Sparse Subspace Embedding for Dimensionality Reduction
Venue: arXiv cs.LG 2020
Year: 2020

Sparse random projection (RP) is a popular tool for dimensionality reduction that shows promising performance with low computational complexity. However, in the existing sparse RP matrices, the positions of non-zero entries are usually randomly selected. Although they adopt uniform sampling with replacement, due to large sampling variance, the number of non-zeros is uneven among rows of the projection matrix which is generated in one trial, and more data information may be lost after dimension reduction. To break this bottleneck, based on random sampling without replacement in statistics, this paper builds a stable sparse subspace embedded matrix (S-SSE), in which non-zeros are uniformly distributed. It is proved that the S-SSE is stabler than the existing matrix, and it can maintain Euclidean distance between points well after dimension reduction. Our empirical studies corroborate our theoretical findings and demonstrate that our approach can indeed achieve satisfactory performance.
Representation of Reinforcement Learning Policies in Reproducing Kernel Hilbert Spaces
Venue: arXiv cs.LG 2020
Year: 2020

We propose a general framework for policy representation for reinforcement learning tasks. This framework involves finding a low-dimensional embedding of the policy on a reproducing kernel Hilbert space (RKHS). The usage of RKHS based methods allows us to derive strong theoretical guarantees on the expected return of the reconstructed policy. Such guarantees are typically lacking in black-box models, but are very desirable in tasks requiring stability. We conduct several experiments on classic RL domains. The results confirm that the policies can be robustly embedded in a low-dimensional space while the embedded policy incurs almost no decrease in return.
A Translational Three-Degrees-of-Freedom Parallel Mechanism With Partial Motion Decoupling and Analytic Direct Kinematics
Venue: arXiv cs.RO 2020
Year: 2020

According to the topological design theory and method of parallel mechanism (PM) based on position and orientation characteristic (POC) equations, this paper studied a 3-DOF translational PM that has three advantages, i.e., (i) it consists of three fixed actuated prismatic joints, (ii) the PM has analytic solutions to the direct and inverse kinematic problems, and (iii) the PM is of partial motion decoupling property. Firstly, the main topological characteristics, such as the POC, degree of freedom and coupling degree were calculated for kinematic modeling. Thanks to these properties, the direct and inverse kinematic problems can be readily solved. Further, the conditions of the singular configurations of the PM were analyzed which corresponds to its partial motion decoupling property.
Mario Level Generation From Mechanics Using Scene Stitching
Venue: arXiv cs.AI 2020
Year: 2020

This paper presents a level generation method for Super Mario by stitching together pre-generated "scenes" that contain specific mechanics, using mechanic-sequences from agent playthroughs as input specifications. Given a sequence of mechanics, our system uses an FI-2Pop algorithm and a corpus of scenes to perform automated level authoring. The system outputs levels that have a similar mechanical sequence to the target mechanic sequence but with a different playthrough experience. We compare our system to a greedy method that selects scenes that maximize the target mechanics. Our system is able to maximize the number of matched mechanics while reducing emergent mechanics using the stitching process compared to the greedy approach.
SLACC: Simion-based Language Agnostic Code Clones
Venue: arXiv cs.SE 2020
Year: 2020

Successful cross-language clone detection could enable researchers and developers to create robust language migration tools, facilitate learning additional programming languages once one is mastered, and promote reuse of code snippets over a broader codebase. However, identifying cross-language clones presents special challenges to the clone detection problem. A lack of common underlying representation between arbitrary languages means detecting clones requires one of the following solutions: 1) a static analysis framework replicated across each targeted language with annotations matching language features across all languages, or 2) a dynamic analysis framework that detects clones based on runtime behavior. In this work, we demonstrate the feasibility of the latter solution, a dynamic analysis approach called SLACC for cross-language clone detection. Like prior clone detection techniques, we use input/output behavior to match clones, though we overcome limitations of prior work by amplifying the number of inputs and covering more data types; and as a result, achieve better clusters than prior attempts. Since clusters are generated based on input/output behavior, SLACC supports cross-language clone detection. As an added challenge, we target a static typed language, Java, and a dynamic typed language, Python. Compared to HitoshiIO, a recent clone detection tool for Java, SLACC retrieves 6 times as many clusters and has higher precision (86.7% vs. 30.7%). This is the first work to perform clone detection for dynamic typed languages (precision = 87.3%) and the first to perform clone detection across languages that lack a common underlying representation (precision = 94.1%). It provides a first step towards the larger goal of scalable language migration tools.
Massive Unsourced Random Access Based on Uncoupled Compressive Sensing: Another Blessing of Massive MIMO
Venue: arXiv cs.IT 2020
Year: 2020

We put forward a new algorithmic solution to the massive unsourced random access (URA) problem, by leveraging the rich spatial dimensionality offered by large-scale antenna arrays. This paper makes an observation that spatial signature is key to URA in massive connectivity setups. The proposed scheme relies on a slotted transmission framework but eliminates the need for concatenated coding that was introduced in the context of the coupled compressive sensing (CCS) paradigm. Indeed, all existing works on CCS-based URA rely on an inner/outer tree-based encoder/decoder to stitch the slot-wise recovered sequences. This paper takes a different path by harnessing the nature-provided correlations between the slotwise reconstructed channels of each user in order to put together its decoded sequences. The required slot-wise channel estimates and decoded sequences are first obtained through the hybrid generalized approximate message passing (HyGAMP) algorithm which systematically accommodates the multiantenna-induced group sparsity. Then, a channel correlation-aware clustering framework based on the expectation-maximization (EM) concept is used together with the Hungarian algorithm to find the slotwise optimal assignment matrices by enforcing two clustering constraints that are very specific to the problem at hand. Stitching is then accomplished by associating the decoded sequences to their respective users according to the ensuing assignment matrices. Exhaustive computer simulations reveal that the proposed scheme can bring performance improvements, at high spectral efficiencies, as compared to a state-of-the-art technique that investigates the use of large-scale antenna arrays in the context of massive URA.
Lessons Learned Developing and Extending a Visual Analytics Solution for Investigative Analysis of Scamming Activities
Venue: arXiv cs.HC 2020
Year: 2020

Cybersecurity analysts work on large communication data sets to perform investigative analysis by painstakingly going over thousands of email conversations to find potential scamming activities and the network of cyber scammers. Traditionally,experts used email clients, database systems and text editors to perform this investigation. With the advent of technology,elaborate tools that summarize data more efficiently by using cutting edge data visualization techniques have come out. Beagle[1] is one such tool which visualizes the large communication data using different panels such that the inspector has better chances of finding the scam network. This paper is a report on our work to implement and improve the work done by Jay Koven et al. [1]. We have proposed and demonstrated via implementation, a few more visualizations that we feel would help in grouping and analyzing the e-mail data more efficiently. Lastly, we have also presented a case study that shows the potential use of our tool in a real-world scenario.
Joint Satellite Gateway Placement and Routing for Integrated Satellite-Terrestrial Networks
Venue: arXiv cs.NI 2020
Year: 2020

With the increasing attention to the integrated satellite-terrestrial networks (ISTNs), the satellite gateway placement problem becomes of paramount importance. The resulting network performance may vary depending on the different design strategies. In this paper, a joint satellite gateway placement and routing strategy for the terrestrial network is proposed to minimize the overall cost of gateway deployment and traffic routing, while adhering to the average delay requirement for traffic demands. Although traffic routing and gateway placement can be solved independently, the dependence between the routing decisions for different demands makes it more realistic to solve an aggregated model instead. We develop a mixed-integer linear program (MILP) formulation for the problem. We relax the integrality constraints to achieve a linear program (LP) which reduces time-complexity at the expense of a sub-optimal solution. We further propose a variant of the proposed model to balance the load between the selected gateways.
Analysis of Random Perturbations for Robust Convolutional Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Recent work has extensively shown that randomized perturbations of neural networks can improve robustness to adversarial attacks. The literature is, however, lacking a detailed compare-and-contrast of the latest proposals to understand what classes of perturbations work, when they work, and why they work. We contribute a detailed evaluation that elucidates these questions and benchmarks perturbation based defenses consistently. In particular, we show five main results: (1) all input perturbation defenses, whether random or deterministic, are equivalent in their efficacy, (2) attacks transfer between perturbation defenses so the attackers need not know the specific type of defense -- only that it involves perturbations, (3) a tuned sequence of noise layers across a network provides the best empirical robustness, (4) perturbation based defenses offer almost no robustness to adaptive attacks unless these perturbations are observed during training, and (5) adversarial examples in a close neighborhood of original inputs show an elevated sensitivity to perturbations in first and second-order analyses.
LAVA NAT: A Non-Autoregressive Translation Model with Look-Around Decoding and Vocabulary Attention
Venue: arXiv cs.CL 2020
Year: 2020

Non-autoregressive translation (NAT) models generate multiple tokens in one forward pass and is highly efficient at inference stage compared with autoregressive translation (AT) methods. However, NAT models often suffer from the multimodality problem, i.e., generating duplicated tokens or missing tokens. In this paper, we propose two novel methods to address this issue, the Look-Around (LA) strategy and the Vocabulary Attention (VA) mechanism. The Look-Around strategy predicts the neighbor tokens in order to predict the current token, and the Vocabulary Attention models long-term token dependencies inside the decoder by attending the whole vocabulary for each position to acquire knowledge of which token is about to generate. %We also propose a dynamic bidirectional decoding approach to accelerate the inference process of the LAVA model while preserving the high-quality of the generated output. Our proposed model uses significantly less time during inference compared with autoregressive models and most other NAT models. Our experiments on four benchmarks (WMT14 En$\rightarrow$De, WMT14 De$\rightarrow$En, WMT16 Ro$\rightarrow$En and IWSLT14 De$\rightarrow$En) show that the proposed model achieves competitive performance compared with the state-of-the-art non-autoregressive and autoregressive models while significantly reducing the time cost in inference phase.
Physical-Layer Security in the Finite Blocklength Regime over Fading Channels
Venue: arXiv cs.IT 2020
Year: 2020

This paper studies physical-layer secure transmissions from a transmitter to a legitimate receiver against an eavesdropper over slow fading channels, taking into account the impact of finite blocklength secrecy coding. A comprehensive analysis and optimization framework is established to investigate secrecy throughput for both single- and multi-antenna transmitter scenarios. Both adaptive and non-adaptive design schemes are devised, in which the secrecy throughput is maximized by exploiting the instantaneous and statistical channel state information of the legitimate receiver, respectively. Specifically, optimal transmission policy, blocklength, and code rates are jointly designed to maximize the secrecy throughput. Additionally, null-space artificial noise is employed to improve the secrecy throughput for the multi-antenna setup with the optimal power allocation derived.Various important insights are developed. In particular, 1) increasing blocklength benefits both reliability and secrecy under the proposed transmission policy; 2) secrecy throughput monotonically increases with blocklength; 3) secrecy throughput initially increases but then decreases as secrecy rate increases, and the optimal secrecy rate maximizing the secrecy throughput should be carefully chosen in order to strike a good balance between rate and decoding correctness. Numerical results are eventually presented to verify theoretical findings.
Predict your Click-out: Modeling User-Item Interactions and Session Actions in an Ensemble Learning Fashion
Venue: arXiv cs.IR 2020
Year: 2020

This paper describes the solution of the POLINKS team to the RecSys Challenge 2019 that focuses on the task of predicting the last click-out in a session-based interaction. We propose an ensemble approach comprising a matrix factorization for modeling the interaction user-item, and a session-aware learning model implemented with a recurrent neural network. This method appears to be effective in predicting the last click-out scoring a 0.60277 of Mean Reciprocal Rank on the local test set.
Multi-Modality Cascaded Fusion Technology for Autonomous Driving
Venue: arXiv cs.CV 2020
Year: 2020

Multi-modality fusion is the guarantee of the stability of autonomous driving systems. In this paper, we propose a general multi-modality cascaded fusion framework, exploiting the advantages of decision-level and feature-level fusion, utilizing target position, size, velocity, appearance and confidence to achieve accurate fusion results. In the fusion process, dynamic coordinate alignment(DCA) is conducted to reduce the error between sensors from different modalities. In addition, the calculation of affinity matrix is the core module of sensor fusion, we propose an affinity loss that improves the performance of deep affinity network(DAN). Last, the proposed step-by-step cascaded fusion framework is more interpretable and flexible compared to the end-toend fusion methods. Extensive experiments on Nuscenes [2] dataset show that our approach achieves the state-of-theart performance.dataset show that our approach achieves the state-of-the-art performance.
Surrogate Assisted Evolutionary Algorithm for Medium Scale Expensive Multi-Objective Optimisation Problems
Venue: arXiv cs.NE 2020
Year: 2020

Building a surrogate model of an objective function has shown to be effective to assist evolutionary algorithms (EAs) to solve real-world complex optimisation problems which involve either computationally expensive numerical simulations or costly physical experiments. However, their effectiveness mostly focuses on small-scale problems with less than 10 decision variables. The scalability of surrogate assisted EAs (SAEAs) have not been well studied yet. In this paper, we propose a Gaussian process surrogate model assisted EA for medium-scale expensive multi-objective optimisation problems with up to 50 decision variables. There are three distinctive features of our proposed SAEA. First, instead of using all decision variables in surrogate model building, we only use those correlated ones to build the surrogate model for each objective function. Second, rather than directly optimising the surrogate objective functions, the original multi-objective optimisation problem is transformed to a new one based on the surrogate models. Last but not the least, a subset selection method is developed to choose a couple of promising candidate solutions for actual objective function evaluations thus to update the training dataset. The effectiveness of our proposed algorithm is validated on benchmark problems with 10, 20, 50 variables, comparing with three state-of-the-art SAEAs.
Towards Reading Beyond Faces for Sparsity-Aware 4D Affect Recognition
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we present a sparsity-aware deep network for automatic 4D facial expression recognition (FER). Given 4D data, we first propose a novel augmentation method to combat the data limitation problem for deep learning. This is achieved by projecting the input data into RGB and depth map images and then iteratively performing randomized channel concatenation. Encoded in the given 3D landmarks, we also introduce an effective way to capture the facial muscle movements from three orthogonal plans (TOP), the TOP-landmarks over multi-views. Importantly, we then present a sparsity-aware deep network to compute the sparse representations of convolutional features over multi-views. This is not only effective for a higher recognition accuracy but is also computationally convenient. For training, the TOP-landmarks and sparse representations are used to train a long short-term memory (LSTM) network. The refined predictions are achieved when the learned features collaborate over multi-views. Extensive experimental results achieved on the BU-4DFE dataset show the significance of our method over the state-of-the-art methods by reaching a promising accuracy of 99.69% for 4D FER.
Polynomial time algorithms for inclusion and equivalence of deterministic omega acceptors
Venue: arXiv cs.FL 2020
Year: 2020

The class of omega languages recognized by deterministic parity acceptors (DPAs) or deterministic Muller acceptors (DMAs) is exactly the regular omega languages. The inclusion problem is the following: given two acceptors A1 and A2, determine whether the language recognized by A1 is a subset of the language recognized by A2, and if not, return an ultimately periodic omega word accepted by A1 but not A2. We describe polynomial time algorithms to solve this problem for two DPAs and for two DMAs. Corollaries include polynomial time algorithms to solve the equivalence problem for DPAs and DMAs, and also the inclusion and equivalence problems for deterministic Buechi and coBuechi acceptors.
SUOD: Toward Scalable Unsupervised Outlier Detection
Venue: arXiv cs.LG 2020
Year: 2020

Outlier detection is a key field of machine learning for identifying abnormal data objects. Due to the high expense of acquiring ground truth, unsupervised models are often chosen in practice. To compensate for the unstable nature of unsupervised algorithms, practitioners from high-stakes fields like finance, health, and security, prefer to build a large number of models for further combination and analysis. However, this poses scalability challenges in high-dimensional large datasets. In this study, we propose a three-module acceleration framework called SUOD to expedite the training and prediction with a large number of unsupervised detection models. SUOD's Random Projection module can generate lower subspaces for high-dimensional datasets while reserving their distance relationship. Balanced Parallel Scheduling module can forecast the training and prediction cost of models with high confidence---so the task scheduler could assign nearly equal amount of taskload among workers for efficient parallelization. SUOD also comes with a Pseudo-supervised Approximation module, which can approximate fitted unsupervised models by lower time complexity supervised regressors for fast prediction on unseen data. It may be considered as an unsupervised model knowledge distillation process. Notably, all three modules are independent with great flexibility to "mix and match"; a combination of modules can be chosen based on use cases. Extensive experiments on more than 30 benchmark datasets have shown the efficacy of SUOD, and a comprehensive future development plan is also presented.
Intrinsic Dimension Estimation via Nearest Constrained Subspace Classifier
Venue: arXiv cs.CV 2020
Year: 2020

We consider the problems of classification and intrinsic dimension estimation on image data. A new subspace based classifier is proposed for supervised classification or intrinsic dimension estimation. The distribution of the data in each class is modeled by a union of of a finite number ofaffine subspaces of the feature space. The affine subspaces have a common dimension, which is assumed to be much less than the dimension of the feature space. The subspaces are found using regression based on the L0-norm. The proposed method is a generalisation of classical NN (Nearest Neighbor), NFL (Nearest Feature Line) classifiers and has a close relationship to NS (Nearest Subspace) classifier. The proposed classifier with an accurately estimated dimension parameter generally outperforms its competitors in terms of classification accuracy. We also propose a fast version of the classifier using a neighborhood representation to reduce its computational complexity. Experiments on publicly available datasets corroborate these claims.
Supervised Quantile Normalization for Low-rank Matrix Approximation
Venue: arXiv cs.LG 2020
Year: 2020

Low rank matrix factorization is a fundamental building block in machine learning, used for instance to summarize gene expression profile data or word-document counts. To be robust to outliers and differences in scale across features, a matrix factorization step is usually preceded by ad-hoc feature normalization steps, such as \texttt{tf-idf} scaling or data whitening. We propose in this work to learn these normalization operators jointly with the factorization itself. More precisely, given a $d\times n$ matrix $X$ of $d$ features measured on $n$ individuals, we propose to learn the parameters of quantile normalization operators that can operate row-wise on the values of $X$ and/or of its factorization $UV$ to improve the quality of the low-rank representation of $X$ itself. This optimization is facilitated by the introduction of a new differentiable quantile normalization operator built using optimal transport, providing new results on top of existing work by (Cuturi et al. 2019). We demonstrate the applicability of these techniques on synthetic and genomics datasets.