OpenRaiser/Intern-Atlas huggingface dataset:
Efficient Training of Deep Classifiers for Wireless Source Identification using Test SNR Estimates
Venue: arXiv cs.LG 2020
Year: 2020

We study efficient deep learning training algorithms that process received wireless signals, if a test Signal to Noise Ratio (SNR) estimate is available. We focus on two tasks that facilitate source identification: 1- Identifying the modulation type, 2- Identifying the wireless technology and channel in the 2.4 GHz ISM band. For benchmarking, we rely on recent literature on testing deep learning algorithms against two well-known datasets. We first demonstrate that using training data corresponding only to the test SNR value leads to dramatic reductions in training time while incurring a small loss in average test accuracy, as it improves the accuracy for low SNR values. Further, we show that an erroneous test SNR estimate with a small positive offset is better for training than another having the same error magnitude with a negative offset. Secondly, we introduce a greedy training SNR Boosting algorithm that leads to uniform improvement in accuracy across all tested SNR values, while using a small subset of training SNR values at each test SNR. Finally, we demonstrate the potential of bootstrap aggregating (Bagging) based on training SNR values to improve generalization at low test SNR values with scarcity of training data.
EVA: An Encrypted Vector Arithmetic Language and Compiler for Efficient Homomorphic Computation
Venue: arXiv cs.CR 2020
Year: 2020

Fully-Homomorphic Encryption (FHE) offers powerful capabilities by enabling secure offloading of both storage and computation, and recent innovations in schemes and implementations have made it all the more attractive. At the same time, FHE is notoriously hard to use with a very constrained programming model, a very unusual performance profile, and many cryptographic constraints. Existing compilers for FHE either target simpler but less efficient FHE schemes or only support specific domains where they can rely on expert-provided high-level runtimes to hide complications. This paper presents a new FHE language called Encrypted Vector Arithmetic (EVA), which includes an optimizing compiler that generates correct and secure FHE programs, while hiding all the complexities of the target FHE scheme. Bolstered by our optimizing compiler, programmers can develop efficient general-purpose FHE applications directly in EVA. For example, we have developed image processing applications using EVA, with a very few lines of code. EVA is designed to also work as an intermediate representation that can be a target for compiling higher-level domain-specific languages. To demonstrate this, we have re-targeted CHET, an existing domain-specific compiler for neural network inference, onto EVA. Due to the novel optimizations in EVA, its programs are on average 5.3x faster than those generated by CHET. We believe that EVA would enable a wider adoption of FHE by making it easier to develop FHE applications and domain-specific FHE compilers.
Formula Derivation and Analysis of the VINS-Mono
Venue: arXiv cs.RO 2020
Year: 2020

The VINS-Mono is a monocular visual-inertial 6 DOF state estimator proposed by Aerial Robotics Group of HKUST in 2017. It can be performed on MAVs, smartphones and many other intelligent platforms. Because of the excellent robustness, accuracy and scalability, it has gained extensive attention worldwide. In this manuscript, the main equations including IMU pre-integration, visual-inertial co-initialization and tightly-coupled nonlinear optimization are derived and analyzed.
Cooperative Perception for 3D Object Detection in Driving Scenarios using Infrastructure Sensors
Venue: arXiv cs.CV 2020
Year: 2020

3D object detection is a common function within the perception system of an autonomous vehicle and outputs a list of 3D bounding boxes around objects of interest. Various 3D object detection methods have relied on fusion of different sensor modalities to overcome limitations of individual sensors. However, occlusion, limited field-of-view and low-point density of the sensor data cannot be reliably and cost-effectively addressed by multi-modal sensing from a single point of view. Alternatively, cooperative perception incorporates information from spatially diverse sensors distributed around the environment as a way to mitigate these limitations. This article proposes two schemes for cooperative 3D object detection using single modality sensors. The early fusion scheme combines point clouds from multiple spatially diverse sensing points of view before detection. In contrast, the late fusion scheme fuses the independently detected bounding boxes from multiple spatially diverse sensors. We evaluate the performance of both schemes, and their hybrid combination, using a synthetic cooperative dataset created in two complex driving scenarios, a T-junction and a roundabout. The evaluation shows that the early fusion approach outperforms late fusion by a significant margin at the cost of higher communication bandwidth. The results demonstrate that cooperative perception can recall more than 95% of the objects as opposed to 30% for single-point sensing in the most challenging scenario. To provide practical insights into the deployment of such system, we report how the number of sensors and their configuration impact the detection performance of the system.
Implicit supervision for fault detection and segmentation of emerging fault types with Deep Variational Autoencoders
Venue: arXiv cs.LG 2020
Year: 2020

Data-driven fault diagnostics of safety-critical systems often faces the challenge of a complete lack of labeled data associated with faulty system conditions (i.e., fault types) at training time. Since an unknown number and nature of fault types can arise during deployment, data-driven fault diagnostics in this scenario is an open-set learning problem. Most of the algorithms for open-set diagnostics are one-class classification and unsupervised algorithms that do not leverage all the available labeled and unlabeled data in the learning algorithm. As a result, their fault detection and segmentation performance (i.e., identifying and separating faults of different types) are sub-optimal. With this work, we propose training a variational autoencoder (VAE) with labeled and unlabeled samples while inducing implicit supervision on the latent representation of the healthy conditions. This, together with a modified sampling process of VAE, creates a compact and informative latent representation that allows good detection and segmentation of unseen fault types using existing one-class and clustering algorithms. We refer to the proposed methodology as "knowledge induced variational autoencoder with adaptive sampling" (KIL-AdaVAE). The fault detection and segmentation capabilities of the proposed methodology are demonstrated in a new simulated case study using the Advanced Geared Turbofan 30000 (AGTF30) dynamical model under real flight conditions. In an extensive comparison, we demonstrate that the proposed method outperforms other learning strategies (supervised learning, supervised learning with embedding and semi-supervised learning) and deep learning algorithms, yielding significant performance improvements on fault detection and fault segmentation.
Joint Sparse Recovery Using Signal Space Matching Pursuit
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we put forth a new joint sparse recovery algorithm called signal space matching pursuit (SSMP). The key idea of the proposed SSMP algorithm is to sequentially investigate the support of jointly sparse vectors to minimize the subspace distance to the residual space. Our performance guarantee analysis indicates that SSMP accurately reconstructs any row $K$-sparse matrix of rank $r$ in the full row rank scenario if the sampling matrix $\mathbf{A}$ satisfies $\text{krank}(\mathbf{A}) \ge K+1$, which meets the fundamental minimum requirement on $\mathbf{A}$ to ensure exact recovery. We also show that SSMP guarantees exact reconstruction in at most $K-r+\lceil \frac{r}{L} \rceil$ iterations, provided that $\mathbf{A}$ satisfies the restricted isometry property (RIP) of order $L(K-r)+r+1$ with $$δ_{L(K-r)+r+1} < \max \left \{ \frac{\sqrt{r}}{\sqrt{K+\frac{r}{4}}+\sqrt{\frac{r}{4}}}, \frac{\sqrt{L}}{\sqrt{K}+1.15 \sqrt{L}} \right \},$$ where $L$ is the number of indices chosen in each iteration. This implies that the requirement on the RIP constant becomes less restrictive when $r$ increases. Such behavior seems to be natural but has not been reported for most of conventional methods. We further show that if $r=1$, then by running more than $K$ iterations, the performance guarantee of SSMP can be improved to $δ_{\lfloor 7.8K \rfloor} \le 0.155$. In addition, we show that under a suitable RIP condition, the reconstruction error of SSMP is upper bounded by a constant multiple of the noise power, which demonstrates the stability of SSMP under measurement noise. Finally, from extensive numerical experiments, we show that SSMP outperforms conventional joint sparse recovery algorithms both in noiseless and noisy scenarios.
Secure Communication Protocol for Smart Transportation Based on Vehicular Cloud
Venue: arXiv cs.CR 2020
Year: 2020

The pioneering concept of connected vehicles has transformed the way of thinking for researchers and entrepreneurs by collecting relevant data from nearby objects. However, this data is useful for a specific vehicle only. Moreover, vehicles get a high amount of data (e.g., traffic, safety, and multimedia infotainment) on the road. Thus, vehicles expect adequate storage devices for this data, but it is infeasible to have a large memory in each vehicle. Hence, the vehicular cloud computing (VCC) framework came into the picture to provide a storage facility by connecting a road-side-unit (RSU) with the vehicular cloud (VC). In this, data should be saved in an encrypted form to preserve security, but there is a challenge to search for information over encrypted data. Next, we understand that many of vehicular communication schemes are inefficient for data transmissions due to its poor performance results and vulnerable to different fundamental security attacks. Accordingly, on-device performance is critical, but data damages and secure on-time connectivity are also significant challenges in a public environment. Therefore, we propose reliable data transmission protocols for cutting-edge architecture to search data from the storage, to resist against various security attacks, and provide better performance results. Thus, the proposed data transmission protocol is useful in diverse smart city applications (business, safety, and entertainment) for the benefits of society.
Cryptanalysis of an Image Block Encryption Algorithm Based on Chaotic Maps
Venue: arXiv cs.CR 2020
Year: 2020

Recently, an image block encryption algorithm was proposed based on some well-known chaotic maps. The authors claim that the encryption algorithm achieves enough security level and high encryption speed at the same time. In this paper, we give a thorough security analysis on the algorithm from the perspective of modern cryptology and report some critical security defects on the algorithm. Given five chosen plain-images and the corresponding cipher-images, the attacker can obtain an equivalent secret key to successfully decrypt the other cipher-images encrypted with the same secret key. In addition, each security metric adopted in the security evaluation on the algorithm is questioned. The drawn lessons are generally applicable to many other image encryption algorithms.
Geometry and Analytics of the Multifacility Weber Problem
Venue: arXiv cs.CG 2020
Year: 2020

For the Weber problem of construction of the minimal cost planar weighted network connecting four terminals with two extra facilities, the solution by radicals is proposed. The conditions for the existence of the network in the assumed topology and the explicit formulae for coordinates of the facilities are presented. The obtained results are utilized for investigation of the network dynamics under variation of parameters. Extension of the results to the general Weber problem is also discussed.
Avoiding Spurious Local Minima in Deep Quadratic Networks
Venue: arXiv cs.LG 2020
Year: 2020

Despite their practical success, a theoretical understanding of the loss landscape of neural networks has proven challenging due to the high-dimensional, non-convex, and highly nonlinear structure of such models. In this paper, we characterize the training landscape of the mean squared error loss for neural networks with quadratic activation functions. We prove existence of spurious local minima and saddle points which can be escaped easily with probability one when the number of neurons is greater than or equal to the input dimension and the norm of the training samples is used as a regressor. We prove that deep overparameterized neural networks with quadratic activations benefit from similar nice landscape properties. Our theoretical results are independent of data distribution and fill the existing gap in theory for two-layer quadratic neural networks. Finally, we empirically demonstrate convergence to a global minimum for these problems.
Dual Adversarial Domain Adaptation
Venue: arXiv cs.LG 2020
Year: 2020

Unsupervised domain adaptation aims at transferring knowledge from the labeled source domain to the unlabeled target domain. Previous adversarial domain adaptation methods mostly adopt the discriminator with binary or $K$-dimensional output to perform marginal or conditional alignment independently. Recent experiments have shown that when the discriminator is provided with domain information in both domains and label information in the source domain, it is able to preserve the complex multimodal information and high semantic information in both domains. Following this idea, we adopt a discriminator with $2K$-dimensional output to perform both domain-level and class-level alignments simultaneously in a single discriminator. However, a single discriminator can not capture all the useful information across domains and the relationships between the examples and the decision boundary are rarely explored before. Inspired by multi-view learning and latest advances in domain adaptation, besides the adversarial process between the discriminator and the feature extractor, we also design a novel mechanism to make two discriminators pit against each other, so that they can provide diverse information for each other and avoid generating target features outside the support of the source domain. To the best of our knowledge, it is the first time to explore a dual adversarial strategy in domain adaptation. Moreover, we also use the semi-supervised learning regularization to make the representations more discriminative. Comprehensive experiments on two real-world datasets verify that our method outperforms several state-of-the-art domain adaptation methods.
Low-Budget Label Query through Domain Alignment Enforcement
Venue: arXiv cs.CV 2020
Year: 2020

Deep learning revolution happened thanks to the availability of a massive amount of labelled data which have contributed to the development of models with extraordinary inference capabilities. Despite the public availability of a large quantity of datasets, to address specific requirements it is often necessary to generate a new set of labelled data. Quite often, the production of labels is costly and sometimes it requires specific know-how to be fulfilled. In this work, we tackle a new problem named low-budget label query that consists in suggesting to the user a small (low budget) set of samples to be labelled, from a completely unlabelled dataset, with the final goal of maximizing the classification accuracy on that dataset. In this work we first improve an Unsupervised Domain Adaptation (UDA) method to better align source and target domains using consistency constraints, reaching the state of the art on a few UDA tasks. Finally, using the previously trained model as reference, we propose a simple yet effective selection method based on uniform sampling of the prediction consistency distribution, which is deterministic and steadily outperforms other baselines as well as competing models on a large variety of publicly available datasets.
Motivic clustering schemes for directed graphs
Venue: arXiv cs.LG 2020
Year: 2020

Motivated by the concept of network motifs we construct certain clustering methods (functors) which are parametrized by a given collection of motifs (or representers).
Video Saliency Prediction Using Enhanced Spatiotemporal Alignment Network
Venue: arXiv cs.CV 2020
Year: 2020

Due to a variety of motions across different frames, it is highly challenging to learn an effective spatiotemporal representation for accurate video saliency prediction (VSP). To address this issue, we develop an effective spatiotemporal feature alignment network tailored to VSP, mainly including two key sub-networks: a multi-scale deformable convolutional alignment network (MDAN) and a bidirectional convolutional Long Short-Term Memory (Bi-ConvLSTM) network. The MDAN learns to align the features of the neighboring frames to the reference one in a coarse-to-fine manner, which can well handle various motions. Specifically, the MDAN owns a pyramidal feature hierarchy structure that first leverages deformable convolution (Dconv) to align the lower-resolution features across frames, and then aggregates the aligned features to align the higher-resolution features, progressively enhancing the features from top to bottom. The output of MDAN is then fed into the Bi-ConvLSTM for further enhancement, which captures the useful long-time temporal information along forward and backward timing directions to effectively guide attention orientation shift prediction under complex scene transformation. Finally, the enhanced features are decoded to generate the predicted saliency map. The proposed model is trained end-to-end without any intricate post processing. Extensive evaluations on four VSP benchmark datasets demonstrate that the proposed method achieves favorable performance against state-of-the-art methods. The source codes and all the results will be released.
Cost of Dietary Data Acquisition with Smart Group Catering
Venue: arXiv cs.CY 2020
Year: 2020

The need for dietary data management is growing with public awareness of food intakes. As a result, there are increasing deployments of smart canteens where dietary data is collected through either Radio Frequency Identification (RFID) or Computer Vision(CV)-based solutions. As human labor is involved in both cases, manpower allocation is critical to data quality. Where manpower requirements are underestimated, data quality is compromised. This paper has studied the relation between the quality of dietary data and the manpower invested, using numerical simulations based on real data collected from multiple smart canteens. We found that in both RFID and CV-based systems, the long-term cost of dietary data acquisition is dominated by manpower. Our study provides a comprehensive understanding of the cost composition for dietary data acquisition and useful insights toward future cost effective systems.
Phase-based Information for Voice Pathology Detection
Venue: arXiv cs.SD 2020
Year: 2020

In most current approaches of speech processing, information is extracted from the magnitude spectrum. However recent perceptual studies have underlined the importance of the phase component. The goal of this paper is to investigate the potential of using phase-based features for automatically detecting voice disorders. It is shown that group delay functions are appropriate for characterizing irregularities in the phonation. Besides the respect of the mixed-phase model of speech is discussed. The proposed phase-based features are evaluated and compared to other parameters derived from the magnitude spectrum. Both streams are shown to be interestingly complementary. Furthermore phase-based features turn out to convey a great amount of relevant information, leading to high discrimination performance.
Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics
Venue: arXiv cs.LG 2020
Year: 2020

Many real-world control problems involve both discrete decision variables - such as the choice of control modes, gear switching or digital outputs - as well as continuous decision variables - such as velocity setpoints, control gains or analogue outputs. However, when defining the corresponding optimal control or reinforcement learning problem, it is commonly approximated with fully continuous or fully discrete action spaces. These simplifications aim at tailoring the problem to a particular algorithm or solver which may only support one type of action space. Alternatively, expert heuristics are used to remove discrete actions from an otherwise continuous space. In contrast, we propose to treat hybrid problems in their 'native' form by solving them with hybrid reinforcement learning, which optimizes for discrete and continuous actions simultaneously. In our experiments, we first demonstrate that the proposed approach efficiently solves such natively hybrid reinforcement learning problems. We then show, both in simulation and on robotic hardware, the benefits of removing possibly imperfect expert-designed heuristics. Lastly, hybrid reinforcement learning encourages us to rethink problem definitions. We propose reformulating control problems, e.g. by adding meta actions, to improve exploration or reduce mechanical wear and tear.
$4$-uniform BCT permutations from generalized butterfly structure
Venue: arXiv cs.IT 2020
Year: 2020

As a generalization of Dillon's APN permutation, butterfly structure and generalizations have been of great interest since they generate permutations with the best known differential and nonlinear properties over the field of size $2^{4k+2}$. Complementary to these results, we show in this paper that butterfly structure, more precisely the closed butterfly also yields permutations with the best boomerang uniformity, a new and important parameter related to boomerang-style attacks. This is the sixth known infinite family of permutations in the literature with the best known boomerang uniformity over such fields.
Reject Illegal Inputs with Generative Classifier Derived from Any Discriminative Classifier
Venue: arXiv cs.LG 2020
Year: 2020

Generative classifiers have been shown promising to detect illegal inputs including adversarial examples and out-of-distribution samples. Supervised Deep Infomax~(SDIM) is a scalable end-to-end framework to learn generative classifiers. In this paper, we propose a modification of SDIM termed SDIM-\emph{logit}. Instead of training generative classifier from scratch, SDIM-\emph{logit} first takes as input the logits produced any given discriminative classifier, and generate logit representations; then a generative classifier is derived by imposing statistical constraints on logit representations. SDIM-\emph{logit} could inherit the performance of the discriminative classifier without loss. SDIM-\emph{logit} incurs a negligible number of additional parameters, and can be efficiently trained with base classifiers fixed. We perform \emph{classification with rejection}, where test samples whose class conditionals are smaller than pre-chosen thresholds will be rejected without predictions. Experiments on illegal inputs, including adversarial examples, samples with common corruptions, and out-of-distribution~(OOD) samples show that allowed to reject a portion of test samples, SDIM-\emph{logit} significantly improves the performance on the left test sets.