OpenRaiser/Intern-Atlas huggingface dataset:
Scaling Pseudonymous Authentication for Large Mobile Systems
Venue: arXiv cs.CR 2019
Year: 2019

The central building block of secure and privacy-preserving Vehicular Communication (VC) systems is a Vehicular Public-Key Infrastructure (VPKI), which provides vehicles with multiple anonymized credentials, termed pseudonyms. These pseudonyms are used to ensure message authenticity and integrity while preserving vehicle (thus passenger) privacy. In the light of emerging large-scale multi-domain VC environments, the efficiency of the VPKI and, more broadly, its scalability are paramount. By the same token, preventing misuse of the credentials, in particular, Sybil-based misbehavior, and managing "honest-but-curious" insiders are other facets of a challenging problem. In this paper, we leverage a state-of-the-art VPKI system and enhance its functionality towards a highly-available, dynamically-scalable, and resilient design; this ensures that the system remains operational in the presence of benign failures or resource depletion attacks, and that it dynamically scales out, or possibly scales in, according to request arrival rates. Our full-blown implementation on the Google Cloud Platform shows that deploying large-scale and efficient VPKI can be cost-effective.
Segmentation-Aware Hyperspectral Image Classification
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we propose an unified hyperspectral image classification method which takes three-dimensional hyperspectral data cube as an input and produces a classification map. In the proposed method, a deep neural network which uses spectral and spatial information together with residual connections, and pixel affinity network based segmentation-aware superpixels are used together. In the architecture, segmentation-aware superpixels run on the initial classification map of deep residual network, and apply majority voting on obtained results. Experimental results show that our propoped method yields state-of-the-art results in two benchmark datasets. Moreover, we also show that the segmentation-aware superpixels have great contribution to the success of hyperspectral image classification methods in cases where training data is insufficient.
Separating Overlapping Tissue Layers from Microscopy Images
Venue: arXiv cs.CV 2019
Year: 2019

Manual preparation of tissue slices for microscopy imaging can introduce tissue tears and overlaps. Typically, further digital processing algorithms such as registration and 3D reconstruction from tissue image stacks cannot handle images with tissue tear/overlap artifacts, and so such images are usually discarded. In this paper, we propose an imaging model and an algorithm to digitally separate overlapping tissue data of mouse brain images into two layers. We show the correctness of our model and the algorithm by comparing our results with the ground truth.
Convergence Analyses of Online ADAM Algorithm in Convex Setting and Two-Layer ReLU Neural Network
Venue: arXiv cs.LG 2019
Year: 2019

Nowadays, online learning is an appealing learning paradigm, which is of great interest in practice due to the recent emergence of large scale applications such as online advertising placement and online web ranking. Standard online learning assumes a finite number of samples while in practice data is streamed infinitely. In such a setting gradient descent with a diminishing learning rate does not work. We first introduce regret with rolling window, a new performance metric for online streaming learning, which measures the performance of an algorithm on every fixed number of contiguous samples. At the same time, we propose a family of algorithms based on gradient descent with a constant or adaptive learning rate and provide very technical analyses establishing regret bound properties of the algorithms. We cover the convex setting showing the regret of the order of the square root of the size of the window in the constant and dynamic learning rate scenarios. Our proof is applicable also to the standard online setting where we provide the first analysis of the same regret order (the previous proofs have flaws). We also study a two layer neural network setting with ReLU activation. In this case we establish that if initial weights are close to a stationary point, the same square root regret bound is attainable. We conduct computational experiments demonstrating a superior performance of the proposed algorithms.
FQL: An Extensible Feature Query Language and Toolkit on Searching Software Characteristics for HPC Applications
Venue: arXiv cs.SE 2019
Year: 2019

The amount of large-scale scientific computing software is dramatically increasing. In this work, we designed a new language, named feature query language (FQL), to collect and extract software features from a quick static code analysis. We designed and implemented an FQL toolkit to automatically detect and present the software features using an extensible query repository. Several large-scale, high performance computing (HPC) scientific codes have been used in the paper to demonstrate the HPC-related feature extraction and information collection. Although we emphasized the HPC features in the study, the toolkit can be easily extended to answer general software feature questions, such as coding pattern and hardware dependency.
Causal Discovery with Cascade Nonlinear Additive Noise Models
Venue: arXiv cs.LG 2019
Year: 2019

Identification of causal direction between a causal-effect pair from observed data has recently attracted much attention. Various methods based on functional causal models have been proposed to solve this problem, by assuming the causal process satisfies some (structural) constraints and showing that the reverse direction violates such constraints. The nonlinear additive noise model has been demonstrated to be effective for this purpose, but the model class is not transitive--even if each direct causal relation follows this model, indirect causal influences, which result from omitted intermediate causal variables and are frequently encountered in practice, do not necessarily follow the model constraints; as a consequence, the nonlinear additive noise model may fail to correctly discover causal direction. In this work, we propose a cascade nonlinear additive noise model to represent such causal influences--each direct causal relation follows the nonlinear additive noise model but we observe only the initial cause and final effect. We further propose a method to estimate the model, including the unmeasured intermediate variables, from data, under the variational auto-encoder framework. Our theoretical results show that with our model, causal direction is identifiable under suitable technical conditions on the data generation process. Simulation results illustrate the power of the proposed method in identifying indirect causal relations across various settings, and experimental results on real data suggest that the proposed model and method greatly extend the applicability of causal discovery based on functional causal models in nonlinear cases.
Combine PPO with NES to Improve Exploration
Venue: arXiv cs.LG 2019
Year: 2019

We introduce two approaches for combining neural evolution strategy (NES) and proximal policy optimization (PPO): parameter transfer and parameter space noise. Parameter transfer is a PPO agent with parameters transferred from a NES agent. Parameter space noise is to directly add noise to the PPO agent`s parameters. We demonstrate that PPO could benefit from both methods through experimental comparison on discrete action environments as well as continuous control tasks
Incorporating Human Domain Knowledge in 3D LiDAR-based Semantic Segmentation
Venue: arXiv cs.RO 2019
Year: 2019

This work studies semantic segmentation using 3D LiDAR data. Popular deep learning methods applied for this task require a large number of manual annotations to train the parameters. We propose a new method that makes full use of the advantages of traditional methods and deep learning methods via incorporating human domain knowledge into the neural network model to reduce the demand for large numbers of manual annotations and improve the training efficiency. We first pretrain a model with autogenerated samples from a rule-based classifier so that human knowledge can be propagated into the network. Based on the pretrained model, only a small set of annotations is required for further fine-tuning. Quantitative experiments show that the pretrained model achieves better performance than random initialization in almost all cases; furthermore, our method can achieve similar performance with fewer manual annotations.
Network Slicing for Vehicular Communication
Venue: arXiv cs.NI 2019
Year: 2019

Ultra-reliable vehicle-to-everything (V2X) communication is essential for enabling the next generation of intelligent vehicles. V2X communication is a growing area of communication, that connects vehicles to neighboring vehicles (V2V), infrastructure (V2I) and pedestrians (V2P). Network slicing is one of the promising technology for connectivity of the next generation devices, creating several logical networks on a common and programmable physical infrastructure. Network slicing offers an efficient way to satisfy the diverse use case requirements by exploiting the benefits of shared physical infrastructure. In this regard, we propose a network slicing based communication solution for vehicular networks. In this work, we model a highway scenario with vehicles having heterogeneous traffic demands. The autonomous driving slice (safety messages) and the infotainment slice (video stream) are the two logical slices created on a common infrastructure. We formulated a network clustering and slicing algorithm to partition the vehicles into different clusters and allocate slice leaders to each cluster. Slice leaders serve its clustered vehicles with high quality V2V links and forwards safety information with low latency. On the other hand, road side unit provides infotainment service using high quality V2I links. An extensive Long Term Evolution Advanced (LTE-A) system level simulator with enhancement of cellular V2X (C-V2X) standard is used to evaluate the performance of the proposed method, in which it is shown that the proposed network slicing technique achieves low latency and high-reliability communication.
Non-monotone DR-submodular Maximization: Approximation and Regret Guarantees
Venue: arXiv cs.LG 2019
Year: 2019

Diminishing-returns (DR) submodular optimization is an important field with many real-world applications in machine learning, economics and communication systems. It captures a subclass of non-convex optimization that provides both practical and theoretical guarantees. In this paper, we study the fundamental problem of maximizing non-monotone DR-submodular functions over down-closed and general convex sets in both offline and online settings. First, we show that for offline maximizing non-monotone DR-submodular functions over a general convex set, the Frank-Wolfe algorithm achieves an approximation guarantee which depends on the convex set. Next, we show that the Stochastic Gradient Ascent algorithm achieves a 1/4-approximation ratio with the regret of $O(1/\sqrt{T})$ for the problem of maximizing non-monotone DR-submodular functions over down-closed convex sets. These are the first approximation guarantees in the corresponding settings. Finally we benchmark these algorithms on problems arising in machine learning domain with the real-world datasets.
Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks
Venue: arXiv cs.CV 2019
Year: 2019

The Convolutional Neural Networks (CNNs) generate the feature representation of complex objects by collecting hierarchical and different parts of semantic sub-features. These sub-features can usually be distributed in grouped form in the feature vector of each layer, representing various semantic entities. However, the activation of these sub-features is often spatially affected by similar patterns and noisy backgrounds, resulting in erroneous localization and identification. We propose a Spatial Group-wise Enhance (SGE) module that can adjust the importance of each sub-feature by generating an attention factor for each spatial location in each semantic group, so that every individual group can autonomously enhance its learnt expression and suppress possible noise. The attention factors are only guided by the similarities between the global and local feature descriptors inside each group, thus the design of SGE module is extremely lightweight with \emph{almost no extra parameters and calculations}. Despite being trained with only category supervisions, the SGE component is extremely effective in highlighting multiple active areas with various high-order semantics (such as the dog's eyes, nose, etc.). When integrated with popular CNN backbones, SGE can significantly boost the performance of image recognition tasks. Specifically, based on ResNet50 backbones, SGE achieves 1.2\% Top-1 accuracy improvement on the ImageNet benchmark and 1.0$\sim$2.0\% AP gain on the COCO benchmark across a wide range of detectors (Faster/Mask/Cascade RCNN and RetinaNet). Codes and pretrained models are available at https://github.com/implus/PytorchInsight.
On the Average Case of MergeInsertion
Venue: arXiv cs.DS 2019
Year: 2019

MergeInsertion, also known as the Ford-Johnson algorithm, is a sorting algorithm which, up to today, for many input sizes achieves the best known upper bound on the number of comparisons. Indeed, it gets extremely close to the information-theoretic lower bound. While the worst-case behavior is well understood, only little is known about the average case. This work takes a closer look at the average case behavior. In particular, we establish an upper bound of $n \log n - 1.4005n + o(n)$ comparisons. We also give an exact description of the probability distribution of the length of the chain a given element is inserted into and use it to approximate the average number of comparisons numerically. Moreover, we compute the exact average number of comparisons for $n$ up to 148. Furthermore, we experimentally explore the impact of different decision trees for binary insertion. To conclude, we conduct experiments showing that a slightly different insertion order leads to a better average case and we compare the algorithm to the recent combination with (1,2)-Insertionsort by Iwama and Teruyama.
Towards Artificial Learning Companions for Mental Imagery-based Brain-Computer Interfaces
Venue: arXiv cs.HC 2019
Year: 2019

Mental Imagery based Brain-Computer Interfaces (MI-BCI) enable their users to control an interface, e.g., a prosthesis, by performing mental imagery tasks only, such as imagining a right arm movement while their brain activity is measured and processed by the system. Designing and using a BCI requires users to learn how to produce different and stable patterns of brain activity for each of the mental imagery tasks. However, current training protocols do not enable every user to acquire the skills required to use BCIs. These training protocols are most likely one of the main reasons why BCIs remain not reliable enough for wider applications outside research laboratories. Learning companions have been shown to improve training in different disciplines, but they have barely been explored for BCIs so far. This article aims at investigating the potential benefits learning companions could bring to BCI training by improving the feedback, i.e., the information provided to the user, which is primordial to the learning process and yet have proven both theoretically and practically inadequate in BCI. This paper first presents the potentials of BCI and the limitations of current training approaches. Then, it reviews both the BCI and learning companion literature regarding three main characteristics of feedback: its appearance, its social and emotional components and its cognitive component. From these considerations, this paper draws some guidelines, identify open challenges and suggests potential solutions to design and use learning companions for BCIs.
Nullspace Structure in Model Predictive Control
Venue: arXiv cs.RO 2019
Year: 2019

Robotic tasks can be accomplished by exploiting different forms of redundancies. This work focuses on planning redundancy within Model Predictive Control (MPC) in which several paths can be considered within the MPC time horizon. We present the nullspace structure in MPC with a quadratic approximation of the cost and a linearization of the dynamics. We exploit the low rank structure of the precision matrices used in MPC (encapsulating spatiotemporal information) to perform hierarchical task planning, and show how nullspace computation can be treated as a fusion problem (computed with a product of Gaussian experts). We illustrate the approach using proof-of-concept examples with point mass objects and simulated robotics applications.
Price of Dependence: Stochastic Submodular Maximization with Dependent Items
Venue: arXiv cs.SI 2019
Year: 2019

In this paper, we study the stochastic submodular maximization problem with dependent items subject to packing constraints such as matroid and knapsack constraints. The input of our problem is a finite set of items, and each item is in a particular state from a set of possible states. After picking an item, we are able to observe its state. We assume a monotone and submodular utility function over items and states, and our objective is to select a group of items adaptively so as to maximize the expected utility. Previous studies on stochastic submodular maximization often assume that items' states are independent, however, this assumption may not hold in general. This motivates us to study the stochastic submodular maximization problem with dependent items. We first introduce the concept of \emph{degree of independence} to capture the degree to which one item's state is dependent on others'. Then we propose a non-adaptive policy that approximates the optimal adaptive policy within a factor of $α(1-e^{-\fracκ{2}+\fracκ{18m^2}}-\frac{κ+2}{3mκ})$ where the value of $α$ is depending on the type of constraints, e.g., $α=1$ for matroid constraint, $κ$ is the degree of independence, e.g., $κ=1$ for independent items, and $m$ is the number of items. We also analyze the adaptivity gap, i.e., the ratio of the values of best adaptive policy and best non-adaptive policy, of our problem with prefix-closed constraints.
A consistent and comprehensive computational approach for general Fluid-Structure-Contact Interaction problems
Venue: arXiv cs.CE 2019
Year: 2019

We present a consistent approach that allows to solve challenging general nonlinear fluid-structure-contact interaction (FSCI) problems. The underlying continuous formulation includes both "no-slip" fluid-structure interaction as well as frictionless contact between multiple elastic bodies. The respective interface conditions in normal and tangential orientation and especially the role of the fluid stress within the region of closed contact are discussed for the general problem of FSCI. To ensure continuity of the tangential constraints from no-slip to frictionless contact, a transition is enabled by using the general Navier condition with varying slip length. Moreover, the fluid stress in the contact zone is obtained by an extension approach as it plays a crucial role for the lift-off behavior of contacting bodies. With the given continuity of the spatially continuous formulation, continuity of the discrete problem (which is essential for the convergence of Newton's method) is reached naturally. As topological changes of the fluid domain are an inherent challenge in FSCI configurations, a non-interface fitted Cut Finite Element Method (CutFEM) is applied to discretize the fluid domain. All interface conditions, that is the `no-slip' FSI, the general Navier condition, and frictionless contact are incorporated using Nitsche based methods, thus retaining the continuity and consistency of the model. To account for the strong interaction between the fluid and solid discretization, the overall coupled discrete system is solved monolithically. Numerical examples of varying complexity are presented to corroborate the developments.
Adversarially Robust Distillation
Venue: arXiv cs.LG 2019
Year: 2019

Knowledge distillation is effective for producing small, high-performance neural networks for classification, but these small networks are vulnerable to adversarial attacks. This paper studies how adversarial robustness transfers from teacher to student during knowledge distillation. We find that a large amount of robustness may be inherited by the student even when distilled on only clean images. Second, we introduce Adversarially Robust Distillation (ARD) for distilling robustness onto student networks. In addition to producing small models with high test accuracy like conventional distillation, ARD also passes the superior robustness of large networks onto the student. In our experiments, we find that ARD student models decisively outperform adversarially trained networks of identical architecture in terms of robust accuracy, surpassing state-of-the-art methods on standard robustness benchmarks. Finally, we adapt recent fast adversarial training methods to ARD for accelerated robust distillation.
Evaluating the Effects of Control Surfaces Failure on the GTM
Venue: arXiv cs.SY 2019
Year: 2019

Despite the advances in aircraft guidance and control systems technology, Loss of Control remains as the main cause of the fatal accidents of large transport aircraft. Loss of Control is defined as excursion beyond the allowable flight envelope and is often a consequence of upset condition if improper maneuver is implemented by the pilot. Hence, extensive research in recent years has focused on improving the current fault tolerant control systems and developing new strategies for loss of control prevention and recovery systems. However, success of such systems requires the perception of the damaged aircraft's dynamic behavior and performance, and understanding of its new flight envelope. This paper provides a comprehensive understanding of lateral control surfaces' failure effect on the NASA Generic Transport Model's maneuvering flight envelope; which is a set of attainable steady state maneuvers herein referred to as trim points. The study utilizes a massive database of the Generic Transport Model's high-fidelity maneuvering flight envelopes computed for the unimpaired case and wide ranges of aileron and rudder failure cases at different flight conditions. Flight envelope boundary is rigorously investigated and the key parameters confining the trim points at different boundary sections are identified. Trend analysis of the impaired flight envelopes and the corresponding limiting factors is performed which demonstrates the effect of various failure degrees on the remaining feasible trim points. Results of the post-failure analysis can be employed in emergency path planning and have potential uses in the development of aircraft resilient control and upset recovery systems.