OpenRaiser/Intern-Atlas huggingface dataset:
An Optimal Game Approach for Heterogeneous Vehicular Network Selection with Varying Network Performance
Venue: arXiv cs.NI 2019
Year: 2019

Most conventional heterogeneous network selection strategies applied in heterogeneous vehicular network regard the performance of each network constant in various traffic scenarios. This assumption leads such strategies to be ineffective in the real-world performance-changing scenarios. To solve this problem, we propose an optimal game approach for heterogeneous vehicular network selection under conditions in which the performance parameters of some networks are changing. Terminals attempting to switch to the network with higher evaluation is formulated as a multi-play non-cooperative game. Heterogeneous vehicular network characteristics are thoroughly accounted for to adjust the game strategy and adapt to the vehicular environment for stability and rapid convergence. A multi-play non-cooperative game model is built to formulate network selection. A probabilistic strategy is used to gradually drive players toward convergence to prevent instability. Furthermore, a system prototype was built at the Connected and Automated Vehicle Test bed of Chang'an University (CAVTest). Its corresponding test results indicate that the proposed approach can effectively suppress the ping-pong effect caused by massive handoffs due to varying network performance and thus well outperforms the single-play strategy.
An Open-Source Benchmark Suite for Cloud and IoT Microservices
Venue: arXiv cs.DC 2019
Year: 2019

Cloud services have recently started undergoing a major shift from monolithic applications, to graphs of hundreds of loosely-coupled microservices. Microservices fundamentally change a lot of assumptions current cloud systems are designed with, and present both opportunities and challenges when optimizing for quality of service (QoS) and utilization. In this paper we explore the implications microservices have across the cloud system stack. We first present DeathStarBench, a novel, open-source benchmark suite built with microservices that is representative of large end-to-end services, modular and extensible. DeathStarBench includes a social network, a media service, an e-commerce site, a banking system, and IoT applications for coordination control of UAV swarms. We then use DeathStarBench to study the architectural characteristics of microservices, their implications in networking and operating systems, their challenges with respect to cluster management, and their trade-offs in terms of application design and programming frameworks. Finally, we explore the tail at scale effects of microservices in real deployments with hundreds of users, and highlight the increased pressure they put on performance predictability.
Learning to Auto Weight: Entirely Data-driven and Highly Efficient Weighting Framework
Venue: arXiv cs.LG 2019
Year: 2019

Example weighting algorithm is an effective solution to the training bias problem, however, most previous typical methods are usually limited to human knowledge and require laborious tuning of hyperparameters. In this paper, we propose a novel example weighting framework called Learning to Auto Weight (LAW). The proposed framework finds step-dependent weighting policies adaptively, and can be jointly trained with target networks without any assumptions or prior knowledge about the dataset. It consists of three key components: Stage-based Searching Strategy (3SM) is adopted to shrink the huge searching space in a complete training process; Duplicate Network Reward (DNR) gives more accurate supervision by removing randomness during the searching process; Full Data Update (FDU) further improves the updating efficiency. Experimental results demonstrate the superiority of weighting policy explored by LAW over standard training pipeline. Compared with baselines, LAW can find a better weighting schedule which achieves much more superior accuracy on both biased CIFAR and ImageNet.
Self-Calibration Methods for Uncontrolled Environments in Sensor Networks: A Reference Survey
Venue: arXiv cs.NI 2019
Year: 2019

Growing progress in sensor technology has constantly expanded the number and range of low-cost, small, and portable sensors on the market, increasing the number and type of physical phenomena that can be measured with wirelessly connected sensors. Large-scale deployments of wireless sensor networks (WSN) involving hundreds or thousands of devices and limited budgets often constrain the choice of sensing hardware, which generally has reduced accuracy, precision, and reliability. Therefore, it is challenging to achieve good data quality and maintain error-free measurements during the whole system lifetime. Self-calibration or recalibration in ad hoc sensor networks to preserve data quality is essential, yet challenging, for several reasons, such as the existence of random noise and the absence of suitable general models. Calibration performed in the field, without accurate and controlled instrumentation, is said to be in an uncontrolled environment. This paper provides current and fundamental self-calibration approaches and models for wireless sensor networks in uncontrolled environments.
Detecting Missing Dependencies and Notifiers in Puppet Programs
Venue: arXiv cs.SE 2019
Year: 2019

Puppet is a popular computer system configuration management tool. It provides abstractions that enable administrators to setup their computer systems declaratively. Its use suffers from two potential pitfalls. First, if ordering constraints are not specified whenever an abstraction depends on another, the non-deterministic application of abstractions can lead to race conditions. Second, if a service is not tied to its resources through notification constructs, the system may operate in a stale state whenever a resource gets modified. Such faults can degrade a computing infrastructure's availability and functionality. We have developed an approach that identifies these issues through the analysis of a Puppet program and its system call trace. Specifically, we present a formal model for traces, which allows us to capture the interactions of Puppet abstractions with the file system. By analyzing these interactions we identify (1) abstractions that are related to each other (e.g., operate on the same file), and (2) abstractions that should act as notifiers so that changes are correctly propagated. We then check the relationships from the trace's analysis against the program's dependency graph: a representation containing all the ordering constraints and notifications declared in the program. If a mismatch is detected, our system reports a potential fault. We have evaluated our method on a large set of Puppet modules, and discovered 57 previously unknown issues in 30 of them. Benchmarking further shows that our approach can analyze in minutes real-world configurations with a magnitude measured in thousands of lines and millions of system calls.
Experimental Interference Robustness Evaluation of IEEE 802.15.4-2015 OQPSK-DSSS and SUN-OFDM Physical Layers
Venue: arXiv cs.NI 2019
Year: 2019

In this paper, we experimentally evaluate and compare the robustness against interference of the OQPSK-DSSS (Offset Quadrature Phase Shift Keying - Direct Sequence Spread Spectrum) and the SUN-OFDM (Smart Utility Network - Orthogonal Frequency Division Multiplexing) physical layers, as defined in the IEEE 802.15.4-2015 standard. The objective of this study is to provide a comprehensive analysis of the impact different types of interference produce on these modulations, in terms of the resulting PDR (Packet Delivery Ratio) and depending on the length of the packet being transmitted. The results show that the SUN-OFDM physical layer provides significant benefits compared to the ubiquitous OQPSK-DSSS in terms of interference robustness, regardless of the interference type and the packet length. Overall, this demonstrates the suitability of choosing the SUN-OFDM physical layer when deploying low-power wireless networks in industrial scenarios, specially taking into consideration the possibility of trading-off robustness and spectrum efficiency depending on the application requirements.
On Timed Scope-bounded Context-sensitive Languages
Venue: arXiv cs.FL 2019
Year: 2019

In (DLT 2016) we studied timed context sensitive languages characterized by multiple stack push down automata (MPA), with an explicit bound on number of stages where in each stage at most one stack is used (k-round MPA). In this paper, we continue our work on timed MPA and study a subclass in which a symbol corresponding to a stack being pushed in it must be popped within fixed number of contexts of that stack---scope-bounded push-down automata with multiple stacks (k-scope MPA). We use Visibly Push-down Alphabet and Event Clocks to show that timed k-scope MPA have decidable reachability problem; are closed under Boolean operations; and have an equivalent logical characterization.
Radial Prediction Layer
Venue: arXiv cs.LG 2019
Year: 2019

For a broad variety of critical applications, it is essential to know how confident a classification prediction is. In this paper, we discuss the drawbacks of softmax to calculate class probabilities and to handle uncertainty in Bayesian neural networks. We introduce a new kind of prediction layer called radial prediction layer (RPL) to overcome these issues. In contrast to the softmax classification, RPL is based on the open-world assumption. Therefore, the class prediction probabilities are much more meaningful to assess the uncertainty concerning the novelty of the input. We show that neural networks with RPLs can be learned in the same way as neural networks using softmax. On a 2D toy data set (spiral data), we demonstrate the fundamental principles and advantages. On the real-world ImageNet data set, we show that the open-world properties are beneficially fulfilled. Additionally, we show that RPLs are less sensible to adversarial attacks on the MNIST data set. Due to its features, we expect RPL to be beneficial in a broad variety of applications, especially in critical environments, such as medicine or autonomous driving.
Temporally Coupled Dynamical Movement Primitives in Cartesian Space
Venue: arXiv cs.RO 2019
Year: 2019

Control of robot orientation in Cartesian space implicates some difficulties, because the rotation group SO(3) is not contractible, and only globally contractible state spaces support continuous and globally asymptotically stable feedback control systems. In this paper, unit quaternions are used to represent orientations, and it is first shown that the unit quaternion set minus one single point is contractible. This is used to design a control system for temporally coupled dynamical movement primitives (DMPs) in Cartesian space. The functionality of the control system is verified experimentally on an industrial robot.
A Novel General Compact Model Approach for 7nm Technology Node Circuit Optimization from Device Perspective and Beyond
Venue: arXiv cs.ET 2019
Year: 2019

This work presents a novel general compact model for 7nm technology node devices like FinFETs. As an extension of previous conventional compact model that based on some less accurate elements including one-dimensional Poisson equation for three-dimensional devices and analytical equations for short channel effects, quantum effects and other physical effects, the general compact model combining few TCAD calibrated compact models with statistical methods can eliminate the tedious physical derivations. The general compact model has the advantages of efficient extraction, high accuracy, strong scaling capability and excellent transfer capability. As a demo application, two key design knobs of FinFET and their multiple impacts on RC control ESD power clamp circuit are systematically evaluated with implementation of the newly proposed general compact model, accounting for device design, circuit performance optimization and variation control. The performance of ESD power clamp can be improved extremely. This framework is also suitable for pathfinding researches on 5nm node gate-all-around devices, like nanowire (NW) FETs, nanosheet (NSH) FETs and beyond.
Kernel Conditional Density Operators
Venue: arXiv cs.LG 2019
Year: 2019

We introduce a novel conditional density estimation model termed the conditional density operator (CDO). It naturally captures multivariate, multimodal output densities and shows performance that is competitive with recent neural conditional density models and Gaussian processes. The proposed model is based on a novel approach to the reconstruction of probability densities from their kernel mean embeddings by drawing connections to estimation of Radon-Nikodym derivatives in the reproducing kernel Hilbert space (RKHS). We prove finite sample bounds for the estimation error in a standard density reconstruction scenario, independent of problem dimensionality. Interestingly, when a kernel is used that is also a probability density, the CDO allows us to both evaluate and sample the output density efficiently. We demonstrate the versatility and performance of the proposed model on both synthetic and real-world data.
Expression of Fractals Through Neural Network Functions
Venue: arXiv cs.LG 2019
Year: 2019

To help understand the underlying mechanisms of neural networks (NNs), several groups have, in recent years, studied the number of linear regions $\ell$ of piecewise linear functions generated by deep neural networks (DNN). In particular, they showed that $\ell$ can grow exponentially with the number of network parameters $p$, a property often used to explain the advantages of DNNs over shallow NNs in approximating complicated functions. Nonetheless, a simple dimension argument shows that DNNs cannot generate all piecewise linear functions with $\ell$ linear regions as soon as $\ell > p$. It is thus natural to seek to characterize specific families of functions with $\ell$ linear regions that can be constructed by DNNs. Iterated Function Systems (IFS) generate sequences of piecewise linear functions $F_k$ with a number of linear regions exponential in $k$. We show that, under mild assumptions, $F_k$ can be generated by a NN using only $\mathcal{O}(k)$ parameters. IFS are used extensively to generate, at low computational cost, natural-looking landscape textures in artificial images. They have also been proposed for compression of natural images, albeit with less commercial success. The surprisingly good performance of this fractal-based compression suggests that our visual system may lock in, to some extent, on self-similarities in images. The combination of this phenomenon with the capacity, demonstrated here, of DNNs to efficiently approximate IFS may contribute to the success of DNNs, particularly striking for image processing tasks, as well as suggest new algorithms for representing self similarities in images based on the DNN mechanism.
Trust but Verify: An Information-Theoretic Explanation for the Adversarial Fragility of Machine Learning Systems, and a General Defense against Adversarial Attacks
Venue: arXiv cs.CR 2019
Year: 2019

Deep-learning based classification algorithms have been shown to be susceptible to adversarial attacks: minor changes to the input of classifiers can dramatically change their outputs, while being imperceptible to humans. In this paper, we present a simple hypothesis about a feature compression property of artificial intelligence (AI) classifiers and present theoretical arguments to show that this hypothesis successfully accounts for the observed fragility of AI classifiers to small adversarial perturbations. Drawing on ideas from information and coding theory, we propose a general class of defenses for detecting classifier errors caused by abnormally small input perturbations. We further show theoretical guarantees for the performance of this detection method. We present experimental results with (a) a voice recognition system, and (b) a digit recognition system using the MNIST database, to demonstrate the effectiveness of the proposed defense methods. The ideas in this paper are motivated by a simple analogy between AI classifiers and the standard Shannon model of a communication system.
Learning In Practice: Reasoning About Quantization
Venue: arXiv cs.LG 2019
Year: 2019

There is a mismatch between the standard theoretical analyses of statistical machine learning and how learning is used in practice. The foundational assumption supporting the theory is that we can represent features and models using real-valued parameters. In practice, however, we do not use real numbers at any point during training or deployment. Instead, we rely on discrete and finite quantizations of the reals, typically floating points. In this paper, we propose a framework for reasoning about learning under arbitrary quantizations. Using this formalization, we prove the convergence of quantization-aware versions of the Perceptron and Frank-Wolfe algorithms. Finally, we report the results of an extensive empirical study of the impact of quantization using a broad spectrum of datasets.
Enhancing Salient Object Segmentation Through Attention
Venue: arXiv cs.CV 2019
Year: 2019

Segmenting salient objects in an image is an important vision task with ubiquitous applications. The problem becomes more challenging in the presence of a cluttered and textured background, low resolution and/or low contrast images. Even though existing algorithms perform well in segmenting most of the object(s) of interest, they often end up segmenting false positives due to resembling salient objects in the background. In this work, we tackle this problem by iteratively attending to image patches in a recurrent fashion and subsequently enhancing the predicted segmentation mask. Saliency features are estimated independently for every image patch, which are further combined using an aggregation strategy based on a Convolutional Gated Recurrent Unit (ConvGRU) network. The proposed approach works in an end-to-end manner, removing background noise and false positives incrementally. Through extensive evaluation on various benchmark datasets, we show superior performance to the existing approaches without any post-processing.
CGaP: Continuous Growth and Pruning for Efficient Deep Learning
Venue: arXiv cs.CV 2019
Year: 2019

Today a canonical approach to reduce the computation cost of Deep Neural Networks (DNNs) is to pre-define an over-parameterized model before training to guarantee the learning capacity, and then prune unimportant learning units (filters and neurons) during training to improve model compactness. We argue it is unnecessary to introduce redundancy at the beginning of the training but then reduce redundancy for the ultimate inference model. In this paper, we propose a Continuous Growth and Pruning (CGaP) scheme to minimize the redundancy from the beginning. CGaP starts the training from a small network seed, then expands the model continuously by reinforcing important learning units, and finally prunes the network to obtain a compact and accurate model. As the growth phase favors important learning units, CGaP provides a clear learning purpose to the pruning phase. Experimental results on representative datasets and DNN architectures demonstrate that CGaP outperforms previous pruning-only approaches that deal with pre-defined structures. For VGG-19 on CIFAR-100 and SVHN datasets, CGaP reduces the number of parameters by 78.9% and 85.8%, FLOPs by 53.2% and 74.2%, respectively; For ResNet-110 On CIFAR-10, CGaP reduces 64.0% number of parameters and 63.3% FLOPs.
Local Label Propagation for Large-Scale Semi-Supervised Learning
Venue: arXiv cs.CV 2019
Year: 2019

A significant issue in training deep neural networks to solve supervised learning tasks is the need for large numbers of labelled datapoints. The goal of semi-supervised learning is to leverage ubiquitous unlabelled data, together with small quantities of labelled data, to achieve high task performance. Though substantial recent progress has been made in developing semi-supervised algorithms that are effective for comparatively small datasets, many of these techniques do not scale readily to the large (unlaballed) datasets characteristic of real-world applications. In this paper we introduce a novel approach to scalable semi-supervised learning, called Local Label Propagation (LLP). Extending ideas from recent work on unsupervised embedding learning, LLP first embeds datapoints, labelled and otherwise, in a common latent space using a deep neural network. It then propagates pseudolabels from known to unknown datapoints in a manner that depends on the local geometry of the embedding, taking into account both inter-point distance and local data density as a weighting on propagation likelihood. The parameters of the deep embedding are then trained to simultaneously maximize pseudolabel categorization performance as well as a metric of the clustering of datapoints within each psuedo-label group, iteratively alternating stages of network training and label propagation. We illustrate the utility of the LLP method on the ImageNet dataset, achieving results that outperform previous state-of-the-art scalable semi-supervised learning algorithms by large margins, consistently across a wide variety of training regimes. We also show that the feature representation learned with LLP transfers well to scene recognition in the Places 205 dataset.
Clairvoyant State Machine Replication
Venue: arXiv cs.DC 2019
Year: 2019

We propose a new protocol for the generalized consensus problem in asynchronous systems subject to Byzantine server failures. The protocol solves the consensus problem in a setting in which information about conflict between transactions is available (such information can be in the form of transaction read and write sets). Unlike most prior proposals (for generalized or classical consensus), which use a leader to order transactions, this protocol is leaderless, and relies on non-skipping timestamps for transaction ordering. Being leaderless, the protocol does not need to pause for leader elections. The use of non-skipping timestamps permits servers to commit transactions as soon as they know that no conflicting transaction can be ordered earlier. For n servers of which f may be faulty, this protocol requires n > 4f.
DSReg: Using Distant Supervision as a Regularizer
Venue: arXiv cs.CL 2019
Year: 2019

In this paper, we aim at tackling a general issue in NLP tasks where some of the negative examples are highly similar to the positive examples, i.e., hard-negative examples. We propose the distant supervision as a regularizer (DSReg) approach to tackle this issue. The original task is converted to a multi-task learning problem, in which distant supervision is used to retrieve hard-negative examples. The obtained hard-negative examples are then used as a regularizer. The original target objective of distinguishing positive examples from negative examples is jointly optimized with the auxiliary task objective of distinguishing softened positive (i.e., hard-negative examples plus positive examples) from easy-negative examples. In the neural context, this can be done by outputting the same representation from the last neural layer to different $softmax$ functions. Using this strategy, we can improve the performance of baseline models in a range of different NLP tasks, including text classification, sequence labeling and reading comprehension.