OpenRaiser/Intern-Atlas huggingface dataset:
Towards Demystifying Membership Inference Attacks
Venue: arXiv cs.CR 2019
Year: 2019

Membership inference attacks seek to infer membership of individual training instances of a model to which an adversary has black-box access through a machine learning-as-a-service API. In providing an in-depth characterization of membership privacy risks against machine learning models, this paper presents a comprehensive study towards demystifying membership inference attacks from two complimentary perspectives. First, we provide a generalized formulation of the development of a black-box membership inference attack model. Second, we characterize the importance of model choice on model vulnerability through a systematic evaluation of a variety of machine learning models and model combinations using multiple datasets. Through formal analysis and empirical evidence from extensive experimentation, we characterize under what conditions a model may be vulnerable to such black-box membership inference attacks. We show that membership inference vulnerability is data-driven and corresponding attack models are largely transferable. Though different model types display different vulnerabilities to membership inference, so do different datasets. Our empirical results additionally show that (1) using the type of target model under attack within the attack model may not increase attack effectiveness and (2) collaborative learning exposes vulnerabilities to membership inference risks when the adversary is a participant. We also discuss countermeasure and mitigation strategies.
Shape of the Cloak: Formal Analysis of Clock Skew-Based Intrusion Detection System in Controller Area Networks
Venue: arXiv cs.CR 2019
Year: 2019

This paper presents a new masquerade attack called the cloaking attack and provides formal analyses for clock skew-based Intrusion Detection Systems (IDSs) that detect masquerade attacks in the Controller Area Network (CAN) in automobiles. In the cloaking attack, the adversary manipulates the message inter-transmission times of spoofed messages by adding delays so as to emulate a desired clock skew and avoid detection. In order to predict and characterize the impact of the cloaking attack in terms of the attack success probability on a given CAN bus and IDS, we develop formal models for two clock skew-based IDSs, i.e., the state-of-the-art (SOTA) IDS and its adaptation to the widely used Network Time Protocol (NTP), using parameters of the attacker, the detector, and the hardware platform. To the best of our knowledge, this is the first paper that provides formal analyses of clock skew-based IDSs in automotive CAN. We implement the cloaking attack on two hardware testbeds, a prototype and a real vehicle (the University of Washington (UW) EcoCAR), and demonstrate its effectiveness against both the SOTA and NTP-based IDSs. We validate our formal analyses through extensive experiments for different messages, IDS settings, and vehicles. By comparing each predicted attack success probability curve against its experimental curve, we find that the average prediction error is within 3.0% for the SOTA IDS and 5.7% for the NTP-based IDS.
Searching for the closest-pair in a query translate
Venue: arXiv cs.CG 2019
Year: 2019

We consider a range-search variant of the closest-pair problem. Let $\varGamma$ be a fixed shape in the plane. We are interested in storing a given set of $n$ points in the plane in some data structure such that for any specified translate of $\varGamma$, the closest pair of points contained in the translate can be reported efficiently. We present results on this problem for two important settings: when $\varGamma$ is a polygon (possibly with holes) and when $\varGamma$ is a general convex body whose boundary is smooth. When $\varGamma$ is a polygon, we present a data structure using $O(n)$ space and $O(\log n)$ query time, which is asymptotically optimal. When $\varGamma$ is a general convex body with a smooth boundary, we give a near-optimal data structure using $O(n \log n)$ space and $O(\log^2 n)$ query time. Our results settle some open questions posed by Xue et al. [SoCG 2018].
A Market-Based Framework for Multi-Resource Allocation in Fog Computing
Venue: arXiv cs.GT 2019
Year: 2019

Fog computing is transforming the network edge into an intelligent platform by bringing storage, computing, control, and networking functions closer to end-users, things, and sensors. How to allocate multiple resource types (e.g., CPU, memory, bandwidth) of capacity-limited heterogeneous fog nodes to competing services with diverse requirements and preferences in a fair and efficient manner is a challenging task. To this end, we propose a novel market-based resource allocation framework in which the services act as buyers and fog resources act as divisible goods in the market. The proposed framework aims to compute a market equilibrium (ME) solution at which every service obtains its favorite resource bundle under the budget constraint while the system achieves high resource utilization. This work extends the General Equilibrium literature by considering a practical case of satiated utility functions. Also, we introduce the notions of non-wastefulness and frugality for equilibrium selection, and rigorously demonstrate that all the non-wasteful and frugal ME are the optimal solutions to a convex program. Furthermore, the proposed equilibrium is shown to possess salient fairness properties including envy-freeness, sharing-incentive, and proportionality. Another major contribution of this work is to develop a privacy-preserving distributed algorithm, which is of independent interest, for computing an ME while allowing market participants to obfuscate their private information. Finally, extensive performance evaluation is conducted to verify our theoretical analyses.
Modular Mechanistic Networks: On Bridging Mechanistic and Phenomenological Models with Deep Neural Networks in Natural Language Processing
Venue: arXiv cs.CL 2019
Year: 2019

Natural language processing (NLP) can be done using either top-down (theory driven) and bottom-up (data driven) approaches, which we call mechanistic and phenomenological respectively. The approaches are frequently considered to stand in opposition to each other. Examining some recent approaches in deep learning we argue that deep neural networks incorporate both perspectives and, furthermore, that leveraging this aspect of deep learning may help in solving complex problems within language technology, such as modelling language and perception in the domain of spatial cognition.
A Unified Approximation Framework for Compressing and Accelerating Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Deep neural networks (DNNs) have achieved significant success in a variety of real world applications, i.e., image classification. However, tons of parameters in the networks restrict the efficiency of neural networks due to the large model size and the intensive computation. To address this issue, various approximation techniques have been investigated, which seek for a light weighted network with little performance degradation in exchange of smaller model size or faster inference. Both low-rankness and sparsity are appealing properties for the network approximation. In this paper we propose a unified framework to compress the convolutional neural networks (CNNs) by combining these two properties, while taking the nonlinear activation into consideration. Each layer in the network is approximated by the sum of a structured sparse component and a low-rank component, which is formulated as an optimization problem. Then, an extended version of alternating direction method of multipliers (ADMM) with guaranteed convergence is presented to solve the relaxed optimization problem. Experiments are carried out on VGG-16, AlexNet and GoogLeNet with large image classification datasets. The results outperform previous work in terms of accuracy degradation, compression rate and speedup ratio. The proposed method is able to remarkably compress the model (with up to 4.9x reduction of parameters) at a cost of little loss or without loss on accuracy.
On Generation of Virtual Outputs via Signal Injection: Application to Observer Design for Electromechanical Systems
Venue: arXiv cs.SY 2019
Year: 2019

Probing signal injection is a well-established technique to extract additional information from a weakly (or non) observable dynamical system. Using averaging theory, a framework to analyse such schemes for general nonlinear systems has been recently proposed in [Combes et. al., 2016], where it is shown that the signal injection may be used to generate a new high frequency component of the systems output that can be used for state observation or controller design. A key step for the success of this technique is the implementation of a filter to reconstruct this virtual output from the measurement of the overall systems output. The main contribution of this paper is to propose a new filter with guaranteed convergence properties that outperforms the classical designs. The method is applied to a general class of electromechanical systems, and its performance is assessed via simulations and experiments on the benchmark example of a 1-dof magnetic levitation system.
Aggregated Learning: A Deep Learning Framework Based on Information-Bottleneck Vector Quantization
Venue: arXiv cs.LG 2019
Year: 2019

Based on the notion of information bottleneck (IB), we formulate a quantization problem called "IB quantization". We show that IB quantization is equivalent to learning based on the IB principle. Under this equivalence, the standard neural network models can be viewed as scalar (single sample) IB quantizers. It is known, from conventional rate-distortion theory, that scalar quantizers are inferior to vector (multi-sample) quantizers. Such a deficiency then inspires us to develop a novel learning framework, AgrLearn, that corresponds to vector IB quantizers for learning with neural networks. Unlike standard networks, AgrLearn simultaneously optimizes against multiple data samples. We experimentally verify that AgrLearn can result in significant improvements when applied to several current deep learning architectures for image recognition and text classification. We also empirically show that AgrLearn can reduce up to 80% of the training samples needed for ResNet training.
A Many-Objective Evolutionary Algorithm Based on Decomposition and Local Dominance
Venue: arXiv cs.NE 2019
Year: 2019

Many-objective evolutionary algorithms (MOEAs), especially the decomposition-based MOEAs, have attracted wide attention in recent years. Recent studies show that a well designed combination of the decomposition method and the domination method can improve the performance ,i.e., convergence and diversity, of a MOEA. In this paper, a novel way of combining the decomposition method and the domination method is proposed. More precisely, a set of weight vectors is employed to decompose a given many-objective optimization problem(MaOP), and a hybrid method of the penalty-based boundary intersection function and dominance is proposed to compare local solutions within a subpopulation defined by a weight vector. A MOEA based on the hybrid method is implemented and tested on problems chosen from two famous test suites, i.e., DTLZ and WFG. The experimental results show that our algorithm is very competitive in dealing with MaOPs. Subsequently, our algorithm is extended to solve constraint MaOPs, and the constrained version of our algorithm also shows good performance in terms of convergence and diversity. These reveals that using dominance locally and combining it with the decomposition method can effectively improve the performance of a MOEA.
Automatic Short Answer Grading and Feedback Using Text Mining Methods
Venue: arXiv cs.CL 2019
Year: 2019

Automatic grading is not a new approach but the need to adapt the latest technology to automatic grading has become very important. As the technology has rapidly became more powerful on scoring exams and essays, especially from the 1990s onwards, partially or wholly automated grading systems using computational methods have evolved and have become a major area of research. In particular, the demand of scoring of natural language responses has created a need for tools that can be applied to automatically grade these responses. In this paper, we focus on the concept of automatic grading of short answer questions such as are typical in the UK GCSE system, and providing useful feedback on their answers to students. We present experimental results on a dataset provided from the introductory computer science class in the University of North Texas. We first apply standard data mining techniques to the corpus of student answers for the purpose of measuring similarity between the student answers and the model answer. This is based on the number of common words. We then evaluate the relation between these similarities and marks awarded by scorers. We then consider an approach that groups student answers into clusters. Each cluster would be awarded the same mark, and the same feedback given to each answer in a cluster. In this manner, we demonstrate that clusters indicate the groups of students who are awarded the same or the similar scores. Words in each cluster are compared to show that clusters are constructed based on how many and which words of the model answer have been used. The main novelty in this paper is that we design a model to predict marks based on the similarities between the student answers and the model answer.
Filter Distillation for Network Compression
Venue: arXiv cs.CV 2019
Year: 2019

In this paper we introduce Principal Filter Analysis (PFA), an easy to use and effective method for neural network compression. PFA exploits the correlation between filter responses within network layers to recommend a smaller network that maintain as much as possible the accuracy of the full model. We propose two algorithms: the first allows users to target compression to specific network property, such as number of trainable variable (footprint), and produces a compressed model that satisfies the requested property while preserving the maximum amount of spectral energy in the responses of each layer, while the second is a parameter-free heuristic that selects the compression used at each layer by trying to mimic an ideal set of uncorrelated responses. Since PFA compresses networks based on the correlation of their responses we show in our experiments that it gains the additional flexibility of adapting each architecture to a specific domain while compressing. PFA is evaluated against several architectures and datasets, and shows considerable compression rates without compromising accuracy, e.g., for VGG-16 on CIFAR-10, CIFAR-100 and ImageNet, PFA achieves a compression rate of 8x, 3x, and 1.4x with an accuracy gain of 0.4%, 1.4% points, and 2.4% respectively. Our tests show that PFA is competitive with state-of-the-art approaches while removing adoption barriers thanks to its practical implementation, intuitive philosophy and ease of use.
Physical Unclonable Function-based Key Sharing for IoT Security
Venue: arXiv cs.CR 2019
Year: 2019

In many Industry Internet of Things (IIoT) applications, resources like CPU, memory, and battery power are limited and cannot afford the classic cryptographic security solutions. Silicon Physical Unclonable Function (PUF) is a lightweight security primitive that exploits manufacturing variations during the chip fabrication process for key generation and/or device authentication. However, traditional weak PUFs such as Ring Oscillator (RO) PUF generate chip-unique key for each device, which restricts their application in security protocols where the same key is required to be shared in resource-constrained devices. In order to address this issue, we propose a PUF-based key sharing method for the first time. The basic idea is to implement one-to-one input-output mapping with Lookup Table (LUT)-based interstage crossing structures in each level of inverters of RO PUF. Individual customization on configuration bits of interstage crossing structure and different RO selections with challenges bring high flexibility. Therefore, with the flexible configuration of interstage crossing structures and challenges, CRO PUF can generate the same shared key for resource-constrained devices, which enables a new application for lightweight key sharing protocols.
Comparison of FaaS Orchestration Systems
Venue: arXiv cs.DC 2019
Year: 2019

Since the appearance of Amazon Lambda in 2014, all major cloud providers have embraced the Function as a Service (FaaS) model, because of its enormous potential for a wide variety of applications. As expected (and also desired), the competition is fierce in the serverless world, and includes aspects such as the run-time support for the orchestration of serverless functions. In this regard, the three major production services are currently Amazon Step Functions (December 2016), Azure Durable Functions (June 2017), and IBM Composer (October 2017), still young and experimental projects with a long way ahead. In this article, we will compare and analyze these three serverless orchestration systems under a common evaluation framework. We will study their architectures, programming and billing models, and their effective support for parallel execution, among others. Through a series of experiments, we will also evaluate the run-time overhead of the different infrastructures for different types of workflows.
Guidesort: Simpler Optimal Deterministic Sorting for the Parallel Disk Model
Venue: arXiv cs.DS 2019
Year: 2019

A new algorithm, Guidesort, for sorting in the uniprocessor variant of the parallel disk model (PDM) of Vitter and Shriver is presented. The algorithm is deterministic and executes a number of (parallel) I/O operations that comes within a constant factor $C$ of the optimum. The algorithm and its analysis are simpler than those proposed in previous work, and the achievable constant factor $C$ of essentially 3 appears to be smaller than for all other known deterministic algorithms, at least for plausible parameter values.
Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Tracking
Venue: arXiv cs.CV 2019
Year: 2019

With efficient appearance learning models, Discriminative Correlation Filter (DCF) has been proven to be very successful in recent video object tracking benchmarks and competitions. However, the existing DCF paradigm suffers from two major issues, i.e., spatial boundary effect and temporal filter degradation. To mitigate these challenges, we propose a new DCF-based tracking method. The key innovations of the proposed method include adaptive spatial feature selection and temporal consistent constraints, with which the new tracker enables joint spatial-temporal filter learning in a lower dimensional discriminative manifold. More specifically, we apply structured spatial sparsity constraints to multi-channel filers. Consequently, the process of learning spatial filters can be approximated by the lasso regularisation. To encourage temporal consistency, the filter model is restricted to lie around its historical value and updated locally to preserve the global structure in the manifold. Last, a unified optimisation framework is proposed to jointly select temporal consistency preserving spatial features and learn discriminative filters with the augmented Lagrangian method. Qualitative and quantitative evaluations have been conducted on a number of well-known benchmarking datasets such as OTB2013, OTB50, OTB100, Temple-Colour, UAV123 and VOT2018. The experimental results demonstrate the superiority of the proposed method over the state-of-the-art approaches.
Who needs category theory?
Venue: arXiv cs.LO 2019
Year: 2019

In mathematical applications, category theory remains a contentious issue, with enthusiastic fans and a skeptical majority. In a muted form this split applies to the authors of this note. When we learned that the only mathematically sound foundation of topological quantum computing in the literature is based on category theory, the skeptical author suggested to "decategorize" the foundation. But we discovered, to our surprise, that category theory (or something like it) is necessary for the purpose, for computational reasons. The goal of this note is to give a high-level explanation of that necessity, which avoids details and which suggests that the case of topological quantum computing is far from unique.
A note on full weight spectrum codes
Venue: arXiv cs.IT 2019
Year: 2019

A linear $ [n,k]_q $ code $ C $ is said to be a full weight spectrum (FWS) code if there exist codewords of each nonzero weight less than or equal to $ n $. In this brief communication we determine necessary and sufficient conditions for the existence of linear $ [n,k]_q $ full weight spectrum (FWS) codes. Central to our approach is the geometric view of linear codes, whereby columns of a generator matrix correspond to points in $ PG(k-1,q) $.
Critical Observability for Automata and Petri Nets
Venue: arXiv cs.SY 2019
Year: 2019

Critical observability is a property of cyber-physical systems to detect whether the current state belongs to a set of critical states. In safety-critical applications, critical states model operations that may be unsafe or of a particular interest. De Santis et al. introduced critical observability for linear switching systems, and Pola et al. adapted it for discrete-event systems, focusing on algorithmic complexity. We study the computational complexity of deciding critical observability for systems modeled as (networks of) finite-state automata and Petri nets. We show that deciding critical observability is (i) NL-complete for finite automata, that is, it is efficiently verifiable on parallel computers, (ii) PSPACE-complete for networks of finite automata, that is, it is very unlikely solvable in polynomial time, and (iii) undecidable for labeled Petri nets, but becoming decidable if the set of critical states (markings) is finite or co-finite, in which case the problem is as hard as the non-reachability problem for Petri nets.
Robust Tracking with Model Mismatch for Fast and Safe Planning: an SOS Optimization Approach
Venue: arXiv cs.SY 2019
Year: 2019

In the pursuit of real-time motion planning, a commonly adopted practice is to compute a trajectory by running a planning algorithm on a simplified, low-dimensional dynamical model, and then employ a feedback tracking controller that tracks such a trajectory by accounting for the full, high-dimensional system dynamics. While this strategy of planning with model mismatch generally yields fast computation times, there are no guarantees of dynamic feasibility, which hampers application to safety-critical systems. Building upon recent work that addressed this problem through the lens of Hamilton-Jacobi (HJ) reachability, we devise an algorithmic framework whereby one computes, offline, for a pair of "planner" (i.e., low-dimensional) and "tracking" (i.e., high-dimensional) models, a feedback tracking controller and associated tracking bound. This bound is then used as a safety margin when generating motion plans via the low-dimensional model. Specifically, we harness the computational tool of sum-of-squares (SOS) programming to design a bilinear optimization algorithm for the computation of the feedback tracking controller and associated tracking bound. The algorithm is demonstrated via numerical experiments, with an emphasis on investigating the trade-off between the increased computational scalability afforded by SOS and its intrinsic conservativeness. Collectively, our results enable scaling the appealing strategy of planning with model mismatch to systems that are beyond the reach of HJ analysis, while maintaining safety guarantees.