OpenRaiser/Intern-Atlas huggingface dataset:
Delay-optimal policies in partial fork-join systems with redundancy and random slowdowns
Venue: arXiv cs.PF 2019
Year: 2019

We consider a large distributed service system consisting of $n$ homogeneous servers with infinite capacity FIFO queues. Jobs arrive as a Poisson process of rate $λn/k_n$ (for some positive constant $λ$ and integer $k_n$). Each incoming job consists of $k_n$ identical tasks that can be executed in parallel, and that can be encoded into at least $k_n$ "replicas" of the same size (by introducing redundancy) so that the job is considered to be completed when any $k_n$ replicas associated with it finish their service. Moreover, we assume that servers can experience random slowdowns in their processing rate so that the service time of a replica is the product of its size and a random slowdown. First, we assume that the server slowdowns are shifted exponential and independent of the replica sizes. In this setting we show that the delay of a typical job is asymptotically minimized (as $n\to\infty$) when the number of replicas per task is a constant that only depends on the arrival rate $λ$, and on the expected slowdown of servers. Second, we introduce a new model for the server slowdowns in which larger tasks experience less variable slowdowns than smaller tasks. In this setting we show that, under the class of policies where all replicas start their service at the same time, the delay of a typical job is asymptotically minimized (as $n\to\infty$) when the number of replicas per task is made to depend on the actual size of the tasks being replicated, with smaller tasks being replicated more than larger tasks.
Non-Gaussianity of Stochastic Gradient Noise
Venue: arXiv cs.LG 2019
Year: 2019

What enables Stochastic Gradient Descent (SGD) to achieve better generalization than Gradient Descent (GD) in Neural Network training? This question has attracted much attention. In this paper, we study the distribution of the Stochastic Gradient Noise (SGN) vectors during the training. We observe that for batch sizes 256 and above, the distribution is best described as Gaussian at-least in the early phases of training. This holds across data-sets, architectures, and other choices.
The SWAX Benchmark: Attacking Biometric Systems with Wax Figures
Venue: arXiv cs.CV 2019
Year: 2019

A face spoofing attack occurs when an intruder attempts to impersonate someone who carries a gainful authentication clearance. It is a trending topic due to the increasing demand for biometric authentication on mobile devices, high-security areas, among others. This work introduces a new database named Sense Wax Attack dataset (SWAX), comprised of real human and wax figure images and videos that endorse the problem of face spoofing detection. The dataset consists of more than 1800 face images and 110 videos of 55 people/waxworks, arranged in training, validation and test sets with a large range in expression, illumination and pose variations. Experiments performed with baseline methods show that despite the progress in recent years, advanced spoofing methods are still vulnerable to high-quality violation attempts.
Multiphase flow prediction with deep neural networks
Venue: arXiv cs.LG 2019
Year: 2019

This paper proposes a deep neural network approach for predicting multiphase flow in heterogeneous domains with high computational efficiency. The deep neural network model is able to handle permeability heterogeneity in high dimensional systems, and can learn the interplay of viscous, gravity, and capillary forces from small data sets. Using the example of carbon dioxide (CO2) storage, we demonstrate that the model can generate highly accurate predictions of a CO2 saturation distribution given a permeability field, injection duration, injection rate, and injection location. The trained neural network model has an excellent ability to interpolate and to a limited extent, the ability to extrapolate beyond the training data ranges. To improve the prediction accuracy when the neural network model needs to extrapolate, we propose a transfer learning (fine-tuning) procedure that can quickly teach the neural network model new information without going through massive data collection and retraining. Based on this trained neural network model, a web-based tool is provided that allows users to perform CO2-water multiphase flow calculations online. With the tools provided in this paper, the deep neural network approach can provide a computationally efficient substitute for repetitive forward multiphase flow simulations, which can be adopted to the context of history matching and uncertainty quantification.
Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight
Venue: arXiv cs.RO 2019
Year: 2019

We propose a joint simulation and real-world learning framework for mapping navigation instructions and raw first-person observations to continuous control. Our model estimates the need for environment exploration, predicts the likelihood of visiting environment positions during execution, and controls the agent to both explore and visit high-likelihood positions. We introduce Supervised Reinforcement Asynchronous Learning (SuReAL). Learning uses both simulation and real environments without requiring autonomous flight in the physical environment during training, and combines supervised learning for predicting positions to visit and reinforcement learning for continuous control. We evaluate our approach on a natural language instruction-following task with a physical quadcopter, and demonstrate effective execution and exploration behavior.
Collaborative Graph Walk for Semi-supervised Multi-Label Node Classification
Venue: arXiv cs.LG 2019
Year: 2019

In this work, we study semi-supervised multi-label node classification problem in attributed graphs. Classic solutions to multi-label node classification follow two steps, first learn node embedding and then build a node classifier on the learned embedding. To improve the discriminating power of the node embedding, we propose a novel collaborative graph walk, named Multi-Label-Graph-Walk, to finely tune node representations with the available label assignments in attributed graphs via reinforcement learning. The proposed method formulates the multi-label node classification task as simultaneous graph walks conducted by multiple label-specific agents. Furthermore, policies of the label-wise graph walks are learned in a cooperative way to capture first the predictive relation between node labels and structural attributes of graphs; and second, the correlation among the multiple label-specific classification tasks. A comprehensive experimental study demonstrates that the proposed method can achieve significantly better multi-label classification performance than the state-of-the-art approaches and conduct more efficient graph exploration.
A deep active learning system for species identification and counting in camera trap images
Venue: arXiv cs.LG 2019
Year: 2019

Biodiversity conservation depends on accurate, up-to-date information about wildlife population distributions. Motion-activated cameras, also known as camera traps, are a critical tool for population surveys, as they are cheap and non-intrusive. However, extracting useful information from camera trap images is a cumbersome process: a typical camera trap survey may produce millions of images that require slow, expensive manual review. Consequently, critical information is often lost due to resource limitations, and critical conservation questions may be answered too slowly to support decision-making. Computer vision is poised to dramatically increase efficiency in image-based biodiversity surveys, and recent studies have harnessed deep learning techniques for automatic information extraction from camera trap images. However, the accuracy of results depends on the amount, quality, and diversity of the data available to train models, and the literature has focused on projects with millions of relevant, labeled training images. Many camera trap projects do not have a large set of labeled images and hence cannot benefit from existing machine learning techniques. Furthermore, even projects that do have labeled data from similar ecosystems have struggled to adopt deep learning methods because image classification models overfit to specific image backgrounds (i.e., camera locations). In this paper, we focus not on automating the labeling of camera trap images, but on accelerating this process. We combine the power of machine intelligence and human intelligence to build a scalable, fast, and accurate active learning system to minimize the manual work required to identify and count animals in camera trap images. Our proposed scheme can match the state of the art accuracy on a 3.2 million image dataset with as few as 14,100 manual labels, which means decreasing manual labeling effort by over 99.5%.
Intelligence via ultrafilters: structural properties of some intelligence comparators of deterministic Legg-Hutter agents
Venue: arXiv cs.AI 2019
Year: 2019

Legg and Hutter, as well as subsequent authors, considered intelligent agents through the lens of interaction with reward-giving environments, attempting to assign numeric intelligence measures to such agents, with the guiding principle that a more intelligent agent should gain higher rewards from environments in some aggregate sense. In this paper, we consider a related question: rather than measure numeric intelligence of one Legg- Hutter agent, how can we compare the relative intelligence of two Legg-Hutter agents? We propose an elegant answer based on the following insight: we can view Legg-Hutter agents as candidates in an election, whose voters are environments, letting each environment vote (via its rewards) which agent (if either) is more intelligent. This leads to an abstract family of comparators simple enough that we can prove some structural theorems about them. It is an open question whether these structural theorems apply to more practical intelligence measures.
Assessment of the Local Tchebichef Moments Method for Texture Classification by Fine Tuning Extraction Parameters
Venue: arXiv cs.CV 2019
Year: 2019

In this paper we use machine learning to study the application of Local Tchebichef Moments (LTM) to the problem of texture classification. The original LTM method was proposed by Mukundan (2014). The LTM method can be used for texture analysis in many different ways, either using the moment values directly, or more simply creating a relationship between the moment values of different orders, producing a histogram similar to those of Local Binary Pattern (LBP) based methods. The original method was not fully tested with large datasets, and there are several parameters that should be characterised for performance. Among these parameters are the kernel size, the moment orders and the weights for each moment. We implemented the LTM method in a flexible way in order to allow for the modification of the parameters that can affect its performance. Using four subsets from the Outex dataset (a popular benchmark for texture analysis), we used Random Forests to create models and to classify texture images, recording the standard metrics for each classifier. We repeated the process using several variations of the LBP method for comparison. This allowed us to find the best combination of orders and weights for the LTM method for texture classification.
Behavioral Security in Covert Communication Systems
Venue: arXiv cs.CR 2019
Year: 2019

The purpose of the covert communication system is to implement the communication process without causing third party perception. In order to achieve complete covert communication, two aspects of security issues need to be considered. The first one is to cover up the existence of information, that is, to ensure the content security of information; the second one is to cover up the behavior of transmitting information, that is, to ensure the behavioral security of communication. However, most of the existing information hiding models are based on the "Prisoners' Model", which only considers the content security of carriers, while ignoring the behavioral security of the sender and receiver. We think that this is incomplete for the security of covert communication. In this paper, we propose a new covert communication framework, which considers both content security and behavioral security in the process of information transmission. In the experimental part, we analyzed a large amount of collected real Twitter data to illustrate the security risks that may be brought to covert communication if we only consider content security and neglect behavioral security. Finally, we designed a toy experiment, pointing out that in addition to most of the existing content steganography, under the proposed new framework of covert communication, we can also use user's behavior to implement behavioral steganography. We hope this new proposed framework will help researchers to design better covert communication systems.
Face representation by deep learning: a linear encoding in a parameter space?
Venue: arXiv cs.LG 2019
Year: 2019

Recently, Convolutional Neural Networks (CNNs) have achieved tremendous performances on face recognition, and one popular perspective regarding CNNs' success is that CNNs could learn discriminative face representations from face images with complex image feature encoding. However, it is still unclear what is the intrinsic mechanism of face representation in CNNs. In this work, we investigate this problem by formulating face images as points in a shape-appearance parameter space, and our results demonstrate that: (i) The encoding and decoding of the neuron responses (representations) to face images in CNNs could be achieved under a linear model in the parameter space, in agreement with the recent discovery in primate IT face neurons, but different from the aforementioned perspective on CNNs' face representation with complex image feature encoding; (ii) The linear model for face encoding and decoding in the parameter space could achieve close or even better performances on face recognition and verification than state-of-the-art CNNs, which might provide new lights on the design strategies for face recognition systems; (iii) The neuron responses to face images in CNNs could not be adequately modelled by the axis model, a model recently proposed on face modelling in primate IT cortex. All these results might shed some lights on the often complained blackbox nature behind CNNs' tremendous performances on face recognition.
A Coordinated View of Cyberspace
Venue: arXiv cs.NI 2019
Year: 2019

Cyberspace is an online world created by growing network of computing and communication technologies. It is a virtual space of the Internet, paralleled to geographic space we are living on. As becoming a recognized component of our society, cyberspace gradually draws more attention in academic research. Many prior efforts have tried to represent and visualize cyberspace in geographic coordinate system (GCS) and network coordinate system (NCS). However, there are some disadvantages on these views. Firstly, mapping cyberspace in geographic space only reveals a partial characteristics of cyberspace, especially geographic characteristic of cyberspace. All what we could see is only the geographic information of cyberspace and tip of the iceberg of cyberspace. Secondly, NCS is established according to network topology and maps the position of each node in the coordinate system according to RTT (Round Trip Time) or network delays. However, this coordinate system changes dynamically with RTT changes or host connection status, resulting in the coordinate system not stable. Cyberspace, regarded as a second space in human life, is rather complex and multi-dimension. However, it is little known to us. It is in a great need of establishing its own coordinate system to tackle the challenging task of efficiently visualizing complex multi-dimensional cyberspace and get to know more about cyberspace. This paper aims to explore and visualize cyberspace. To best of our knowledge, we are firstly to establish a Cyberspace Coordination System (CyberCS) to represent and visualize cyberspace. CyberCS will make the representation of cyberspace easier or more concrete which is similar to Fourier transform. With the help of CyberCS, different parts and degrees of cyberspace are efficiently visualized and user can easily filter out the specific details of interest.
Orthogonal variance decomposition based feature selection
Venue: arXiv cs.LG 2019
Year: 2019

Existing feature selection methods fail to properly account for interactions between features when evaluating feature subsets. In this paper, we attempt to remedy this issue by using orthogonal variance decomposition to evaluate features. The orthogonality of the decomposition allows us to directly calculate the total contribution of a feature to the output variance. Thus we obtain an efficient algorithm for feature evaluation which takes into account interactions among features. Numerical experiments demonstrate that our method accurately identifies relevant features and improves the accuracy of numerical models.
Towards Combinational Relation Linking over Knowledge Graphs
Venue: arXiv cs.AI 2019
Year: 2019

Given a natural language phrase, relation linking aims to find a relation (predicate or property) from the underlying knowledge graph to match the phrase. It is very useful in many applications, such as natural language question answering, personalized recommendation and text summarization. However, the previous relation linking algorithms usually produce a single relation for the input phrase and pay little attention to a more general and challenging problem, i.e., combinational relation linking that extracts a subgraph pattern to match the compound phrase (e.g. mother-in-law). In this paper, we focus on the task of combinational relation linking over knowledge graphs. To resolve the problem, we design a systematic method based on the data-driven relation assembly technique, which is performed under the guidance of meta patterns. We also introduce external knowledge to enhance the system understanding ability. Finally, we conduct extensive experiments over the real knowledge graph to study the performance of the proposed method.
Kernel computations from large-scale random features obtained by Optical Processing Units
Venue: arXiv cs.ET 2019
Year: 2019

Approximating kernel functions with random features (RFs)has been a successful application of random projections for nonparametric estimation. However, performing random projections presents computational challenges for large-scale problems. Recently, a new optical hardware called Optical Processing Unit (OPU) has been developed for fast and energy-efficient computation of large-scale RFs in the analog domain. More specifically, the OPU performs the multiplication of input vectors by a large random matrix with complex-valued i.i.d. Gaussian entries, followed by the application of an element-wise squared absolute value operation - this last nonlinearity being intrinsic to the sensing process. In this paper, we show that this operation results in a dot-product kernel that has connections to the polynomial kernel, and we extend this computation to arbitrary powers of the feature map. Experiments demonstrate that the OPU kernel and its RF approximation achieve competitive performance in applications using kernel ridge regression and transfer learning for image classification. Crucially, thanks to the use of the OPU, these results are obtained with time and energy savings.
Designing Security and Privacy Requirements in Internet of Things: A Survey
Venue: arXiv cs.SE 2019
Year: 2019

The design and development process for the Internet of Things (IoT) applications is more complicated than that for desktop, mobile, or web applications. First, IoT applications require both software and hardware to work together across different nodes with different capabilities under different conditions. Secondly, IoT application development involves different software engineers such as desktop, web, embedded and mobile to cooperate. In addition, the development process required different software\hardware stacks to integrated together. Due to above complexities, more often non-functional requirements (such as security and privacy) tend to get ignored in IoT application development process. In this paper, we have reviewed techniques, methods and tools that are being developed to support incorporating security and privacy requirements into traditional application designs. By doing so, we aim to explore how those techniques could be applicable to the IoT domain. In this paper, we primarily focused on two different aspects: (1) design notations, models, and languages that facilitate capturing non-functional requirements (i.e., security and privacy), and (2) proactive and reactive interaction techniques that can be used to support and augment the IoT application design process. Our goal is not only to analyse past research work but also to discuss their applicability towards the IoT.
Scalable and Accurate Modeling of the Millimeter Wave Channel
Venue: arXiv cs.NI 2019
Year: 2019

Communication at millimeter wave (mmWave) frequencies is one of the main novelties introduced in the 5th generation (5G) of cellular networks. The opportunities and challenges associated with such high frequencies have stimulated a number of studies that rely on simulation for the evaluation of the proposed solutions. The accuracy of simulations largely depends on that of the channel model, but popular channel models for mmWaves, such as the Spatial Channel Models (SCMs), have high computational complexity and limit the scalability of the scenarios. This paper profiles the implementation of a widely-used SCM model for mmWave frequencies, and proposes a simplified version of the 3GPP SCM that reduces the computation time by up to 12.5 times while providing essentially the same distributions of several metrics, such as the Signal-to-Interference-plus-Noise Ratio (SINR) in large scale scenarios. We also give insights on the use cases in which using a simplified model can still yield valid results.
Improving Transformer-based Speech Recognition Using Unsupervised Pre-training
Venue: arXiv cs.CL 2019
Year: 2019

Speech recognition technologies are gaining enormous popularity in various industrial applications. However, building a good speech recognition system usually requires large amounts of transcribed data, which is expensive to collect. To tackle this problem, an unsupervised pre-training method called Masked Predictive Coding is proposed, which can be applied for unsupervised pre-training with Transformer based model. Experiments on HKUST show that using the same training data, we can achieve CER 23.3%, exceeding the best end-to-end model by over 0.2% absolute CER. With more pre-training data, we can further reduce the CER to 21.0%, or a 11.8% relative CER reduction over baseline.
Abnormal Client Behavior Detection in Federated Learning
Venue: arXiv cs.LG 2019
Year: 2019

In federated learning systems, clients are autonomous in that their behaviors are not fully governed by the server. Consequently, a client may intentionally or unintentionally deviate from the prescribed course of federated model training, resulting in abnormal behaviors, such as turning into a malicious attacker or a malfunctioning client. Timely detecting those anomalous clients is therefore critical to minimize their adverse impacts. In this work, we propose to detect anomalous clients at the server side. In particular, we generate low-dimensional surrogates of model weight vectors and use them to perform anomaly detection. We evaluate our solution through experiments on image classification model training over the FEMNIST dataset. Experimental results show that the proposed detection-based approach significantly outperforms the conventional defense-based methods.