OpenRaiser/Intern-Atlas huggingface dataset:
On Correlation of Features Extracted by Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Redundancy in deep neural network (DNN) models has always been one of their most intriguing and important properties. DNNs have been shown to overparameterize, or extract a lot of redundant features. In this work, we explore the impact of size (both width and depth), activation function, and weight initialization on the susceptibility of deep neural network models to extract redundant features. To estimate the number of redundant features in each layer, all the features of a given layer are hierarchically clustered according to their relative cosine distances in feature space and a set threshold. It is shown that both network size and activation function are the two most important components that foster the tendency of DNNs to extract redundant features. The concept is illustrated using deep multilayer perceptron and convolutional neural networks on MNIST digits recognition and CIFAR-10 dataset, respectively.
Securing Fog-to-Things Environment Using Intrusion Detection System Based On Ensemble Learning
Venue: arXiv cs.CR 2019
Year: 2019

The growing interest in the Internet of Things (IoT) applications is associated with an augmented volume of security threats. In this vein, the Intrusion detection systems (IDS) have emerged as a viable solution for the detection and prevention of malicious activities. Unlike the signature-based detection approaches, machine learning-based solutions are a promising means for detecting unknown attacks. However, the machine learning models need to be accurate enough to reduce the number of false alarms. More importantly, they need to be trained and evaluated on realistic datasets such that their efficacy can be validated on real-time deployments. Many solutions proposed in the literature are reported to have high accuracy but are ineffective in real applications due to the non-representativity of the dataset used for training and evaluation of the underlying models. On the other hand, some of the existing solutions overcome these challenges but yield low accuracy which hampers their implementation for commercial tools. These solutions are majorly based on single learners and are therefore directly affected by the intrinsic limitations of each learning algorithm. The novelty of this paper is to use the most realistic dataset available for intrusion detection called NSL-KDD, and combine multiple learners to build ensemble learners that increase the accuracy of the detection. Furthermore, a deployment architecture in a fog-to-things environment that employs two levels of classifications is proposed. In such architecture, the first level performs an anomaly detection which reduces the latency of the classification substantially, while the second level, executes attack classifications, enabling precise prevention measures. Finally, the experimental results demonstrate the effectiveness of the proposed IDS in comparison with the other state-of-the-arts on the NSL-KDD dataset.
New Results about the Boomerang Uniformity of Permutation Polynomials
Venue: arXiv cs.CR 2019
Year: 2019

In EUROCRYPT 2018, Cid et al. \cite{BCT2018} introduced a new concept on the cryptographic property of S-boxes: Boomerang Connectivity Table (BCT for short) for evaluating the subtleties of boomerang-style attacks. Very recently, BCT and the boomerang uniformity, the maximum value in BCT, were further studied by Boura and Canteaut \cite{BC2018}. Aiming at providing new insights, we show some new results about BCT and the boomerang uniformity of permutations in terms of theory and experiment in this paper. Firstly, we present an equivalent technique to compute BCT and the boomerang uniformity, which seems to be much simpler than the original definition from \cite{BCT2018}. Secondly, thanks to Carlet's idea \cite{Carlet2018}, we give a characterization of functions $f$ from $\mathbb{F}_{2}^n$ to itself with boomerang uniformity $δ_{f}$ by means of the Walsh transform. Thirdly, by our method, we consider boomerang uniformities of some specific permutations, mainly the ones with low differential uniformity. Finally, we obtain another class of $4$-uniform BCT permutation polynomials over $\mathbb{F}_{2^n}$, which is the first binomial.
A Convolutional Neural Network for the Automatic Diagnosis of Collagen VI related Muscular Dystrophies
Venue: arXiv cs.CV 2019
Year: 2019

The development of machine learning systems for the diagnosis of rare diseases is challenging mainly due the lack of data to study them. Despite this challenge, this paper proposes a system for the Computer Aided Diagnosis (CAD) of low-prevalence, congenital muscular dystrophies from confocal microscopy images. The proposed CAD system relies on a Convolutional Neural Network (CNN) which performs an independent classification for non-overlapping patches tiling the input image, and generates an overall decision summarizing the individual decisions for the patches on the query image. This decision scheme points to the possibly problematic areas in the input images and provides a global quantitative evaluation of the state of the patients, which is fundamental for diagnosis and to monitor the efficiency of therapies.
Resolution enhancement in scanning electron microscopy using deep learning
Venue: arXiv cs.CV 2019
Year: 2019

We report resolution enhancement in scanning electron microscopy (SEM) images using a generative adversarial network. We demonstrate the veracity of this deep learning-based super-resolution technique by inferring unresolved features in low-resolution SEM images and comparing them with the accurately co-registered high-resolution SEM images of the same samples. Through spatial frequency analysis, we also report that our method generates images with frequency spectra matching higher resolution SEM images of the same fields-of-view. By using this technique, higher resolution SEM images can be taken faster, while also reducing both electron charging and damage to the samples.
Which Surrogate Works for Empirical Performance Modelling? A Case Study with Differential Evolution
Venue: arXiv cs.NE 2019
Year: 2019

It is not uncommon that meta-heuristic algorithms contain some intrinsic parameters, the optimal configuration of which is crucial for achieving their peak performance. However, evaluating the effectiveness of a configuration is expensive, as it involves many costly runs of the target algorithm. Perhaps surprisingly, it is possible to build a cheap-to-evaluate surrogate that models the algorithm's empirical performance as a function of its parameters. Such surrogates constitute an important building block for understanding algorithm performance, algorithm portfolio/selection, and the automatic algorithm configuration. In principle, many off-the-shelf machine learning techniques can be used to build surrogates. In this paper, we take the differential evolution (DE) as the baseline algorithm for proof-of-concept study. Regression models are trained to model the DE's empirical performance given a parameter configuration. In particular, we evaluate and compare four popular regression algorithms both in terms of how well they predict the empirical performance with respect to a particular parameter configuration, and also how well they approximate the parameter versus the empirical performance landscapes.
Generic Connectivity-Based CGRA Mapping via Integer Linear Programming
Venue: arXiv cs.AR 2019
Year: 2019

Coarse-grained reconfigurable architectures (CGRAs) are programmable logic devices with large coarse-grained ALU-like logic blocks, and multi-bit datapath-style routing. CGRAs often have relatively restricted data routing networks, so they attract CAD mapping tools that use exact methods, such as Integer Linear Programming (ILP). However, tools that target general architectures must use large constraint systems to fully describe an architecture's flexibility, resulting in lengthy run-times. In this paper, we propose to derive connectivity information from an otherwise generic device model, and use this to create simpler ILPs, which we combine in an iterative schedule and retain most of the exactness of a fully-generic ILP approach. This new approach has a speed-up geometric mean of 5.88x when considering benchmarks that do not hit a time-limit of 7.5 hours on the fully-generic ILP, and 37.6x otherwise. This was measured using the set of benchmarks used to originally evaluate the fully-generic approach and several more benchmarks representing computation tasks, over three different CGRA architectures. All run-times of the new approach are less than 20 minutes, with 90th percentile time of 410 seconds. The proposed mapping techniques are integrated into, and evaluated using the open-source CGRA-ME architecture modelling and exploration framework.
A Bad Arm Existence Checking Problem
Venue: arXiv cs.LG 2019
Year: 2019

We study a bad arm existing checking problem in which a player's task is to judge whether a positive arm exists or not among given K arms by drawing as small number of arms as possible. Here, an arm is positive if its expected loss suffered by drawing the arm is at least a given threshold. This problem is a formalization of diagnosis of disease or machine failure. An interesting structure of this problem is the asymmetry of positive and negative (non-positive) arms' roles; finding one positive arm is enough to judge existence while all the arms must be discriminated as negative to judge non-existence. We propose an algorithms with arm selection policy (policy to determine the next arm to draw) and stopping condition (condition to stop drawing arms) utilizing this asymmetric problem structure and prove its effectiveness theoretically and empirically.
Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search
Venue: arXiv cs.DC 2019
Year: 2019

A fundamental question lies in almost every application of deep neural networks: what is the optimal neural architecture given a specific dataset? Recently, several Neural Architecture Search (NAS) frameworks have been developed that use reinforcement learning and evolutionary algorithm to search for the solution. However, most of them take a long time to find the optimal architecture due to the huge search space and the lengthy training process needed to evaluate each candidate. In addition, most of them aim at accuracy only and do not take into consideration the hardware that will be used to implement the architecture. This will potentially lead to excessive latencies beyond specifications, rendering the resulting architectures useless. To address both issues, in this paper we use Field Programmable Gate Arrays (FPGAs) as a vehicle to present a novel hardware-aware NAS framework, namely FNAS, which will provide an optimal neural architecture with latency guaranteed to meet the specification. In addition, with a performance abstraction model to analyze the latency of neural architectures without training, our framework can quickly prune architectures that do not satisfy the specification, leading to higher efficiency. Experimental results on common data set such as ImageNet show that in the cases where the state-of-the-art generates architectures with latencies 7.81x longer than the specification, those from FNAS can meet the specs with less than 1% accuracy loss. Moreover, FNAS also achieves up to 11.13x speedup for the search process. To the best of the authors' knowledge, this is the very first hardware aware NAS.
Multistage Knapsack
Venue: arXiv cs.DS 2019
Year: 2019

Many systems have to be maintained while the underlying constraints, costs and/or profits change over time. Although the state of a system may evolve during time, a non-negligible transition cost is incured for transitioning from one state to another. In order to model such situations, Gupta et al. (ICALP 2014) and Eisenstat et al. (ICALP 2014) introduced a multistage model where the input is a sequence of instances (one for each time step), and the goal is to find a sequence of solutions (one for each time step) that are both (i) near optimal for each time step and (ii) as stable as possible. We focus on the multistage version of the Knapsack problem where we are given a time horizon t=1,2,...,T, and a sequence of knapsack instances I_1,I_2,...,I_T, one for each time step, defined on a set of n objects. In every time step t we have to choose a feasible knapsack S_t of I_t, which gives a knapsack profit. To measure the stability/similarity of two consecutive solutions S_t and S_{t+1}, we identify the objects for which the decision, to be picked or not, remains the same in S_t and S_{t+1}, giving a transition profit. We are asked to produce a sequence of solutions S_1,S_2,...,S_T so that the total knapsack profit plus the overall transition profit is maximized. We propose a PTAS for the Multistage Knapsack problem. Then, we prove that there is no FPTAS for the problem even in the case where T=2, unless P=NP. Furthermore, we give a pseudopolynomial time algorithm for the case where the number of steps is bounded by a fixed constant and we show that otherwise the problem remains NP-hard even in the case where all the weights, profits and capacities are 0 or 1.
Conversational Networks for Automatic Online Moderation
Venue: arXiv cs.IR 2019
Year: 2019

Moderation of user-generated content in an online community is a challenge that has great socio-economical ramifications. However, the costs incurred by delegating this work to human agents are high. For this reason, an automatic system able to detect abuse in user-generated content is of great interest. There are a number of ways to tackle this problem, but the most commonly seen in practice are word filtering or regular expression matching. The main limitations are their vulnerability to intentional obfuscation on the part of the users, and their context-insensitive nature. Moreover, they are language-dependent and may require appropriate corpora for training. In this paper, we propose a system for automatic abuse detection that completely disregards message content. We first extract a conversational network from raw chat logs and characterize it through topological measures. We then use these as features to train a classifier on our abuse detection task. We thoroughly assess our system on a dataset of user comments originating from a French Massively Multiplayer Online Game. We identify the most appropriate network extraction parameters and discuss the discriminative power of our features, relatively to their topological and temporal nature. Our method reaches an F-measure of 83.89 when using the full feature set, improving on existing approaches. With a selection of the most discriminative features, we dramatically cut computing time while retaining most of the performance (82.65).
Hyperbox based machine learning algorithms: A comprehensive survey
Venue: arXiv cs.LG 2019
Year: 2019

With the rapid development of digital information, the data volume generated by humans and machines is growing exponentially. Along with this trend, machine learning algorithms have been formed and evolved continuously to discover new information and knowledge from different data sources. Learning algorithms using hyperboxes as fundamental representational and building blocks are a branch of machine learning methods. These algorithms have enormous potential for high scalability and online adaptation of predictors built using hyperbox data representations to the dynamically changing environments and streaming data. This paper aims to give a comprehensive survey of literature on hyperbox-based machine learning models. In general, according to the architecture and characteristic features of the resulting models, the existing hyperbox-based learning algorithms may be grouped into three major categories: fuzzy min-max neural networks, hyperbox-based hybrid models, and other algorithms based on hyperbox representations. Within each of these groups, this paper shows a brief description of the structure of models, associated learning algorithms, and an analysis of their advantages and drawbacks. Main applications of these hyperbox-based models to the real-world problems are also described in this paper. Finally, we discuss some open problems and identify potential future research directions in this field.
A Tractable Framework for Coverage Analysis of Cellular-Connected UAV Networks
Venue: arXiv cs.IT 2019
Year: 2019

Unmanned aerial vehicles (UAVs) have recently found abundant applications in the public and civil domains. To ensure reliable control and navigation, connecting UAVs to controllers via existing cellular network infrastructure, i.e., ground base stations (GBSs), has been proposed as a promising solution. Nevertheless, it is highly challenging to characterize the communication performance of cellular-connected UAVs, due to their unique propagation conditions. This paper proposes a tractable framework for the coverage analysis of cellular-connected UAV networks, which consists of a new blockage model and an effective approach to handle general fading channels. In particular, a line-of-sight (LoS) ball model is proposed to capture the probabilistic propagation in UAV communication systems, and a tractable expression is derived for the Laplace transform of the aggregate interference with general Nakagami fading. This framework leads to a tractable expression for the coverage probability, which in turn helps to investigate the impact of the GBS density. Specifically, a tight lower bound on the optimal density that maximizes the coverage probability is derived. Numerical results show that the proposed LoS ball model is accurate, and the optimal GBS density decreases when the UAV altitude increases.
Learning of High Dengue Incidence with Clustering and FP-Growth Algorithm using WHO Historical Data
Venue: arXiv cs.DB 2019
Year: 2019

This paper applies FP-Growth algorithm in mining fuzzy association rules for a prediction system of dengue. The system mines its rules through input of historic predictor variables for dengue. The rules will be used to build a rule-based classifier to predict the dengue incidence for the next month for the years 2001-2006 in the Philippines. The FP-Growth Algorithm was compared to Apriori Algorithm by Sensitivity, Specificity, PPV, NPV, execution time and memory usage. The results showed that FP-Growth Algorithm is significantly better in execution time, numerically better in memory and comparable in Sensitivity, Specificity, PPV and NPV to Apriori Algorithm.
MONet: Unsupervised Scene Decomposition and Representation
Venue: arXiv cs.CV 2019
Year: 2019

The ability to decompose scenes in terms of abstract building blocks is crucial for general intelligence. Where those basic building blocks share meaningful properties, interactions and other regularities across scenes, such decompositions can simplify reasoning and facilitate imagination of novel scenarios. In particular, representing perceptual observations in terms of entities should improve data efficiency and transfer performance on a wide range of tasks. Thus we need models capable of discovering useful decompositions of scenes by identifying units with such regularities and representing them in a common format. To address this problem, we have developed the Multi-Object Network (MONet). In this model, a VAE is trained end-to-end together with a recurrent attention network -- in a purely unsupervised manner -- to provide attention masks around, and reconstructions of, regions of images. We show that this model is capable of learning to decompose and represent challenging 3D scenes into semantically meaningful components, such as objects and background elements.
Partition Pruning: Parallelization-Aware Pruning for Deep Neural Networks
Venue: arXiv cs.CV 2019
Year: 2019

Parameters of recent neural networks require a huge amount of memory. These parameters are used by neural networks to perform machine learning tasks when processing inputs. To speed up inference, we develop Partition Pruning, an innovative scheme to reduce the parameters used while taking into consideration parallelization. We evaluated the performance and energy consumption of parallel inference of partitioned models, which showed a 7.72x speed up of performance and a 2.73x reduction in the energy used for computing pruned layers of TinyVGG16 in comparison to running the unpruned model on a single accelerator. In addition, our method showed a limited reduction some numbers in accuracy while partitioning fully connected layers.
Effect of Various Regularizers on Model Complexities of Neural Networks in Presence of Input Noise
Venue: arXiv cs.LG 2019
Year: 2019

Deep neural networks are over-parameterized, which implies that the number of parameters are much larger than the number of samples used to train the network. Even in such a regime deep architectures do not overfit. This phenomenon is an active area of research and many theories have been proposed trying to understand this peculiar observation. These include the Vapnik Chervonenkis (VC) dimension bounds and Rademacher complexity bounds which show that the capacity of the network is characterized by the norm of weights rather than the number of parameters. However, the effect of input noise on these measures for shallow and deep architectures has not been studied. In this paper, we analyze the effects of various regularization schemes on the complexity of a neural network which we characterize with the loss, $L_2$ norm of the weights, Rademacher complexities (Directly Approximately Regularizing Complexity-DARC1), VC dimension based Low Complexity Neural Network (LCNN) when subject to varying degrees of Gaussian input noise. We show that $L_2$ regularization leads to a simpler hypothesis class and better generalization followed by DARC1 regularizer, both for shallow as well as deeper architectures. Jacobian regularizer works well for shallow architectures with high level of input noises. Spectral normalization attains highest test set accuracies both for shallow and deeper architectures. We also show that Dropout alone does not perform well in presence of input noise. Finally, we show that deeper architectures are robust to input noise as opposed to their shallow counterparts.
Efficient and secure modular operations using the Adapted Modular Number System
Venue: arXiv cs.CR 2019
Year: 2019

The Adapted Modular Number System (AMNS) is a sytem of representation of integers to speed up arithmetic operations modulo a prime p. Such a system can be defined by a tuple (p, n, γ, ρ, E) where E is in Z[X]. In [13] conditions are given to build AMNS with E(X) = {X^n +1}. In this paper, we generalize their results and show how to generate multiple AMNS for a given prime p with E(X)={X^n-λ} and λ in Z. Moreover, we propose a complete set of algorithms without conditional branching to perform arithmetic and conversion operations in the AMNS, using a Montgomery-like method described in [15].
Inducing Sparse Coding and And-Or Grammar from Generator Network
Venue: arXiv cs.LG 2019
Year: 2019

We introduce an explainable generative model by applying sparse operation on the feature maps of the generator network. Meaningful hierarchical representations are obtained using the proposed generative model with sparse activations. The convolutional kernels from the bottom layer to the top layer of the generator network can learn primitives such as edges and colors, object parts, and whole objects layer by layer. From the perspective of the generator network, we propose a method for inducing both sparse coding and the AND-OR grammar for images. Experiments show that our method is capable of learning meaningful and explainable hierarchical representations.