OpenRaiser/Intern-Atlas huggingface dataset:
DeepZip: Lossless Data Compression using Recurrent Neural Networks
Venue: arXiv cs.CL 2018
Year: 2018

Sequential data is being generated at an unprecedented pace in various forms, including text and genomic data. This creates the need for efficient compression mechanisms to enable better storage, transmission and processing of such data. To solve this problem, many of the existing compressors attempt to learn models for the data and perform prediction-based compression. Since neural networks are known as universal function approximators with the capability to learn arbitrarily complex mappings, and in practice show excellent performance in prediction tasks, we explore and devise methods to compress sequential data using neural network predictors. We combine recurrent neural network predictors with an arithmetic coder and losslessly compress a variety of synthetic, text and genomic datasets. The proposed compressor outperforms Gzip on the real datasets and achieves near-optimal compression for the synthetic datasets. The results also help understand why and where neural networks are good alternatives for traditional finite context models
Orthographic Feature Transform for Monocular 3D Object Detection
Venue: arXiv cs.CV 2018
Year: 2018

3D object detection from monocular images has proven to be an enormously challenging task, with the performance of leading systems not yet achieving even 10\% of that of LiDAR-based counterparts. One explanation for this performance gap is that existing systems are entirely at the mercy of the perspective image-based representation, in which the appearance and scale of objects varies drastically with depth and meaningful distances are difficult to infer. In this work we argue that the ability to reason about the world in 3D is an essential element of the 3D object detection task. To this end, we introduce the orthographic feature transform, which enables us to escape the image domain by mapping image-based features into an orthographic 3D space. This allows us to reason holistically about the spatial configuration of the scene in a domain where scale is consistent and distances between objects are meaningful. We apply this transformation as part of an end-to-end deep learning architecture and achieve state-of-the-art performance on the KITTI 3D object benchmark.\footnote{We will release full source code and pretrained models upon acceptance of this manuscript for publication.
Effect of correlation on the traffic capacity of Time Varying Communication Network
Venue: arXiv cs.NI 2018
Year: 2018

The network topology and the routing strategy are major factors to affect the traffic dynamics of the network. In this work, we aim to design an optimal time-varying network structure and an efficient route is allocated to each user in the network. The network topology is designed by considering addition, removal, and rewiring of links. At each time instants, a new node connects with an existing node based on the degree and correlation with its neighbor. Traffic congestion is handled by rewiring of some congested links along with the removal of the anti-preferential and correlated links. Centrality plays an important role to find the most important node in the network. The more a node is central, the more it can be used for the shortest route of the user pairs and it can be congested due to a large number of data coming from its neighborhood. Therefore, routes of the users are selected such that the sum of the centrality of the nodes appearing in the user's route is minimum. Thereafter, we analyze the network structure by using various network properties such as the clustering coefficient, centrality, average shortest path, rich club coefficient, average packet travel time and order parameter.
Attentive Neural Architecture Incorporating Song Features For Music Recommendation
Venue: arXiv cs.IR 2018
Year: 2018

Recommender Systems are an integral part of music sharing platforms. Often the aim of these systems is to increase the time, the user spends on the platform and hence having a high commercial value. The systems which aim at increasing the average time a user spends on the platform often need to recommend songs which the user might want to listen to next at each point in time. This is different from recommendation systems which try to predict the item which might be of interest to the user at some point in the user lifetime but not necessarily in the very near future. Prediction of the next song the user might like requires some kind of modeling of the user interests at the given point of time. Attentive neural networks have been exploiting the sequence in which the items were selected by the user to model the implicit short-term interests of the user for the task of next item prediction, however we feel that the features of the songs occurring in the sequence could also convey some important information about the short-term user interest which only the items cannot. In this direction, we propose a novel attentive neural architecture which in addition to the sequence of items selected by the user, uses the features of these items to better learn the user short-term preferences and recommend the next song to the user.
Brain-Inspired Stigmergy Learning
Venue: arXiv cs.NE 2018
Year: 2018

Stigmergy has proved its great superiority in terms of distributed control, robustness and adaptability, thus being regarded as an ideal solution for large-scale swarm control problems. Based on new discoveries on astrocytes in regulating synaptic transmission in the brain, this paper has mapped stigmergy mechanism into the interaction between synapses and investigated its characteristics and advantages. Particularly, we have divided the interaction between synapses which are not directly connected into three phases and proposed a stigmergic learning model. In this model, the state change of a stigmergy agent will expand its influence to affect the states of others. The strength of the interaction is determined by the level of neural activity as well as the distance between stigmergy agents. Inspired by the morphological and functional changes in astrocytes during environmental enrichment, it is likely that the regulation of distance between stigmergy agents plays a critical role in the stigmergy learning process. Simulation results have verified its importance and indicated that the well-regulated distance between stigmergy agents can help to obtain stigmergy learning gain.
Contingency Training
Venue: arXiv cs.LG 2018
Year: 2018

When applied to high-dimensional datasets, feature selection algorithms might still leave dozens of irrelevant variables in the dataset. Therefore, even after feature selection has been applied, classifiers must be prepared to the presence of irrelevant variables. This paper investigates a new training method called Contingency Training which increases the accuracy as well as the robustness against irrelevant attributes. Contingency training is classifier independent. By subsampling and removing information from each sample, it creates a set of constraints. These constraints aid the method to automatically find proper importance weights of the dataset's features. Experiments are conducted with the contingency training applied to neural networks over traditional datasets as well as datasets with additional irrelevant variables. For all of the tests, contingency training surpassed the unmodified training on datasets with irrelevant variables and even outperformed slightly when only a few or no irrelevant variables were present.
Geometry of Friston's active inference
Venue: arXiv cs.AI 2018
Year: 2018

We reconstruct Karl Friston's active inference and give a geometrical interpretation of it.
Interplay of Game Incentives, Player Profiles and Task Difficulty in Games with a Purpose
Venue: arXiv cs.HC 2018
Year: 2018

How to take multiple factors into account when evaluating a Game with a Purpose? How is player behaviour or participation influenced by different incentives? How does player engagement impact their accuracy in solving tasks? In this paper, we present a detailed investigation of multiple factors affecting the evaluation of a GWAP and we show how they impact on the achieved results. We inform our study with the experimental assessment of a GWAP designed to solve a multinomial classification task.
Stability Based Filter Pruning for Accelerating Deep CNNs
Venue: arXiv cs.CV 2018
Year: 2018

Convolutional neural networks (CNN) have achieved impressive performance on the wide variety of tasks (classification, detection, etc.) across multiple domains at the cost of high computational and memory requirements. Thus, leveraging CNNs for real-time applications necessitates model compression approaches that not only reduce the total number of parameters but reduce the overall computation as well. In this work, we present a stability-based approach for filter-level pruning of CNNs. We evaluate our proposed approach on different architectures (LeNet, VGG-16, ResNet, and Faster RCNN) and datasets and demonstrate its generalizability through extensive experiments. Moreover, our compressed models can be used at run-time without requiring any special libraries or hardware. Our model compression method reduces the number of FLOPS by an impressive factor of 6.03X and GPU memory footprint by more than 17X, significantly outperforming other state-of-the-art filter pruning methods.
Multi-layer Pruning Framework for Compressing Single Shot MultiBox Detector
Venue: arXiv cs.CV 2018
Year: 2018

We propose a framework for compressing state-of-the-art Single Shot MultiBox Detector (SSD). The framework addresses compression in the following stages: Sparsity Induction, Filter Selection, and Filter Pruning. In the Sparsity Induction stage, the object detector model is sparsified via an improved global threshold. In Filter Selection & Pruning stage, we select and remove filters using sparsity statistics of filter weights in two consecutive convolutional layers. This results in the model with the size smaller than most existing compact architectures. We evaluate the performance of our framework with multiple datasets and compare over multiple methods. Experimental results show that our method achieves state-of-the-art compression of 6.7X and 4.9X on PASCAL VOC dataset on models SSD300 and SSD512 respectively. We further show that the method produces maximum compression of 26X with SSD512 on German Traffic Sign Detection Benchmark (GTSDB). Additionally, we also empirically show our method's adaptability for classification based architecture VGG16 on datasets CIFAR and German Traffic Sign Recognition Benchmark (GTSRB) achieving a compression rate of 125X and 200X with the reduction in flops by 90.50% and 96.6% respectively with no loss of accuracy. In addition to this, our method does not require any special libraries or hardware support for the resulting compressed models.
Structured Pruning for Efficient ConvNets via Incremental Regularization
Venue: arXiv cs.CV 2018
Year: 2018

Parameter pruning is a promising approach for CNN compression and acceleration by eliminating redundant model parameters with tolerable performance loss. Despite its effectiveness, existing regularization-based parameter pruning methods usually drive weights towards zero with large and constant regularization factors, which neglects the fact that the expressiveness of CNNs is fragile and needs a more gentle way of regularization for the networks to adapt during pruning. To solve this problem, we propose a new regularization-based pruning method (named IncReg) to incrementally assign different regularization factors to different weight groups based on their relative importance, whose effectiveness is proved on popular CNNs compared with state-of-the-art methods.
Intermediate Level Adversarial Attack for Enhanced Transferability
Venue: arXiv cs.LG 2018
Year: 2018

Neural networks are vulnerable to adversarial examples, malicious inputs crafted to fool trained models. Adversarial examples often exhibit black-box transfer, meaning that adversarial examples for one model can fool another model. However, adversarial examples may be overfit to exploit the particular architecture and feature representation of a source model, resulting in sub-optimal black-box transfer attacks to other target models. This leads us to introduce the Intermediate Level Attack (ILA), which attempts to fine-tune an existing adversarial example for greater black-box transferability by increasing its perturbation on a pre-specified layer of the source model. We show that our method can effectively achieve this goal and that we can decide a nearly-optimal layer of the source model to perturb without any knowledge of the target models.
Tight Approximation Ratio for Minimum Maximal Matching
Venue: arXiv cs.CC 2018
Year: 2018

We study a combinatorial problem called Minimum Maximal Matching, where we are asked to find in a general graph the smallest that can not be extended. We show that this problem is hard to approximate with a constant smaller than 2, assuming the Unique Games Conjecture. As a corollary we show, that Minimum Maximal Matching in bipartite graphs is hard to approximate with constant smaller than $\frac{4}{3}$, with the same assumption. With a stronger variant of the Unique Games Conjecture --- that is Small Set Expansion Hypothesis --- we are able to improve the hardness result up to the factor of $\frac{3}{2}$.
How the Softmax Output is Misleading for Evaluating the Strength of Adversarial Examples
Venue: arXiv cs.LG 2018
Year: 2018

Even before deep learning architectures became the de facto models for complex computer vision tasks, the softmax function was, given its elegant properties, already used to analyze the predictions of feedforward neural networks. Nowadays, the output of the softmax function is also commonly used to assess the strength of adversarial examples: malicious data points designed to fail machine learning models during the testing phase. However, in this paper, we show that it is possible to generate adversarial examples that take advantage of some properties of the softmax function, leading to undesired outcomes when interpreting the strength of the adversarial examples at hand. Specifically, we argue that the output of the softmax function is a poor indicator when the strength of an adversarial example is analyzed and that this indicator can be easily tricked by already existing methods for adversarial example generation.
Tetris
Venue: arXiv cs.MA 2018
Year: 2018

Tetris is an Asynchronous Byzantine Fault Tolerance consensus algorithm designed for next generation high-throughput permission and permissionless blockchain. The core concept of Tetris is derived from Reasoning About Knowledge, which we believe to be the most appropriate tools for revealing and analyzing the fundamental complexity of distributed systems. By analyzing the states of knowledge that each participant attained in an unreliable system, we can capture some of the basis underlying structure of the system, then help us designing effective & efficient protocols. Plus the adoption of Full Information Protocol (FIP) with the optimized message traffic model, Tetris has finally got high performance, with proved safety. Tetris achieve consensus finality in seconds, means transactions can be confirmed greatly faster than other scheme like Pow/Dpos. Tetris also achieve fairness, which is critically important in some areas such as stock market etc.
Passivity-Based Generalization of Primal-Dual Dynamics for Non-Strictly Convex Cost Functions
Venue: arXiv cs.SY 2018
Year: 2018

In this paper, we revisit primal-dual dynamics for convex optimization and present a generalization of the dynamics based on the concept of passivity. It is then proved that supplying a stable zero to one of the integrators in the dynamics allows one to eliminate the assumption of strict convexity on the cost function based on the passivity paradigm together with the invariance principle for Caratheodory systems. We then show that the present algorithm is also a generalization of existing augmented Lagrangian-based primal-dual dynamics, and discuss the benefit of the present generalization in terms of noise reduction and convergence speed.
Malicious Web Request Detection Using Character-level CNN
Venue: arXiv cs.CR 2018
Year: 2018

Web parameter injection attacks are common and powerful. In this kind of attacks, malicious attackers can employ HTTP requests to implement attacks against servers by injecting some malicious codes into the parameters of the HTTP requests. Against the web parameter injection attacks, most of the existing Web Intrusion Detection Systems (WIDS) cannot find unknown new attacks and have a high false positive rate (FPR), since they lack the ability of re-learning and rarely pay attention to the intrinsic relationship between the characters. In this paper, we propose a malicious requests detection system with re-learning ability based on an improved convolution neural network (CNN) model. We add a character-level embedding layer before the convolution layer, which makes our model able to learn the intrinsic relationship between the characters of the query string. Further, we modify the filters of CNN and the modified filters can extract the fine-grained features of the query string. The test results demonstrate that our model has lower FPR compared with support vector machine (SVM) and random forest (RF).
AttentionMask: Attentive, Efficient Object Proposal Generation Focusing on Small Objects
Venue: arXiv cs.CV 2018
Year: 2018

We propose a novel approach for class-agnostic object proposal generation, which is efficient and especially well-suited to detect small objects. Efficiency is achieved by scale-specific objectness attention maps which focus the processing on promising parts of the image and reduce the amount of sampled windows strongly. This leads to a system, which is $33\%$ faster than the state-of-the-art and clearly outperforming state-of-the-art in terms of average recall. Secondly, we add a module for detecting small objects, which are often missed by recent models. We show that this module improves the average recall for small objects by about $53\%$.
Beyond the Desktop: Emerging Technologies for Supporting 3D Collaborative Teams
Venue: arXiv cs.HC 2018
Year: 2018

The emergence of several trends, including the increased availability of wireless networks, miniaturization of electronics and sensing technologies, and novel input and output devices, is creating a demand for integrated, full-time displays for use across a wide range of applications, including collaborative environments. In this paper, we present and discuss emerging visualization methods we are developing particularly as they relate to deployable displays and displays worn on the body to support mobile users.
Multi-View Inpainting for RGB-D Sequence
Venue: arXiv cs.CV 2018
Year: 2018

In this work we propose a novel approach to remove undesired objects from RGB-D sequences captured with freely moving cameras, which enables static 3D reconstruction. Our method jointly uses existing information from multiple frames as well as generates new one via inpainting techniques. We use balanced rules to select source frames; local homography based image warping method for alignment and Markov random field (MRF) based approach for combining existing information. For the left holes, we employ exemplar based multi-view inpainting method to deal with the color image and coherently use it as guidance to complete the depth correspondence. Experiments show that our approach is qualified for removing the undesired objects and inpainting the holes.
Super Diffusion for Salient Object Detection
Venue: arXiv cs.CV 2018
Year: 2018

One major branch of saliency object detection methods is diffusion-based which construct a graph model on a given image and diffuse seed saliency values to the whole graph by a diffusion matrix. While their performance is sensitive to specific feature spaces and scales used for the diffusion matrix definition, little work has been published to systematically promote the robustness and accuracy of salient object detection under the generic mechanism of diffusion. In this work, we firstly present a novel view of the working mechanism of the diffusion process based on mathematical analysis, which reveals that the diffusion process is actually computing the similarity of nodes with respect to the seeds based on diffusion maps. Following this analysis, we propose super diffusion, a novel inclusive learning-based framework for salient object detection, which makes the optimum and robust performance by integrating a large pool of feature spaces, scales and even features originally computed for non-diffusion-based salient object detection. A closed-form solution of the optimal parameters for the integration is determined through supervised learning. At the local level, we propose to promote each individual diffusion before the integration. Our mathematical analysis reveals the close relationship between saliency diffusion and spectral clustering. Based on this, we propose to re-synthesize each individual diffusion matrix from the most discriminative eigenvectors and the constant eigenvector (for saliency normalization). The proposed framework is implemented and experimented on prevalently used benchmark datasets, consistently leading to state-of-the-art performance.
Conditioning Optimization of Extreme Learning Machine by Multitask Beetle Antennae Swarm Algorithm
Venue: arXiv cs.NE 2018
Year: 2018

Extreme learning machine (ELM) as a simple and rapid neural network has been shown its good performance in various areas. Different from the general single hidden layer feedforward neural network (SLFN), the input weights and biases in hidden layer of ELM are generated randomly, so that it only takes a little computation overhead to train the model. However, the strategy of selecting input weights and biases at random may result in ill-posed problem. Aiming to optimize the conditioning of ELM, we propose an effective particle swarm heuristic algorithm called Multitask Beetle Antennae Swarm Algorithm (MBAS), which is inspired by the structures of artificial bee colony (ABS) algorithm and Beetle Antennae Search (BAS) algorithm. Then, the proposed MBAS is applied to optimize the input weights and biases of ELM. Experiment results show that the proposed method is capable of simultaneously reducing the condition number and regression error, and achieving good generalization performances.
Using External Archive for Improved Performance in Multi-Objective Optimization
Venue: arXiv cs.NE 2018
Year: 2018

It is shown that the use of an external archive, purely for storage purposes, can bring substantial benefits in multi-objective optimization. A new scheme for archive management for the above purpose is described. The new scheme is combined with the NSGA-II algorithm for solving two multi-objective optimization problems, and it is demonstrated that this combination gives significantly improved sets of Pareto-optimal solutions. The additional computational effort because of the external archive is found to be insignificant when the objective functions are expensive to evaluate.