OpenRaiser/Intern-Atlas huggingface dataset:
Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator
Venue: arXiv cs.AR 2018
Year: 2018

Neural networks are an increasingly attractive algorithm for natural language processing and pattern recognition. Deep networks with >50M parameters are made possible by modern GPU clusters operating at <50 pJ per op and more recently, production accelerators capable of <5pJ per operation at the board level. However, with the slowing of CMOS scaling, new paradigms will be required to achieve the next several orders of magnitude in performance per watt gains. Using an analog resistive memory (ReRAM) crossbar to perform key matrix operations in an accelerator is an attractive option. This work presents a detailed design using a state of the art 14/16 nm PDK for of an analog crossbar circuit block designed to process three key kernels required in training and inference of neural networks. A detailed circuit and device-level analysis of energy, latency, area, and accuracy are given and compared to relevant designs using standard digital ReRAM and SRAM operations. It is shown that the analog accelerator has a 270x energy and 540x latency advantage over a similar block utilizing only digital ReRAM and takes only 11 fJ per multiply and accumulate (MAC). Compared to an SRAM based accelerator, the energy is 430X better and latency is 34X better. Although training accuracy is degraded in the analog accelerator, several options to improve this are presented. The possible gains over a similar digital-only version of this accelerator block suggest that continued optimization of analog resistive memories is valuable. This detailed circuit and device analysis of a training accelerator may serve as a foundation for further architecture-level studies.
PIVO: Probabilistic Inertial-Visual Odometry for Occlusion-Robust Navigation
Venue: arXiv cs.CV 2018
Year: 2018

This paper presents a novel method for visual-inertial odometry. The method is based on an information fusion framework employing low-cost IMU sensors and the monocular camera in a standard smartphone. We formulate a sequential inference scheme, where the IMU drives the dynamical model and the camera frames are used in coupling trailing sequences of augmented poses. The novelty in the model is in taking into account all the cross-terms in the updates, thus propagating the inter-connected uncertainties throughout the model. Stronger coupling between the inertial and visual data sources leads to robustness against occlusion and feature-poor environments. We demonstrate results on data collected with an iPhone and provide comparisons against the Tango device and using the EuRoC data set.
DSOD: Learning Deeply Supervised Object Detectors from Scratch
Venue: arXiv cs.CV 2018
Year: 2018

We present Deeply Supervised Object Detector (DSOD), a framework that can learn object detectors from scratch. State-of-the-art object objectors rely heavily on the off-the-shelf networks pre-trained on large-scale classification datasets like ImageNet, which incurs learning bias due to the difference on both the loss functions and the category distributions between classification and detection tasks. Model fine-tuning for the detection task could alleviate this bias to some extent but not fundamentally. Besides, transferring pre-trained models from classification to detection between discrepant domains is even more difficult (e.g. RGB to depth images). A better solution to tackle these two critical problems is to train object detectors from scratch, which motivates our proposed DSOD. Previous efforts in this direction mostly failed due to much more complicated loss functions and limited training data in object detection. In DSOD, we contribute a set of design principles for training object detectors from scratch. One of the key findings is that deep supervision, enabled by dense layer-wise connections, plays a critical role in learning a good detector. Combining with several other principles, we develop DSOD following the single-shot detection (SSD) framework. Experiments on PASCAL VOC 2007, 2012 and MS COCO datasets demonstrate that DSOD can achieve better results than the state-of-the-art solutions with much more compact models. For instance, DSOD outperforms SSD on all three benchmarks with real-time detection speed, while requires only 1/2 parameters to SSD and 1/10 parameters to Faster RCNN. Our code and models are available at: https://github.com/szq0214/DSOD .
Variance-Reduced Stochastic Learning under Random Reshuffling
Venue: arXiv cs.LG 2018
Year: 2018

Several useful variance-reduced stochastic gradient algorithms, such as SVRG, SAGA, Finito, and SAG, have been proposed to minimize empirical risks with linear convergence properties to the exact minimizer. The existing convergence results assume uniform data sampling with replacement. However, it has been observed in related works that random reshuffling can deliver superior performance over uniform sampling and, yet, no formal proofs or guarantees of exact convergence exist for variance-reduced algorithms under random reshuffling. This paper makes two contributions. First, it resolves this open issue and provides the first theoretical guarantee of linear convergence under random reshuffling for SAGA; the argument is also adaptable to other variance-reduced algorithms. Second, under random reshuffling, the paper proposes a new amortized variance-reduced gradient (AVRG) algorithm with constant storage requirements compared to SAGA and with balanced gradient computations compared to SVRG. AVRG is also shown analytically to converge linearly.
Using Effective Generator Impedance for Forced Oscillation Source Location
Venue: arXiv cs.SY 2018
Year: 2018

Locating the sources of forced low-frequency oscillations in power systems is an important problem. A number of proposed methods demonstrate their practical usefulness, but many of them rely on strong modeling assumptions and provide poor performance in certain cases for reasons still not well understood. This paper proposes a systematic method for locating the source of a forced oscillation by considering a generator's response to fluctuations of its terminal voltages and currents. It is shown that a generator can be represented as an effective admittance matrix with respect to low-frequency oscillations, and an explicit form for this matrix, for various generator models, is derived. Furthermore, it is shown that a source generator, in addition to its effective admittance, is characterized by the presence of an effective current source thus giving a natural qualitative distinction between source and nonsource generators. Detailed descriptions are given of a source detection procedure based on this developed representation, and the method's effectiveness is confirmed by simulations on the recommended testbeds (eg. WECC 179-bus system). This method is free of strong modeling assumptions and is also shown to be robust in the presence of measurement noise and generator parameter uncertainty.
Focal Loss for Dense Object Detection
Venue: arXiv cs.CV 2018
Year: 2018

The highest accuracy object detectors to date are based on a two-stage approach popularized by R-CNN, where a classifier is applied to a sparse set of candidate object locations. In contrast, one-stage detectors that are applied over a regular, dense sampling of possible object locations have the potential to be faster and simpler, but have trailed the accuracy of two-stage detectors thus far. In this paper, we investigate why this is the case. We discover that the extreme foreground-background class imbalance encountered during training of dense detectors is the central cause. We propose to address this class imbalance by reshaping the standard cross entropy loss such that it down-weights the loss assigned to well-classified examples. Our novel Focal Loss focuses training on a sparse set of hard examples and prevents the vast number of easy negatives from overwhelming the detector during training. To evaluate the effectiveness of our loss, we design and train a simple dense detector we call RetinaNet. Our results show that when trained with the focal loss, RetinaNet is able to match the speed of previous one-stage detectors while surpassing the accuracy of all existing state-of-the-art two-stage detectors. Code is at: https://github.com/facebookresearch/Detectron.
Phase Transition in the Maximal Influence Problem: When Do We Need Optimization?
Venue: arXiv cs.SI 2018
Year: 2018

Considerable efforts were made in recent years in devising optimization algorithms for influence maximization in networks. Here we ask: "When do we need optimization?" We use results from statistical mechanics and direct simulations on ER networks, small-world networks, power-law networks and a dataset of real-world networks to characterize the parameter-space region where optimization is required. We show that in both synthetic and real-world networks this optimization region is due to a well known physical phase transition of the network, and that it vanishes as a power-law with the network size. We then show that also from a utility-maximization perspective (when considering the costs of the optimization process), for large networks standard optimization is profitable only in a vanishing parameter region near the phase transition. Finally, we introduce a novel constant-time optimization approach, and demonstrate it through a simple algorithm that manages to give similar results to standard optimization methods in terms of the influenced-set size, while improving the results in terms of the net utility.
Early Evaluation of Intel Optane Non-Volatile Memory with HPC I/O Workloads
Venue: arXiv cs.DC 2018
Year: 2018

High performance computing (HPC) applications have a high requirement on storage speed and capacity. Non-volatile memory is a promising technology to replace traditional storage devices to improve HPC performance. Earlier in 2017, Intel and Micron released first NVM product -- Intel Optane SSDs. Optane is much faster and more durable than the traditional storage device. It creates a bridge to narrow the performance gap between DRAM and storage. But is the existing HPC I/O stack still suitable for new NVM devices like Intel Optane? How does HPC I/O workload perform with Intel Optane? In this paper, we analyze the performance of I/O intensive HPC applications with Optane as a block device and try to answer the above questions. We study the performance from three perspectives: (1) basic read and write bandwidth of Optane, (2) a performance comparison study between Optane and HDD, including checkpoint workload, MPI individual I/O vs. POSIX I/O, and MPI individual I/O vs. MPI collective I/O, and (3) the impact of Optane on the performance of a parallel file system, PVFS2.
Generative Statistical Models with Self-Emergent Grammar of Chord Sequences
Venue: arXiv cs.AI 2018
Year: 2018

Generative statistical models of chord sequences play crucial roles in music processing. To capture syntactic similarities among certain chords (e.g. in C major key, between G and G7 and between F and Dm), we study hidden Markov models and probabilistic context-free grammar models with latent variables describing syntactic categories of chord symbols and their unsupervised learning techniques for inducing the latent grammar from data. Surprisingly, we find that these models often outperform conventional Markov models in predictive power, and the self-emergent categories often correspond to traditional harmonic functions. This implies the need for chord categories in harmony models from the informatics perspective.
Odd Multiway Cut in Directed Acyclic Graphs
Venue: arXiv cs.DS 2018
Year: 2018

We investigate the odd multiway node (edge) cut problem where the input is a graph with a specified collection of terminal nodes and the goal is to find a smallest subset of nonterminal nodes (edges) to delete so that the terminal nodes do not have an odd length path between them. In an earlier work, Lokshtanov and Ramanujan showed that both odd multiway node cut and odd multiway edge cut are fixed-parameter tractable (FPT) when parameterized by the size of the solution in undirected graphs. In this work, we focus on directed acyclic graphs (DAGs) and design a fixed-parameter algorithm. Our main contribution is a broadening of the shadow-removal framework to address parity problems in DAGs. We complement our FPT results with tight approximability as well as polyhedral results for 2 terminals in DAGs. Additionally, we show inapproximability results for odd multiway edge cut in undirected graphs even for 2 terminals.
Unconstrained Face Detection and Open-Set Face Recognition Challenge
Venue: arXiv cs.CV 2018
Year: 2018

Face detection and recognition benchmarks have shifted toward more difficult environments. The challenge presented in this paper addresses the next step in the direction of automatic detection and identification of people from outdoor surveillance cameras. While face detection has shown remarkable success in images collected from the web, surveillance cameras include more diverse occlusions, poses, weather conditions and image blur. Although face verification or closed-set face identification have surpassed human capabilities on some datasets, open-set identification is much more complex as it needs to reject both unknown identities and false accepts from the face detector. We show that unconstrained face detection can approach high detection rates albeit with moderate false accept rates. By contrast, open-set face recognition is currently weak and requires much more attention.
Has the Online Discussion Been Manipulated? Quantifying Online Discussion Authenticity within Online Social Media
Venue: arXiv cs.SI 2018
Year: 2018

Online social media (OSM) has a enormous influence in today's world. Some individuals view OSM as fertile ground for abuse and use it to disseminate misinformation and political propaganda, slander competitors, and spread spam. The crowdturfing industry employs large numbers of bots and human workers to manipulate OSM and misrepresent public opinion. The detection of online discussion topics manipulated by OSM \emph{abusers} is an emerging issue attracting significant attention. In this paper, we propose an approach for quantifying the authenticity of online discussions based on the similarity of OSM accounts participating in the discussion to known abusers and legitimate accounts. Our method uses several similarity functions for the analysis and classification of OSM accounts. The proposed methods are demonstrated using Twitter data collected for this study and previously published \emph{Arabic honeypot dataset}. The former includes manually labeled accounts and abusers who participated in crowdturfing platforms. Evaluation of the topic's authenticity, derived from account similarity functions, shows that the suggested approach is effective for discriminating between topics that were strongly promoted by abusers and topics that attracted authentic public interest.
Cell Detection in Microscopy Images with Deep Convolutional Neural Network and Compressed Sensing
Venue: arXiv cs.CV 2018
Year: 2018

The ability to automatically detect certain types of cells or cellular subunits in microscopy images is of significant interest to a wide range of biomedical research and clinical practices. Cell detection methods have evolved from employing hand-crafted features to deep learning-based techniques. The essential idea of these methods is that their cell classifiers or detectors are trained in the pixel space, where the locations of target cells are labeled. In this paper, we seek a different route and propose a convolutional neural network (CNN)-based cell detection method that uses encoding of the output pixel space. For the cell detection problem, the output space is the sparsely labeled pixel locations indicating cell centers. We employ random projections to encode the output space to a compressed vector of fixed dimension. Then, CNN regresses this compressed vector from the input pixels. Furthermore, it is possible to stably recover sparse cell locations on the output pixel space from the predicted compressed vector using $L_1$-norm optimization. In the past, output space encoding using compressed sensing (CS) has been used in conjunction with linear and non-linear predictors. To the best of our knowledge, this is the first successful use of CNN with CS-based output space encoding. We made substantial experiments on several benchmark datasets, where the proposed CNN + CS framework (referred to as CNNCS) achieved the highest or at least top-3 performance in terms of F1-score, compared with other state-of-the-art methods.
Output Reachable Set Estimation and Verification for Multi-Layer Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

In this paper, the output reachable estimation and safety verification problems for multi-layer perceptron neural networks are addressed. First, a conception called maximum sensitivity in introduced and, for a class of multi-layer perceptrons whose activation functions are monotonic functions, the maximum sensitivity can be computed via solving convex optimization problems. Then, using a simulation-based method, the output reachable set estimation problem for neural networks is formulated into a chain of optimization problems. Finally, an automated safety verification is developed based on the output reachable set estimation result. An application to the safety verification for a robotic arm model with two joints is presented to show the effectiveness of proposed approaches.
A Fast Noniterative Algorithm for Compressive Sensing Using Binary Measurement Matrices
Venue: arXiv cs.IT 2018
Year: 2018

In this paper we present a new algorithm for compressive sensing that makes use of binary measurement matrices and achieves exact recovery of ultra sparse vectors, in a single pass and without any iterations. Due to its noniterative nature, our algorithm is hundreds of times faster than $\ell_1$-norm minimization, and methods based on expander graphs, both of which require multiple iterations. Our algorithm can accommodate nearly sparse vectors, in which case it recovers index set of the largest components, and can also accommodate burst noise measurements. Compared to compressive sensing methods that are guaranteed to achieve exact recovery of all sparse vectors, our method requires fewer measurements However, methods that achieve statistical recovery, that is, recovery of almost all but not all sparse vectors, can require fewer measurements than our method.
Temporal Upsampling of Depth Maps Using a Hybrid Camera
Venue: arXiv cs.GR 2018
Year: 2018

In recent years, consumer-level depth cameras have been adopted for various applications. However, they often produce depth maps at only a moderately high frame rate (approximately 30 frames per second), preventing them from being used for applications such as digitizing human performance involving fast motion. On the other hand, low-cost, high-frame-rate video cameras are available. This motivates us to develop a hybrid camera that consists of a high-frame-rate video camera and a low-frame-rate depth camera and to allow temporal interpolation of depth maps with the help of auxiliary color images. To achieve this, we develop a novel algorithm that reconstructs intermediate depth maps and estimates scene flow simultaneously. We test our algorithm on various examples involving fast, non-rigid motions of single or multiple objects. Our experiments show that our scene flow estimation method is more precise than a tracking-based method and the state-of-the-art techniques.
Multi-Level Power-Imbalance Allocation Control for Secondary Frequency Control of Power Systems
Venue: arXiv cs.SY 2018
Year: 2018

A consensus-control-based multi-level control law named Multi-Level Power-Imbalance Allocation Control (MLPIAC) is presented for a large-scale power system partitioned into two or more areas. Centralized control is implemented in each area while distributed control is implemented at the coordination level of the areas. Besides restoring nominal frequency with a minimal control cost, MLPIAC can improve the transient performance of the system through an accelerated convergence of the control inputs without oscillations. At the coordination level of the control areas, because the number of the areas is smaller than that of nodes, MLPIAC is more effective to obtain the minimized control cost than the purely distributed control law. At the level of the control in each area, because the number of nodes is much smaller than the total number of nodes in the whole network, the overheads in the communications and the computations are reduced compared to the pure centralized control. The asymptotic stability of MLPIAC is proven using the Lyapunov method and the performance is evaluated through simulations.
A learning framework for winner-take-all networks with stochastic synapses
Venue: arXiv cs.NE 2018
Year: 2018

Many recent generative models make use of neural networks to transform the probability distribution of a simple low-dimensional noise process into the complex distribution of the data. This raises the question of whether biological networks operate along similar principles to implement a probabilistic model of the environment through transformations of intrinsic noise processes. The intrinsic neural and synaptic noise processes in biological networks, however, are quite different from the noise processes used in current abstract generative networks. This, together with the discrete nature of spikes and local circuit interactions among the neurons, raises several difficulties when using recent generative modeling frameworks to train biologically motivated models. In this paper, we show that a biologically motivated model based on multi-layer winner-take-all (WTA) circuits and stochastic synapses admits an approximate analytical description. This allows us to use the proposed networks in a variational learning setting where stochastic backpropagation is used to optimize a lower bound on the data log likelihood, thereby learning a generative model of the data. We illustrate the generality of the proposed networks and learning technique by using them in a structured output prediction task, and in a semi-supervised learning task. Our results extend the domain of application of modern stochastic network architectures to networks where synaptic transmission failure is the principal noise mechanism.
Economic Factors of Vulnerability Trade and Exploitation
Venue: arXiv cs.CR 2018
Year: 2018

Cybercrime markets support the development and diffusion of new attack technologies, vulnerability exploits, and malware. Whereas the revenue streams of cyber attackers have been studied multiple times in the literature, no quantitative account currently exists on the economics of attack acquisition and deployment. Yet, this understanding is critical to characterize the production of (traded) exploits, the economy that drives it, and its effects on the overall attack scenario. In this paper we provide an empirical investigation of the economics of vulnerability exploitation, and the effects of market factors on likelihood of exploit. Our data is collected first-handedly from a prominent Russian cybercrime market where the trading of the most active attack tools reported by the security industry happens. Our findings reveal that exploits in the underground are priced similarly or above vulnerabilities in legitimate bug-hunting programs, and that the refresh cycle of exploits is slower than currently often assumed. On the other hand, cybercriminals are becoming faster at introducing selected vulnerabilities, and the market is in clear expansion both in terms of players, traded exploits, and exploit pricing. We then evaluate the effects of these market variables on likelihood of attack realization, and find strong evidence of the correlation between market activity and exploit deployment. We discuss implications on vulnerability metrics, economics, and exploit measurement.