OpenRaiser/Intern-Atlas huggingface dataset:
ToPs: Ensemble Learning with Trees of Predictors
Venue: arXiv cs.CV 2018
Year: 2018

We present a new approach to ensemble learning. Our approach constructs a tree of subsets of the feature space and associates a predictor (predictive model) - determined by training one of a given family of base learners on an endogenously determined training set - to each node of the tree; we call the resulting object a tree of predictors. The (locally) optimal tree of predictors is derived recursively; each step involves jointly optimizing the split of the terminal nodes of the previous tree and the choice of learner and training set (hence predictor) for each set in the split. The feature vector of a new instance determines a unique path through the optimal tree of predictors; the final prediction aggregates the predictions of the predictors along this path. We derive loss bounds for the final predictor in terms of the Rademacher complexity of the base learners. We report the results of a number of experiments on a variety of datasets, showing that our approach provides statistically significant improvements over state-of-the-art machine learning algorithms, including various ensemble learning methods. Our approach works because it allows us to endogenously create more complex learners - when needed - and endogenously match both the learner and the training set to the characteristics of the dataset while still avoiding over-fitting.
Group Sparse Precoding for Cloud-RAN with Multiple User Antennas
Venue: arXiv cs.IT 2018
Year: 2018

Cloud radio access network (C-RAN) has become a promising network architecture to support the massive data traffic in the next generation cellular networks. In a C-RAN, a massive number of low-cost remote antenna ports (RAPs) are connected to a single baseband unit (BBU) pool via high-speed low-latency fronthaul links, which enables efficient resource allocation and interference management. As the RAPs are geographically distributed, the group sparse beamforming schemes attracts extensive studies, where a subset of RAPs is assigned to be active and a high spectral efficiency can be achieved. However, most studies assumes that each user is equipped with a single antenna. How to design the group sparse precoder for the multiple antenna users remains little understood, as it requires the joint optimization of the mutual coupling transmit and receive beamformers. This paper formulates an optimal joint RAP selection and precoding design problem in a C-RAN with multiple antennas at each user. Specifically, we assume a fixed transmit power constraint for each RAP, and investigate the optimal tradeoff between the sum rate and the number of active RAPs. Motivated by the compressive sensing theory, this paper formulates the group sparse precoding problem by inducing the $\ell_0$-norm as a penalty and then uses the reweighted $\ell_1$ heuristic to find a solution. By adopting the idea of block diagonalization precoding, the problem can be formulated as a convex optimization, and an efficient algorithm is proposed based on its Lagrangian dual. Simulation results verify that our proposed algorithm can achieve almost the same sum rate as that obtained from exhaustive search.
On The Communication Complexity of High-Dimensional Permutations
Venue: arXiv cs.CC 2018
Year: 2018

We study the multiparty communication complexity of high dimensional permutations, in the Number On the Forehead (NOF) model. This model is due to Chandra, Furst and Lipton (CFL) who also gave a nontrivial protocol for the Exactly-n problem where three players receive integer inputs and need to decide if their inputs sum to a given integer $n$. There is a considerable body of literature dealing with the same problem, where $(\mathbb{N},+)$ is replaced by some other abelian group. Our work can be viewed as a far-reaching extension of this line of work. We show that the known lower bounds for that group-theoretic problem apply to all high dimensional permutations. We introduce new proof techniques that appeal to recent advances in Additive Combinatorics and Ramsey theory. We reveal new and unexpected connections between the NOF communication complexity of high dimensional permutations and a variety of well known and thoroughly studied problems in combinatorics. Previous protocols for Exactly-n all rely on the construction of large sets of integers without a 3-term arithmetic progression. No direct algorithmic protocol was previously known for the problem, and we provide the first such algorithm. This suggests new ways to significantly improve the CFL protocol. Many new open questions are presented throughout.
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Venue: arXiv cs.CV 2018
Year: 2018

Deep learning thrives with large neural networks and large datasets. However, larger networks and larger datasets result in longer training times that impede research and development progress. Distributed synchronous SGD offers a potential solution to this problem by dividing SGD minibatches over a pool of parallel workers. Yet to make this scheme efficient, the per-worker workload must be large, which implies nontrivial growth in the SGD minibatch size. In this paper, we empirically show that on the ImageNet dataset large minibatches cause optimization difficulties, but when these are addressed the trained networks exhibit good generalization. Specifically, we show no loss of accuracy when training with large minibatch sizes up to 8192 images. To achieve this result, we adopt a hyper-parameter-free linear scaling rule for adjusting learning rates as a function of minibatch size and develop a new warmup scheme that overcomes optimization challenges early in training. With these simple techniques, our Caffe2-based system trains ResNet-50 with a minibatch size of 8192 on 256 GPUs in one hour, while matching small minibatch accuracy. Using commodity hardware, our implementation achieves ~90% scaling efficiency when moving from 8 to 256 GPUs. Our findings enable training visual recognition models on internet-scale data with high efficiency.
TIP: Typifying the Interpretability of Procedures
Venue: arXiv cs.AI 2018
Year: 2018

We provide a novel notion of what it means to be interpretable, looking past the usual association with human understanding. Our key insight is that interpretability is not an absolute concept and so we define it relative to a target model, which may or may not be a human. We define a framework that allows for comparing interpretable procedures by linking them to important practical aspects such as accuracy and robustness. We characterize many of the current state-of-the-art interpretable methods in our framework portraying its general applicability. Finally, principled interpretable strategies are proposed and empirically evaluated on synthetic data, as well as on the largest public olfaction dataset that was made recently available \cite{olfs}. We also experiment on MNIST with a simple target model and different oracle models of varying complexity. This leads to the insight that the improvement in the target model is not only a function of the oracle model's performance, but also its relative complexity with respect to the target model. Further experiments on CIFAR-10, a real manufacturing dataset and FICO dataset showcase the benefit of our methods over Knowledge Distillation when the target models are simple and the complex model is a neural network.
LEOS-assisted Inter-GEOS Communication via Distributed-storage Coding
Venue: arXiv cs.IT 2018
Year: 2018

We consider a space communication network consisting of Geosynchronous Earth Orbit satellites (GEOSs) and Low Earth Orbit satellites (LEOSs). In case of no direct communication link between two GEOSs, the data exchange between them is through relay by the LEOSs. In particular, the source GEOS sends coded data to multiple LEOSs based on the distributed storage framework. The destination GEOS then retrieves certain amount of data from each LEOS for data reconstruction. For the GEOS-LEOS downlink, a regenerating-code-based transmission scheme is optimized to guarantee data reconstructability, where the transmission power allocation to the LEOSs is proposed to minimize the total transmission energy. We also consider the power allocation to minimize the total transmission time given the total transmission energy. For the LEOS-GEOS uplink, a flexible partial-downloading coding transmission scheme is proposed to guarantee data reconstructability, where the joint uploaded-data size and power allocations are proposed to minimize the total transmission energy or the total transmission time. Extensive simulation results are presented to evaluate the proposed algorithms and show that regenerating code can achieve lower transmission energy and shorter transmission time for data regeneration than conventional maximum-distance separable (MDS) code.
Battery Recharge Time of a Stochastic Linear and Non-Linear Energy Harvesting System
Venue: arXiv cs.IT 2018
Year: 2018

Systems harvesting energy from a stochastic source have been widely studied in the literature. However, we are not aware of any work that deals with the time it takes for a battery to recharge up to a given level, when the energy source is discrete stochastic. This letter aims to examine the recharge time of a perfect battery. We examine the cases when the energy arrival is a Poisson process, and more generally, a renewal process. We obtain formulas for the distribution of the recharge time as well as the expected value of the recharge time. Using these, we find the switching time of the system, which is then applied to the design of a harvest-then-transmit protocol for green communications. Monte-Carlo simulations verify the obtained formulas.
Segmentation of nearly isotropic overlapped tracks in photomicrographs using successive erosions as watershed markers
Venue: arXiv cs.CV 2018
Year: 2018

The major challenges of automatic track counting are distinguishing tracks and material defects, identifying small tracks and defects of similar size, and detecting overlapping tracks. Here we address the latter issue using WUSEM, an algorithm which combines the watershed transform, morphological erosions and labeling to separate regions in photomicrographs. WUSEM shows reliable results when used in photomicrographs presenting almost isotropic objects. We tested this method in two datasets of diallyl phthalate (DAP) photomicrographs and compared the results when counting manually and using the classic watershed. The mean automatic/manual efficiency ratio when using WUSEM in the test datasets is 0.97 +/- 0.11.
Deep Learning-Based Food Calorie Estimation Method in Dietary Assessment
Venue: arXiv cs.CV 2018
Year: 2018

Obesity treatment requires obese patients to record all food intakes per day. Computer vision has been introduced to estimate calories from food images. In order to increase accuracy of detection and reduce the error of volume estimation in food calorie estimation, we present our calorie estimation method in this paper. To estimate calorie of food, a top view and side view is needed. Faster R-CNN is used to detect the food and calibration object. GrabCut algorithm is used to get each food's contour. Then the volume is estimated with the food and corresponding object. Finally we estimate each food's calorie. And the experiment results show our estimation method is effective.
MATIC: Learning Around Errors for Efficient Low-Voltage Neural Network Accelerators
Venue: arXiv cs.NE 2018
Year: 2018

As a result of the increasing demand for deep neural network (DNN)-based services, efforts to develop dedicated hardware accelerators for DNNs are growing rapidly. However,while accelerators with high performance and efficiency on convolutional deep neural networks (Conv-DNNs) have been developed, less progress has been made with regards to fully-connected DNNs (FC-DNNs). In this paper, we propose MATIC (Memory Adaptive Training with In-situ Canaries), a methodology that enables aggressive voltage scaling of accelerator weight memories to improve the energy-efficiency of DNN accelerators. To enable accurate operation with voltage overscaling, MATIC combines the characteristics of destructive SRAM reads with the error resilience of neural networks in a memory-adaptive training process. Furthermore, PVT-related voltage margins are eliminated using bit-cells from synaptic weights as in-situ canaries to track runtime environmental variation. Demonstrated on a low-power DNN accelerator that we fabricate in 65 nm CMOS, MATIC enables up to 60-80 mV of voltage overscaling (3.3x total energy reduction versus the nominal voltage), or 18.6x application error reduction.
Existence versus Exploitation: The Opacity of Backbones and Backdoors Under a Weak Assumption
Venue: arXiv cs.AI 2018
Year: 2018

Backdoors and backbones of Boolean formulas are hidden structural properties. A natural goal, already in part realized, is that solver algorithms seek to obtain substantially better performance by exploiting these structures. However, the present paper is not intended to improve the performance of SAT solvers, but rather is a cautionary paper. In particular, the theme of this paper is that there is a potential chasm between the existence of such structures in the Boolean formula and being able to effectively exploit them. This does not mean that these structures are not useful to solvers. It does mean that one must be very careful not to assume that it is computationally easy to go from the existence of a structure to being able to get one's hands on it and/or being able to exploit the structure. For example, in this paper we show that, under the assumption that P $\neq$ NP, there are easily recognizable families of Boolean formulas with strong backdoors that are easy to find, yet for which it is hard (in fact, NP-complete) to determine whether the formulas are satisfiable. We also show that, also under the assumption P $\neq$ NP, there are easily recognizable sets of Boolean formulas for which it is hard (in fact, NP-complete) to determine whether they have a large backbone.
Hierarchical Label Inference for Video Classification
Venue: arXiv cs.CV 2018
Year: 2018

Videos are a rich source of high-dimensional structured data, with a wide range of interacting components at varying levels of granularity. In order to improve understanding of unconstrained internet videos, it is important to consider the role of labels at separate levels of abstraction. In this paper, we consider the use of the Bidirectional Inference Neural Network (BINN) for performing graph-based inference in label space for the task of video classification. We take advantage of the inherent hierarchy between labels at increasing granularity. The BINN is evaluated on the first and second release of the YouTube-8M large scale multilabel video dataset. Our results demonstrate the effectiveness of BINN, achieving significant improvements against baseline models.
Face Clustering: Representation and Pairwise Constraints
Venue: arXiv cs.CV 2018
Year: 2018

Clustering face images according to their identity has two important applications: (i) grouping a collection of face images when no external labels are associated with images, and (ii) indexing for efficient large scale face retrieval. The clustering problem is composed of two key parts: face representation and choice of similarity for grouping faces. We first propose a representation based on ResNet, which has been shown to perform very well in image classification problems. Given this representation, we design a clustering algorithm, Conditional Pairwise Clustering (ConPaC), which directly estimates the adjacency matrix only based on the similarity between face images. This allows a dynamic selection of number of clusters and retains pairwise similarity between faces. ConPaC formulates the clustering problem as a Conditional Random Field (CRF) model and uses Loopy Belief Propagation to find an approximate solution for maximizing the posterior probability of the adjacency matrix. Experimental results on two benchmark face datasets (LFW and IJB-B) show that ConPaC outperforms well known clustering algorithms such as k-means, spectral clustering and approximate rank-order. Additionally, our algorithm can naturally incorporate pairwise constraints to obtain a semi-supervised version that leads to improved clustering performance. We also propose an k-NN variant of ConPaC, which has a linear time complexity given a k-NN graph, suitable for large datasets.
Spectral Domain Sampling of Graph Signals
Venue: arXiv cs.IT 2018
Year: 2018

Sampling methods for graph signals in the graph spectral domain are presented. Though conventional sampling of graph signals can be regarded as sampling in the graph vertex domain, it does not have the desired characteristics in regard to the graph spectral domain. With the proposed methods, the down- and upsampled graph signals inherit the frequency domain characteristics of the sampled signals defined in the time/spatial domain. The properties of the sampling effects were evaluated theoretically in comparison with those obtained with the conventional sampling method in the vertex domain. Various examples of signals on simple graphs enable precise understanding of the problem considered. Fractional sampling and Laplacian pyramid representation of graph signals are potential applications of these methods.
Interactive 3D Modeling with a Generative Adversarial Network
Venue: arXiv cs.CV 2018
Year: 2018

This paper proposes the idea of using a generative adversarial network (GAN) to assist a novice user in designing real-world shapes with a simple interface. The user edits a voxel grid with a painting interface (like Minecraft). Yet, at any time, he/she can execute a SNAP command, which projects the current voxel grid onto a latent shape manifold with a learned projection operator and then generates a similar, but more realistic, shape using a learned generator network. Then the user can edit the resulting shape and snap again until he/she is satisfied with the result. The main advantage of this approach is that the projection and generation operators assist novice users to create 3D models characteristic of a background distribution of object shapes, but without having to specify all the details. The core new research idea is to use a GAN to support this application. 3D GANs have previously been used for shape generation, interpolation, and completion, but never for interactive modeling. The new challenge for this application is to learn a projection operator that takes an arbitrary 3D voxel model and produces a latent vector on the shape manifold from which a similar and realistic shape can be generated. We develop algorithms for this and other steps of the SNAP processing pipeline and integrate them into a simple modeling tool. Experiments with these algorithms and tool suggest that GANs provide a promising approach to computer-assisted interactive modeling.
Centralized Multi-Node Repair Regenerating Codes
Venue: arXiv cs.IT 2018
Year: 2018

In a distributed storage system, recovering from multiple failures is a critical and frequent task that is crucial for maintaining the system's reliability and fault-tolerance. In this work, we focus on the problem of repairing multiple failures in a centralized way, which can be desirable in many data storage configurations, and we show that a significant repair traffic reduction is possible. First, the fundamental tradeoff between the repair bandwidth and the storage size for functional repair is established. Using a graph-theoretic formulation, the optimal tradeoff is identified as the solution to an integer optimization problem, for which a closed-form expression is derived. Expressions of the extreme points, namely the minimum storage multi-node repair (MSMR) and minimum bandwidth multi-node repair (MBMR) points, are obtained. Second, we describe a general framework for converting single erasure minimum storage regenerating codes to MSMR codes. The repair strategy for $e$ failures is similar to that for single failure, however certain extra requirements need to be satisfied by the repairing functions for single failure. For illustration, the framework is applied to product-matrix codes and interference alignment codes. Furthermore, we prove that the functional MBMR point is not achievable for linear exact repair codes. We also show that exact-repair minimum bandwidth cooperative repair (MBCR) codes achieve an interior point, that lies near the MBMR point, when $k \equiv 1 \mod e$, $k$ being the minimum number of nodes needed to reconstruct the entire data. Finally, for $k> 2e, e\mid k$ and $e \mid d$, where $d$ is the number of helper nodes during repair, we show that the functional repair tradeoff is not achievable under exact repair, except for maybe a small portion near the MSMR point, which parallels the results for single erasure repair by Shah et al.
Urban Analytics: Multiplexed and Dynamic Community Networks
Venue: arXiv cs.SI 2018
Year: 2018

In the past decade, cities have experienced rapid growth, expansion, and changes in their community structure. Many aspects of critical urban infrastructure are closely coupled with the human communities that they serve. Urban communities are composed of a multiplex of overlapping factors which can be distinguished into cultural, religious, social-economic, political, and geographical layers. In this paper, we review how increasingly available heterogeneous mobile big data sets can be leveraged to detect the community interaction structure using natural language processing and machine learning techniques. A number of community layer and interaction detection algorithms are then reviewed, with a particular focus on robustness, stability, and causality of evolving communities. The better understanding of the structural dynamics and multiplexed relationships can provide useful information to inform both urban planning policies and shape the design of socially coupled urban infrastructure systems.
A Radio-Inertial Localization and Tracking System with BLE Beacons Prior Maps
Venue: arXiv cs.RO 2018
Year: 2018

In this paper, we develop a system for the low-cost indoor localization and tracking problem using radio signal strength indicator, Inertial Measurement Unit (IMU), and magnetometer sensors. We develop a novel and simplified probabilistic IMU motion model as the proposal distribution of the sequential Monte-Carlo technique to track the robot trajectory. Our algorithm can globally localize and track a robot with a priori unknown location, given an informative prior map of the Bluetooth Low Energy (BLE) beacons. Also, we formulate the problem as an optimization problem that serves as the Back-end of the algorithm mentioned above (Front-end). Thus, by simultaneously solving for the robot trajectory and the map of BLE beacons, we recover a continuous and smooth trajectory of the robot, corrected locations of the BLE beacons, and the time-varying IMU bias. The evaluations achieved using hardware show that through the proposed closed-loop system the localization performance can be improved; furthermore, the system becomes robust to the error in the map of beacons by feeding back the optimized map to the Front-end.