OpenRaiser/Intern-Atlas huggingface dataset:
Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks
Venue: arXiv cs.CV 2015
Year: 2015

Convolutional Neural Networks have achieved state-of-the-art performance on a wide range of tasks. Most benchmarks are led by ensembles of these powerful learners, but ensembling is typically treated as a post-hoc procedure implemented by averaging independently trained models with model variation induced by bagging or random initialization. In this paper, we rigorously treat ensembling as a first-class problem to explicitly address the question: what are the best strategies to create an ensemble? We first compare a large number of ensembling strategies, and then propose and evaluate novel strategies, such as parameter sharing (through a new family of models we call TreeNets) as well as training under ensemble-aware and diversity-encouraging losses. We demonstrate that TreeNets can improve ensemble performance and that diverse ensembles can be trained end-to-end under a unified loss, achieving significantly higher "oracle" accuracies than classical ensembles.
Decentralization of a Machine: Some Definitions
Venue: arXiv cs.CC 2015
Year: 2015

We define some notions of the decentralization of a deterministic input-output machine. This opens the possibility for introducing game-theoretic elements -- such as strategic players -- inside the machine, as part of its design.
QBDC: Query by dropout committee for training deep supervised architecture
Venue: arXiv cs.LG 2015
Year: 2015

While the current trend is to increase the depth of neural networks to increase their performance, the size of their training database has to grow accordingly. We notice an emergence of tremendous databases, although providing labels to build a training set still remains a very expensive task. We tackle the problem of selecting the samples to be labelled in an online fashion. In this paper, we present an active learning strategy based on query by committee and dropout technique to train a Convolutional Neural Network (CNN). We derive a commmittee of partial CNNs resulting from batchwise dropout runs on the initial CNN. We evaluate our active learning strategy for CNN on MNIST benchmark, showing in particular that selecting less than 30 % from the annotated database is enough to get similar error rate as using the full training set on MNIST. We also studied the robustness of our method against adversarial examples.
Fast Parallel SAME Gibbs Sampling on General Discrete Bayesian Networks
Venue: arXiv cs.LG 2015
Year: 2015

A fundamental task in machine learning and related fields is to perform inference on Bayesian networks. Since exact inference takes exponential time in general, a variety of approximate methods are used. Gibbs sampling is one of the most accurate approaches and provides unbiased samples from the posterior but it has historically been too expensive for large models. In this paper, we present an optimized, parallel Gibbs sampler augmented with state replication (SAME or State Augmented Marginal Estimation) to decrease convergence time. We find that SAME can improve the quality of parameter estimates while accelerating convergence. Experiments on both synthetic and real data show that our Gibbs sampler is substantially faster than the state of the art sampler, JAGS, without sacrificing accuracy. Our ultimate objective is to introduce the Gibbs sampler to researchers in many fields to expand their range of feasible inference problems.
Neural Network Matrix Factorization
Venue: arXiv cs.LG 2015
Year: 2015

Data often comes in the form of an array or matrix. Matrix factorization techniques attempt to recover missing or corrupted entries by assuming that the matrix can be written as the product of two low-rank matrices. In other words, matrix factorization approximates the entries of the matrix by a simple, fixed function---namely, the inner product---acting on the latent feature vectors for the corresponding row and column. Here we consider replacing the inner product by an arbitrary function that we learn from the data at the same time as we learn the latent feature vectors. In particular, we replace the inner product by a multi-layer feed-forward neural network, and learn by alternating between optimizing the network for fixed latent features, and optimizing the latent features for a fixed network. The resulting approach---which we call neural network matrix factorization or NNMF, for short---dominates standard low-rank techniques on a suite of benchmark but is dominated by some recent proposals that take advantage of the graph features. Given the vast range of architectures, activation functions, regularizers, and optimization techniques that could be used within the NNMF framework, it seems likely the true potential of the approach has yet to be reached.
Deep Metric Learning via Lifted Structured Feature Embedding
Venue: arXiv cs.CV 2015
Year: 2015

Learning the distance metric between pairs of examples is of great importance for learning and visual recognition. With the remarkable success from the state of the art convolutional neural networks, recent works have shown promising results on discriminatively training the networks to learn semantic feature embeddings where similar examples are mapped close to each other and dissimilar examples are mapped farther apart. In this paper, we describe an algorithm for taking full advantage of the training batches in the neural network training by lifting the vector of pairwise distances within the batch to the matrix of pairwise distances. This step enables the algorithm to learn the state of the art feature embedding by optimizing a novel structured prediction objective on the lifted problem. Additionally, we collected Online Products dataset: 120k images of 23k classes of online products for metric learning. Our experiments on the CUB-200-2011, CARS196, and Online Products datasets demonstrate significant improvement over existing deep feature embedding methods on all experimented embedding sizes with the GoogLeNet network.
Faster Parallel Solver for Positive Linear Programs via Dynamically-Bucketed Selective Coordinate Descent
Venue: arXiv cs.DS 2015
Year: 2015

We provide improved parallel approximation algorithms for the important class of packing and covering linear programs. In particular, we present new parallel $ε$-approximate packing and covering solvers which run in $\tilde{O}(1/ε^2)$ expected time, i.e., in expectation they take $\tilde{O}(1/ε^2)$ iterations and they do $\tilde{O}(N/ε^2)$ total work, where $N$ is the size of the constraint matrix and $ε$ is the error parameter, and where the $\tilde{O}$ hides logarithmic factors. To achieve our improvement, we introduce an algorithmic technique of broader interest: dynamically-bucketed selective coordinate descent (DB-SCD). At each step of the iterative optimization algorithm, the DB-SCD method dynamically buckets the coordinates of the gradient into those of roughly equal magnitude, and it updates all the coordinates in one of the buckets. This dynamically-bucketed updating permits us to take steps along several coordinates with similar-sized gradients, thereby permitting more appropriate step sizes at each step of the algorithm. In particular, this technique allows us to use in a straightforward manner the recent analysis from the breakthrough results of Allen-Zhu and Orecchia [2] to achieve our still-further improved bounds. More generally, this method addresses "interference" among coordinates, by which we mean the impact of the update of one coordinate on the gradients of other coordinates. Such interference is a core issue in parallelizing optimization routines that rely on smoothness properties. Since our DB-SCD method reduces interference via updating a selective subset of variables at each iteration, we expect it may also have more general applicability in optimization.
WIDER FACE: A Face Detection Benchmark
Venue: arXiv cs.CV 2015
Year: 2015

Face detection is one of the most studied topics in the computer vision community. Much of the progresses have been made by the availability of face detection benchmark datasets. We show that there is a gap between current face detection performance and the real world requirements. To facilitate future face detection research, we introduce the WIDER FACE dataset, which is 10 times larger than existing datasets. The dataset contains rich annotations, including occlusions, poses, event categories, and face bounding boxes. Faces in the proposed dataset are extremely challenging due to large variations in scale, pose and occlusion, as shown in Fig. 1. Furthermore, we show that WIDER FACE dataset is an effective training source for face detection. We benchmark several representative detection systems, providing an overview of state-of-the-art performance and propose a solution to deal with large scale variation. Finally, we discuss common failure cases that worth to be further investigated. Dataset can be downloaded at: mmlab.ie.cuhk.edu.hk/projects/WIDERFace
Actually, It's About Ethics in Computational Social Science: A Multi-party Risk-Benefit Framework for Online Community Research
Venue: arXiv cs.CY 2015
Year: 2015

Managers regularly face a complex ethical dilemma over how to best govern online communities by evaluating the effectiveness of different social or technical strategies. What ethical considerations should guide researchers and managers when they employ causal research methods that make different community members bear different risks and benefits, under different levels of consent? We introduce a structural framework for evaluating the flows of risks and benefits in social systems with multiple interacting parties. This framework has implications for understanding the governmentality of managing socio-technical systems, for making research ethics discussions more commensurable, and for enumerating alternative goals researchers might pursue with interventions.
Towards Arbitrary-View Face Alignment by Recommendation Trees
Venue: arXiv cs.CV 2015
Year: 2015

Learning to simultaneously handle face alignment of arbitrary views, e.g. frontal and profile views, appears to be more challenging than we thought. The difficulties lay in i) accommodating the complex appearance-shape relations exhibited in different views, and ii) encompassing the varying landmark point sets due to self-occlusion and different landmark protocols. Most existing studies approach this problem via training multiple viewpoint-specific models, and conduct head pose estimation for model selection. This solution is intuitive but the performance is highly susceptible to inaccurate head pose estimation. In this study, we address this shortcoming through learning an Ensemble of Model Recommendation Trees (EMRT), which is capable of selecting optimal model configuration without prior head pose estimation. The unified framework seamlessly handles different viewpoints and landmark protocols, and it is trained by optimising directly on landmark locations, thus yielding superior results on arbitrary-view face alignment. This is the first study that performs face alignment on the full AFLWdataset with faces of different views including profile view. State-of-the-art performances are also reported on MultiPIE and AFW datasets containing both frontaland profile-view faces.
L1 logistic regression as a feature selection step for training stable classification trees for the prediction of severity criteria in imported malaria
Venue: arXiv cs.LG 2015
Year: 2015

Multivariate classification methods using explanatory and predictive models are necessary for characterizing subgroups of patients according to their risk profiles. Popular methods include logistic regression and classification trees with performances that vary according to the nature and the characteristics of the dataset. In the context of imported malaria, we aimed at classifying severity criteria based on a heterogeneous patient population. We investigated these approaches by implementing two different strategies: L1 logistic regression (L1LR) that models a single global solution and classification trees that model multiple local solutions corresponding to discriminant subregions of the feature space. For each strategy, we built a standard model, and a sparser version of it. As an alternative to pruning, we explore a promising approach that first constrains the tree model with an L1LR-based feature selection, an approach we called L1LR-Tree. The objective is to decrease its vulnerability to small data variations by removing variables corresponding to unstable local phenomena. Our study is twofold: i) from a methodological perspective comparing the performances and the stability of the three previous methods, i.e L1LR, classification trees and L1LR-Tree, for the classification of severe forms of imported malaria, and ii) from an applied perspective improving the actual classification of severe forms of imported malaria by identifying more personalized profiles predictive of several clinical criteria based on variables dismissed for the clinical definition of the disease. The main methodological results show that the combined method L1LR-Tree builds sparse and stable models that significantly predicts the different severity criteria and outperforms all the other methods in terms of accuracy.
Study of Sparsity-Aware Distributed Conjugate Gradient Algorithms for Sensor Networks
Venue: arXiv cs.IT 2015
Year: 2015

This paper proposes distributed adaptive algorithms based on the conjugate gradient (CG) method and the diffusion strategy for parameter estimation over sensor networks. We present sparsity-aware conventional and modified distributed CG algorithms using $l_{1}$ and log-sum penalty functions. The proposed sparsity-aware diffusion distributed CG algorithms have an improved performance in terms of mean square deviation (MSD) and convergence as compared with the consensus least-mean square (Diffusion-LMS) algorithm, the diffusion CG algorithms and a close performance to the diffusion distributed recursive least squares (Consensus-RLS) algorithm. Numerical results show that the proposed algorithms are reliable and can be applied in several scenarios.
Deep End2End Voxel2Voxel Prediction
Venue: arXiv cs.CV 2015
Year: 2015

Over the last few years deep learning methods have emerged as one of the most prominent approaches for video analysis. However, so far their most successful applications have been in the area of video classification and detection, i.e., problems involving the prediction of a single class label or a handful of output variables per video. Furthermore, while deep networks are commonly recognized as the best models to use in these domains, there is a widespread perception that in order to yield successful results they often require time-consuming architecture search, manual tweaking of parameters and computationally intensive pre-processing or post-processing methods. In this paper we challenge these views by presenting a deep 3D convolutional architecture trained end to end to perform voxel-level prediction, i.e., to output a variable at every voxel of the video. Most importantly, we show that the same exact architecture can be used to achieve competitive results on three widely different voxel-prediction tasks: video semantic segmentation, optical flow estimation, and video coloring. The three networks learned on these problems are trained from raw video without any form of preprocessing and their outputs do not require post-processing to achieve outstanding performance. Thus, they offer an efficient alternative to traditional and much more computationally expensive methods in these video domains.
Through-Wall Person Localization Using Transceivers in Motion
Venue: arXiv cs.ET 2015
Year: 2015

We develop novel methods for device-free localization (DFL) using transceivers in motion. Such localization technologies are useful in various cross-layer applications/protocols including those that are related to security situations where it is important to know the presence and position of an unauthorized person; in monitoring the daily activities of elderly or special needs individuals; or in emergency situations when police or firefighters can use the locations of people inside of a building in order to save lives. We propose that transceivers mounted on autonomous vehicles could be both quickly deployed and kept moving to ``sweep'' an area for changes in the channel that would indicate the location of moving people and objects. The challenge is that changes to channel measurements are introduced both by changes in the environment and from motion of the transceivers. In this paper, we demonstrate a method to detect human movement despite transceiver motion using ultra-wideband impulse radar (UWB-IR) transceivers. The measurements reliably detect a person's presence on a link line despite small-scale fading. We explore via multiple experiments the ability of mobile UWB-IR transceivers, moving outside of the walls of a room, to measure many lines crossing through the room and accurately locate a person inside within 0.25 m average error.
Semantic Diversity versus Visual Diversity in Visual Dictionaries
Venue: arXiv cs.CV 2015
Year: 2015

Visual dictionaries are a critical component for image classification/retrieval systems based on the bag-of-visual-words (BoVW) model. Dictionaries are usually learned without supervision from a training set of images sampled from the collection of interest. However, for large, general-purpose, dynamic image collections (e.g., the Web), obtaining a representative sample in terms of semantic concepts is not straightforward. In this paper, we evaluate the impact of semantics in the dictionary quality, aiming at verifying the importance of semantic diversity in relation visual diversity for visual dictionaries. In the experiments, we vary the amount of classes used for creating the dictionary and then compute different BoVW descriptors, using multiple codebook sizes and different coding and pooling methods (standard BoVW and Fisher Vectors). Results for image classification show that as visual dictionaries are based on low-level visual appearances, visual diversity is more important than semantic diversity. Our conclusions open the opportunity to alleviate the burden in generating visual dictionaries as we need only a visually diverse set of images instead of the whole collection to create a good dictionary.
EMinRET: Heuristic for Energy-Aware VM Placement with Fixed Intervals and Non-preemption
Venue: arXiv cs.NI 2015
Year: 2015

Infrastructure-as-a-Service (IaaS) clouds have become more popular enabling users to run applications under virtual machines. This paper investigates the energy-aware virtual machine (VM) allocation problems in IaaS clouds along characteristics: multiple resources, and fixed interval times and non-preemption of virtual machines. Many previous works proposed to use a minimum number of physical machines, however, this is not necessarily a good solution to minimize total energy consumption in the VM placement with multiple resources, fixed interval times and non-preemption. We observed that minimizing total energy consumption of physical machines is equivalent to minimize the sum of total completion time of all physical machines. Based on the observation, we propose EMinRET algorithm. The EMinRET algorithm swaps an allocating VM with a suitable overlapped VM, which is of the same VM type and is allocated on the same physical machine, to minimize total completion time of all physical machines. The EMinRET uses resource utilization during executing time period of a physical machine as the evaluation metric, and will then choose a host that minimizes the metric to allocate a new VM. In addition, this work studies some heuristics for sorting the list of virtual machines (e.g., sorting by the earliest starting time, or the longest duration time first, etc.) to allocate VM. Using the realistic log-trace in the Parallel Workloads Archive, our simulation results show that the EMinRET algorithm could reduce from 25% to 45% energy consumption compared with power-aware best-fit decreasing (PABFD)) and vector bin-packing norm-based greedy algorithms. Moreover, the EMinRET heuristic has also less total energy consumption than our previous heuristics (e.g. MinDFT and EPOBF) in the simulations (using same virtual machines sorting method).
Fidelity-Naturalness Evaluation of Single Image Super Resolution
Venue: arXiv cs.CV 2015
Year: 2015

We study the problem of evaluating super resolution methods. Traditional evaluation methods usually judge the quality of super resolved images based on a single measure of their difference with the original high resolution images. In this paper, we proposed to use both fidelity (the difference with original images) and naturalness (human visual perception of super resolved images) for evaluation. For fidelity evaluation, a new metric is proposed to solve the bias problem of traditional evaluation. For naturalness evaluation, we let humans label preference of super resolution results using pair-wise comparison, and test the correlation between human labeling results and image quality assessment metrics' outputs. Experimental results show that our fidelity-naturalness method is better than the traditional evaluation method for super resolution methods, which could help future research on single-image super resolution.
Ironing in the Dark
Venue: arXiv cs.GT 2015
Year: 2015

This paper presents the first polynomial-time algorithm for position and matroid auction environments that learns, from samples from an unknown bounded valuation distribution, an auction with expected revenue arbitrarily close to the maximum possible. In contrast to most previous work, our results apply to arbitrary (not necessarily regular) distributions and the strongest possible benchmark, the Myerson-optimal auction. Learning a near-optimal auction for an irregular distribution is technically challenging because it requires learning the appropriate "ironed intervals," a delicate global property of the distribution.
Differential positivity characterizes one-dimensional normally hyperbolic attractors
Venue: arXiv cs.SY 2015
Year: 2015

The paper shows that normally hyperbolic one-dimensional compact attractors of smooth dynamical systems are characterized by differential positivity, that is, the pointwise infinitesimal contraction of a smooth cone field. The result is analog to the characterization of zero-dimensional hyperbolic attractors by differential stability, which is the pointwise infinitesimal contraction of a Riemannian metric.
Multi-Volume High Resolution RGB-D Mapping with Dynamic Volume Placement
Venue: arXiv cs.RO 2015
Year: 2015

We present a novel RGB-D mapping system for generating 3D maps over spatially extended regions with higher resolution than current methods using multiple, dynamically placed mapping volumes. Our method takes in RGB-D frames and dynamically assigns multiple mapping volumes to the environment, exchanging mapping volumes between the CPU and GPU. Mapping volumes are added or removed as needed to allow for spatially extended, high resolution mapping. Our system is designed to maximize the resolution possible for such volumetric methods, while working on an unbounded space.