Query: neural network

title: Neural Aggregation Network for Video Face Recognition
abstract: This paper presents a Neural Aggregation Network (NAN) for video face recognition. The network takes a face video or face image set of a person with a variable number of face images as its input, and produces a compact, fixed dimension feature representation for recognition. The whole network is composed of two modules. The feature embedding module is a deep Convolutional Neural Network (CNN) which maps each face image to a feature vector. The aggregation module consists of two attention blocks which adaptively aggregate the feature vectors to form a single feature inside the convex hull spanned by them. Due to the attention mechanism, the aggregation is invariant to the image order. Our NAN is trained with a standard classification or verification loss without any extra supervision signal, and we found that it automatically learns to advocate high quality face images while repelling low quality ones such as blurred, occluded and improperly exposed faces. The experiments on IJB A, YouTube Face, Celebrity 1000 video face recognition benchmarks show that it consistently outperforms naive aggregation methods and achieves the state of the art accuracy.

title: An End to End Trainable Neural Network for Image Based Sequence Recognition and Its Application to Scene Text Recognition
abstract: Image based sequence recognition has been a long standing research topic in computer vision. In this paper, we investigate the problem of scene text recognition, which is among the most important and challenging tasks in image based sequence recognition. A novel neural network architecture, which integrates feature extraction, sequence modeling and transcription into a unified framework, is proposed. Compared with previous systems for scene text recognition, the proposed architecture possesses four distinctive properties: (1) It is end to end trainable, in contrast to most of the existing algorithms whose components are separately trained and tuned. (2) It naturally handles sequences in arbitrary lengths, involving no character segmentation or horizontal scale normalization. (3) It is not confined to any predefined lexicon and achieves remarkable performances in both lexicon free and lexicon based scene text recognition tasks. (4) It generates an effective yet much smaller model, which is more practical for real world application scenarios. The experiments on standard benchmarks, including the IIIT 5K, Street View Text and ICDAR datasets, demonstrate the superiority of the proposed algorithm over the prior arts. Moreover, the proposed algorithm performs well in the task of image based music score recognition, which evidently verifies the generality of it.

title: A simple neural network module for relational reasoning
abstract: Relational reasoning is a central component of generally intelligent behavior, but has proven difficult for neural networks to learn. In this paper we describe how to use Relation Networks (RNs) as a simple plug and play module to solve problems that fundamentally hinge on relational reasoning. We tested RN augmented networks on three tasks: visual question answering using a challenging dataset called CLEVR, on which we achieve state of the art, super human performance; text based question answering using the bAbI suite of tasks; and complex reasoning about dynamic physical systems. Then, using a curated dataset called Sort of CLEVR we show that powerful convolutional networks do not have a general capacity to solve relational questions, but can gain this capacity when augmented with RNs. Our work shows how a deep learning architecture equipped with an RN module can implicitly discover and learn to reason about entities and their relations.

title: Deep Convolutional Neural Network for Inverse Problems in Imaging
abstract: In this paper, we propose a novel deep convolutional neural network (CNN) based algorithm for solving ill posed inverse problems. Regularized iterative algorithms have emerged as the standard approach to ill posed inverse problems in the past few decades. These methods produce excellent results, but can be challenging to deploy in practice due to factors including the high computational cost of the forward and adjoint operators and the difficulty of hyperparameter selection. The starting point of this paper is the observation that unrolled iterative methods have the form of a CNN (filtering followed by pointwise non linearity) when the normal operator <inline formula> <tex math notation="LaTeX"$H^}H$ /tex math>/inline formula> where <inline formula> <tex math notation="LaTeX"$H^ /tex math>/inline formula> is the adjoint of the forward imaging operator, <inline formula> <tex math notation="LaTeX"$H$ /tex math>/inline formula> of the forward model is a convolution. Based on this observation, we propose using direct inversion followed by a CNN to solve normal convolutional inverse problems. The direct inversion encapsulates the physical model of the system, but leads to artifacts when the problem is ill posed; the CNN combines multiresolution decomposition and residual learning in order to learn to remove these artifacts while preserving image structure. We demonstrate the performance of the proposed network in sparse view reconstruction (down to 50 views) on parallel beam X ray computed tomography in synthetic phantoms as well as in real experimental sinograms. The proposed network outperforms total variation regularized iterative reconstruction for the more realistic phantoms and requires less than a second to reconstruct a <inline formula> <tex math notation="LaTeX"$512\times 512$ /tex math>/inline formula> image on the GPU.

title: Designing Neural Network Architectures using Reinforcement Learning
abstract: At present, designing convolutional neural network (CNN) architectures requires both human expertise and labor. New architectures are handcrafted by careful experimentation or modified from a handful of existing networks. We introduce MetaQNN, a meta modeling algorithm based on reinforcement learning to automatically generate high performing CNN architectures for a given learning task. The learning agent is trained to sequentially choose CNN layers using $Q$ learning with an \epsilon$ greedy exploration strategy and experience replay. The agent explores a large but finite space of possible architectures and iteratively discovers designs with improved performance on the learning task. On image classification benchmarks, the agent designed networks (consisting of only standard convolution, pooling, and fully connected layers) beat existing networks designed with the same layer types and are competitive against the state of the art methods that use more complex layer types. We also outperform existing meta modeling approaches for network design on image classification tasks.

title: Distilling the Knowledge in a Neural Network
abstract: A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions. Unfortunately, making predictions using a whole ensemble of models is cumbersome and may be too computationally expensive to allow deployment to a large number of users, especially if the individual models are large neural nets. Caruana and his collaborators have shown that it is possible to compress the knowledge in an ensemble into a single model which is much easier to deploy and we develop this approach further using a different compression technique. We achieve some surprising results on MNIST and we show that we can significantly improve the acoustic model of a heavily used commercial system by distilling the knowledge in an ensemble of models into a single model. We also introduce a new type of ensemble composed of one or more full models and many specialist models which learn to distinguish fine grained classes that the full models confuse. Unlike a mixture of experts, these specialist models can be trained rapidly and in parallel.

title: Deep Multi scale Convolutional Neural Network for Dynamic Scene Deblurring
abstract: Non uniform blind deblurring for general dynamic scenes is a challenging computer vision problem as blurs arise not only from multiple object motions but also from camera shake, scene depth variation. To remove these complicated motion blurs, conventional energy optimization based methods rely on simple assumptions such that blur kernel is partially uniform or locally linear. Moreover, recent machine learning based methods also depend on synthetic blur datasets generated under these assumptions. This makes conventional deblurring methods fail to remove blurs where blur kernel is difficult to approximate or parameterize (e.g. object motion boundaries) In this work, we propose a multi scale convolutional neural network that restores sharp images in an end to end manner where blur is caused by various sources. Together, we present multi scale loss function that mimics conventional coarse to fine approaches. Furthermore, we propose a new large scale dataset that provides pairs of realistic blurry image and the corresponding ground truth sharp image that are obtained by a high speed camera. With the proposed model trained on this dataset, we demonstrate empirically that our method achieves the state of the art performance in dynamic scene deblurring not only qualitatively, but also quantitatively.

title: DeepFM: A Factorization Machine based Neural Network for CTR Prediction
abstract: Learning sophisticated feature interactions behind user behaviors is critical in maximizing CTR for recommender systems. Despite great progress, existing methods seem to have a strong bias towards low or high order interactions, or require expertise feature engineering. In this paper, we show that it is possible to derive an end to end learning model that emphasizes both low and high order feature interactions. The proposed model, DeepFM, combines the power of factorization machines for recommendation and deep learning for feature learning in a new neural network architecture. Compared to the latest Wide Deep model from Google, DeepFM has a shared input to its "wide" and "deep" parts, with no need of feature engineering besides raw features. Comprehensive experiments are conducted to demonstrate the effectiveness and efficiency of DeepFM over the existing models for CTR prediction, on both benchmark data and commercial data.

title: FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
abstract: Research has shown that convolutional neural networks contain significant redundancy, and high classification accuracy can be obtained even when weights and activations are reduced from floating point to binary values. In this paper, we present FINN, a framework for building fast and flexible FPGA accelerators using a flexible heterogeneous streaming architecture. By utilizing a novel set of optimizations that enable efficient mapping of binarized neural networks to hardware, we implement fully connected, convolutional and pooling layers, with per layer compute resources being tailored to user provided throughput requirements. On a ZC706 embedded FPGA platform drawing less than 25 W total system power, we demonstrate up to 12.3 million image classifications per second with 0.31 ms latency on the MNIST dataset with 95.8% accuracy, and 21906 image classifications per second with 283 ms latency on the CIFAR 10 and SVHN datasets with respectively 80.1% and 94.9% accuracy. To the best of our knowledge, ours are the fastest classification rates reported to date on these benchmarks.

title: Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine Grained Image Recognition
abstract: Recognizing fine grained categories (e.g. bird species) is difficult due to the challenges of discriminative region localization and fine grained feature learning. Existing approaches predominantly solve these challenges independently, while neglecting the fact that region detection and fine grained feature learning are mutually correlated and thus can reinforce each other. In this paper, we propose a novel recurrent attention convolutional neural network (RA CNN) which recursively learns discriminative region attention and region based feature representation at multiple scales in a mutual reinforced way. The learning at each scale consists of a classification sub network and an attention proposal sub network (APN) The APN starts from full images, and iteratively generates region attention from coarse to fine by taking previous prediction as a reference, while the finer scale network takes as input an amplified attended region from previous scale in a recurrent way. The proposed RA CNN is optimized by an intra scale classification loss and an inter scale ranking loss, to mutually learn accurate region attention and fine grained representation. RA CNN does not need bounding box/part annotations and can be trained end to end. We conduct comprehensive experiments and show that RA CNN achieves the best performance in three fine grained tasks, with relative accuracy gains of 3.3% 3.7% 3.8% on CUB Birds, Stanford Dogs and Stanford Cars, respectively.