mtybilly/Instruct2DS huggingface dataset:
Feature-Independent Action Spotting Without Human Localization, Segmentation or Frame-wise Tracking
Authors: Marshall Tappen, Hassan Foroosh Chuan Sun
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: In this paper, we propose an unsupervised framework for action spotting in videos that does not depend on any specific feature (e.g. HOG/HOF, STIP, silhouette, bag-of-words, etc.). Furthermore, our solution requires no human localization, segmentation, or framewise tracking. This is achieved by treating the problem holistically as that of extracting the internal dynamics of video cuboids by modeling them in their natural form as multilinear tensors. To extract their internal dynamics, we devised a novel Two-Phase Decomposition (TP-Decomp) of a tensor that generates very compact and discriminative representations that are robust to even heavily perturbed data. Technically, a Rank-based Tensor Core Pyramid (Rank-TCP) descriptor is generated by combining multiple tensor cores under multiple ranks, allowing to represent video cuboids in a hierarchical tensor pyramid. The problem then reduces to a template matching problem, which is solved efficiently by using two boosting strategies: (1) to reduce search space, we filter the dense trajectory cloud extracted from the target video; (2) to boost the matching speed, we perform matching in an iterative coarse-to-fine manner. Experiments on 5 benchmarks show that our method outperforms current state-of-the-art under various challenging conditions. We also created a challenging dataset called Heavily Perturbed Video Array (HPVA) to validate the robustness of our framework under heavily perturbed situations.
Multiscale Centerline Detection by Learning a Scale-Space Distance Transform
Authors: Vincent Lepetit, Pascal Fua Amos Sironi
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: We propose a robust and accurate method to extract the centerlines and scale of tubular structures in 2D images and 3D volumes.  Existing techniques rely either on filters designed to respond to ideal cylindrical structures, which lose accuracy when the linear structures become very irregular, or on classification, which is inaccurate because locations on centerlines and locations immediately next to them are extremely difficult to distinguish.  We solve this problem by reformulating centerline detection in terms of a regression problem.  We first train regressors to return the distances to the closest centerline in scale-space, and we apply them to the input images or volumes.  The centerlines and the corresponding scale then correspond to the regressors local maxima, which can be easily identified.  We show that our method outperforms state-of-the-art techniques for various 2D and 3D datasets.
Multivariate General Linear Models (MGLM) on Riemannian Manifolds with Applications to Statistical Analysis of Diffusion Weighted Images
Authors: Nagesh Adluru, Maxwell D. Collins, Moo K. Chung, Barbara B. Bendlin, Sterling C. Johnson, Richard J. Davidson, Vikas Singh Hyunwoo J. Kim
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Linear regression is a parametric model which is ubiquitous in scientific analysis. The classical setup where the observations and responses, i.e., (xi,yi) pairs, are Euclidean is well studied. The setting where yi is manifold valued is a topic of much interest, motivated by applications in shape analysis, topic modeling, and medical imaging. Recent work gives strategies for max-margin classifiers, principal components analysis, and dictionary learning on certain types of manifolds. For parametric regression specifically, results within the last year provide mechanisms to regress one real-valued parameter, xi in R, against a manifold-valued variable, yi in M. We seek to substantially extend the operating range of such methods by deriving schemes for multivariate multiple linear regression — a manifold-valued dependent variable against multiple independent variables, i.e., f : Rn -> M. Our variational algorithm efficiently solves for multiple geodesic bases on the manifold concurrently via gradient updates. This allows us to answer questions such as: what is the relationship of the measurement at voxel y to disease when conditioned on age and gender. We show applications to statistical analysis of diffusion weighted images, which give rise to regression tasks on the manifold GL(n)/O(n) for diffusion tensor images (DTI) and the Hilbert unit sphere for orientation distribution functions (ODF) from high angular resolution acquisition. The companion open-source code is available on nitrc.org/projects/riem_mglm.
Preconditioning for Accelerated Iteratively Reweighted Least Squares in Structured Sparsity Reconstruction
Authors: Junzhou Huang, Lei He, Hongsheng Li Chen Chen
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: In this paper, we propose a novel algorithm for structured sparsity reconstruction. This algorithm is based on the iterative reweighted least squares (IRLS) framework, and accelerated by the preconditioned conjugate gradient method.  The convergence rate of the proposed algorithm is almost the same as that of the traditional IRLS algorithms, that is, exponentially fast. Moreover, with the devised preconditioner, the computational cost for each iteration is significantly less than that of traditional IRLS algorithms, which makes it feasible for large scale problems. Besides the fast convergence, this algorithm can be flexibly applied to standard sparsity, group sparsity, and overlapping group sparsity problems. Experiments are conducted on a practical application compressive sensing magnetic resonance imaging. Results demonstrate that the proposed algorithm achieves superior performance over 9 state-of-the-art algorithms in terms of both accuracy and computational cost.
Joint Coupled-Feature Representation and Coupled Boosting for AD Diagnosis
Authors: Heung-Il Suk, Yang Gao, Dinggang Shen Yinghuan Shi
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Recently, there has been a great interest in computer-aided Alzheimer's Disease (AD) and Mild Cognitive Impairment (MCI) diagnosis. Previous learning based methods defined the diagnosis process as a classification task and directly used the low-level features extracted from neuroimaging data without considering relations among them. However, from a neuroscience point of view, it's well known that a human brain is a complex system that multiple brain regions are anatomically connected and functionally inter- act with each other. Therefore, it is natural to hypothesize that the low-level features extracted from neuroimaging data are related to each other in some ways. To this end, in this paper, we first devise a coupled feature representation by utilizing intra-coupled and inter-coupled interaction relationship. Regarding multi-modal data fusion, we propose a novel coupled boosting algorithm that analyzes the pairwise coupled-diversity correlation between modalities. Specifically, we formulate a new weight updating function, which considers both incorrectly and inconsistently classified samples. In our experiments on the ADNI dataset, the proposed method presented the best performance with accuracies of 94.7% and 80.1% for AD vs. Normal Control (NC) and MCI vs. NC classifications, respectively, outperforming the competing methods and the state-of-the-art methods.
Deformable Registration of Feature-Endowed Point Sets Based on Tensor Fields
Authors: James Ross, George Washko, William M. Wells III, Raul San Jose-Estepar Demian Wassermann
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: The main contribution of this work is a framework to register anatomical structures characterized as a point set where each point has an associated symmetric matrix. These matrices can represent problem-dependent characteristics of the registered structure. For example, in airways, matrices can represent the orientation and thickness of the structure. Our framework relies on a dense tensor field representation which we implement sparsely as a kernel mixture of tensor fields. We equip the space of tensor fields with a norm that serves as a similarity measure. To calculate the optimal transformation between two structures we minimize this measure using an analytical gradient for the similarity measure and the deformation field, which we restrict to be a diffeomorphism. We illustrate the value of our tensor field model by comparing our results with scalar and vector field based models. Finally, we evaluate our registration algorithm on synthetic data sets and validate our approach on manually annotated airway trees.
Tracking Indistinguishable Translucent Objects over Time using Weakly Supervised Structured Learning
Authors: Ferran Diego, Konstantin Gregor, Martin Schiegg, Ullrich Koethe, Marta Zlatic, Fred A. Hamprecht Luca Fiaschi
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: We use weakly supervised structured learning to track and disambiguate the identity of multiple indistinguishable, translucent and deformable objects that can overlap for many frames. For this challenging problem, we propose a novel model which handles occlusions, complex motions and non-rigid deformations by jointly optimizing the flows of multiple latent intensities across frames. These flows are latent variables for which the user cannot directly provide labels. Instead, we leverage a structured learning formulation that uses weak user annotations to find the best hyperparameters of this model. The approach is evaluated on a challenging dataset for the tracking of multiple Drosophila larvae which we make publicly available. Our method tracks multiple larvae in spite of their poor distinguishability and minimizes the number of identity switches during prolonged mutual occlusion.
Scale-space Processing Using Polynomial Representations
Authors: Keiichi Uchimura Gou Koutaki
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: In this study, we propose the application of principal components analysis (PCA) to scale-spaces. PCA is a standard method used in computer vision. The translation of an input image into scale-space is a continuous operation, which requires the extension of conventional finite matrix-based PCA to an infinite number of dimensions. In this study, we use spectral decomposition to resolve this infinite eigenproblem by integration and we propose an approximate solution based on polynomial equations. To clarify its eigensolutions, we apply spectral decomposition to the Gaussian scale-space and scale-normalized Laplacian of Gaussian (LoG) space. As an application of this proposed method, we introduce a method for generating Gaussian blur images and scale-normalized LoG images, where we demonstrate that the accuracy of these images can be very high when calculating an arbitrary scale using a simple linear combination. We also propose a new Scale Invariant Feature Transform (SIFT) detector as a more practical example.
Single Image Layer Separation using Relative Smoothness
Authors: Michael S. Brown Yu Li
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: This paper addresses extracting two layers from an image where one layer is smoother than the other.  This problem arises most notably in intrinsic image decomposition and reflection interference removal.  Layer decomposition from a single-image is inherently ill-posed and solutions require additional constraints to be enforced.  We introduce a novel strategy that regularizes the gradients of the two layers such that one has a long tail distribution and the other a short tail distribution.  While imposing the long tail distribution is a common practice, our introduction of the short tail distribution on the second layer is unique.  We formulate our problem in a probabilistic framework and describe an optimization scheme to solve this regularization with only a few iterations.  We apply our approach to the intrinsic image and reflection removal problems and demonstrate high quality layer separation on par with other techniques but being significantly faster than prevailing methods.
Image Fusion with Local Spectral Consistency and Dynamic Gradient Sparsity
Authors: Yeqing Li, Wei Liu, Junzhou Huang Chen Chen
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: In this paper, we propose a novel method for image fusion from a high resolution panchromatic image and a low resolution multispectral image at the same geographical location. Different from previous methods, we do not make any assumption about the upsampled multispectral image, but only assume that the fused image after downsampling should be close to the original multispectral image. This is a severely ill-posed problem and a dynamic gradient sparsity penalty is thus proposed for regularization. Incorporating the intra- correlations of different bands, this penalty can effectively exploit the prior information (e.g. sharp boundaries) from the panchromatic image. A new convex optimization algorithm is proposed to efficiently solve this problem. Extensive experiments on four multispectral datasets demonstrate that the proposed method significantly outperforms the state-of-the-arts in terms of both spatial and spectral qualities.
Segmentation-Free Dynamic Scene Deblurring
Authors: Kyoung Mu Lee Tae Hyun Kim
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Most state-of-the-art dynamic scene deblurring methods based on accurate motion segmentation assume that motion blur is small or that the specific type of motion causing the blur is known. In this paper, we study a motion segmentation-free dynamic scene deblurring method, which is unlike other conventional methods. When the motion can be approximated to linear motion that is locally (pixel-wise) varying, we can handle various types of blur caused by camera shake, including out-of-plane motion, depth variation, radial distortion, and so on. Thus, we propose a new energy model simultaneously estimating motion flow and the latent image based on robust total variation (TV)-L1 model. This approach is necessary to handle abrupt changes in motion without segmentation. Furthermore, we address the problem of the traditional coarse-to-fine deblurring framework, which gives rise to artifacts when restoring small structures with distinct motion. We thus propose a novel kernel re-initialization method which reduces the error of motion flow propagated from a coarser level. Moreover, a highly effective convex optimization-based solution mitigating the computational difficulties of the TV-L1 model is established. Comparative experimental results on challenging real blurry images demonstrate the efficiency of the proposed method.
Shrinkage Fields for Effective Image Restoration
Authors: Stefan Roth Uwe Schmidt
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Many state-of-the-art image restoration approaches do not scale well to larger images, such as megapixel images common in the consumer segment. Computationally expensive optimization is often the culprit. While efficient alternatives exist, they have not reached the same level of image quality. The goal of this paper is to develop an effective approach to image restoration that offers both computational efficiency and high restoration quality. To that end we propose shrinkage fields, a random field-based architecture that combines the image model and the optimization algorithm in a single unit. The underlying shrinkage operation bears connections to wavelet approaches, but is used here in a random field context. Computational efficiency is achieved by construction through the use of convolution and DFT as the core components; high restoration quality is attained through loss-based training of all model parameters and the use of a cascade architecture. Unlike heavily engineered solutions, our learning approach can be adapted easily to different trade-offs between efficiency and image quality. We demonstrate state-of-the-art restoration results with high levels of computational efficiency, and significant speedup potential through inherent parallelism.
Camouflaging an Object from Many Viewpoints
Authors: Connelly Barnes, Alex Flint, Hanumant Singh, William Freeman Andrew Owens
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: We address the problem of camouflaging a 3D object from the many viewpoints that one might see it from. Given photographs of an object's surroundings, we produce a surface texture that will make the object difficult for a human to detect. To do this, we introduce several background matching algorithms that attempt to make the object look like whatever is behind it. Of course, it is impossible to exactly match the background from every possible viewpoint. Thus our models are forced to make trade-offs between different perceptual factors, such as the conspicuousness of the occlusion boundaries and the amount of texture distortion. We use experiments with human subjects to evaluate the effectiveness of these models for the task of camouflaging a cube, finding that they significantly outperform naïve strategies.
Learning Optimal Seeds for Diffusion-based Salient Object Detection
Authors: Vijay Mahadevan, Nuno Vasconcelos Song Lu
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: In diffusion-based saliency detection, an image is partitioned into superpixels and mapped to a graph, with superpixels as nodes and edge strengths proportional to superpixel similarity. Saliency information is then propagated over the graph using a diffusion process, whose equilibrium state yields the object saliency map. The optimal solution is the product of a propagation matrix and a saliency seed vector that contains a prior saliency assessment. This is obtained from either a bottom-up saliency detector or some heuristics. In this work, we propose a method to learn optimal seeds for object saliency. Two types of features are computed per superpixel: the bottom-up saliency of the superpixel region and a set of mid-level vision features informative of how likely the superpixel is to belong to an object. The combination of features that best discriminates between object and background saliency is then learned, using a large-margin formulation of the discriminant saliency principle. The propagation of the resulting saliency seeds, using a diffusion process, is finally shown to outperform the state of the art on a number of salient object detection datasets.
Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images
Authors: Michael Dorr, David Cox Eleonora Vig
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Saliency prediction typically relies on hand-crafted (multiscale) features that are combined in different ways to form a "master" saliency map, which encodes local image conspicuity. Recent improvements to the state of the art on standard benchmarks such as MIT1003 have been achieved mostly by incrementally adding more and more hand-tuned features (such as car or face detectors) to existing models. In contrast, we here follow an entirely automatic data-driven approach that performs a large-scale search for optimal features. We identify those instances of a richly-parameterized bio-inspired model family (hierarchical neuromorphic networks) that successfully predict image saliency. Because of the high dimensionality of this parameter space, we use automated hyperparameter optimization to efficiently guide the search. The optimal blend of such multilayer features combined with a simple linear classifier achieves excellent performance on several image saliency benchmarks. Our models outperform the state of the art on MIT1003, on which features and classifiers are learned. Without additional training, these models generalize well to two other image saliency data sets, Toronto and NUSEF, despite their different image content. Finally, our algorithm scores best of all the 23 models evaluated to date on the MIT300 saliency challenge, which uses a hidden test set to facilitate an unbiased comparison.
Saliency Detection on Light Field
Authors: Jinwei Ye, Yu Ji, Haibin Ling, Jingyi Yu Nianyi Li
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Existing saliency detection approaches use images as inputs and are sensitive to foreground/background similarities, complex background textures, and occlusions. We explore the problem of using light fields as input for saliency detection. Our technique is enabled by the availability of commercial plenoptic cameras that capture the light field of a scene in a single shot. We show that the unique refocusing capability of light fields provides useful focusness, depths, and objectness cues. We further develop a new saliency detection algorithm tailored for light fields. To validate our approach, we acquire a light field database of a range of indoor and outdoor scenes and generate the ground truth saliency map. Experiments show that our saliency detection scheme can robustly handle challenging scenarios such as similar foreground and background, cluttered background, complex occlusions,  etc., and achieve high accuracy and robustness.
Saliency Optimization from Robust Background Detection
Authors: Shuang Liang, Yichen Wei, Jian Sun Wangjiang Zhu
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Recent progresses in salient object detection have exploited the boundary prior, or background information, to assist other saliency cues such as contrast, achieving state-of-the-art results. However, their usage of boundary prior is very simple, fragile, and the integration with other cues is mostly heuristic. In this work, we present new methods to address these issues. First, we propose a robust background measure, called boundary connectivity. It characterizes the spatial layout of image regions with respect to image boundaries and is much more robust. It has an intuitive geometrical interpretation and presents unique benefits that are absent in previous saliency measures. Second, we propose a principled optimization framework to integrate multiple low level cues, including our background measure, to obtain clean and uniform saliency maps. Our formulation is intuitive, efficient and achieves state-of-the-art results on several benchmark datasets.
A Reverse Hierarchy Model for Predicting Eye Fixations
Authors: Ming Liang, Xiaolin Hu Tianlin Shi
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: A number of psychological and physiological evidences suggest that early visual attention works in a coarse-to-fine way, which lays a basis for the reverse hierarchy theory (RHT). This theory states that attention propagates from the top level of the visual hierarchy that processes gist and abstract information of input, to the bottom level that processes local details. Inspired by the theory, we develop a computational model for saliency detection in images. First, the original image is downsampled to different scales to constitute a pyramid. Then, saliency on each layer is obtained by image super-resolution reconstruction from the layer above, which is defined as unpredictability from this coarse-to-fine reconstruction. Finally, saliency on each layer of the pyramid is fused into stochastic fixations through a probabilistic model, where attention initiates from the top layer and propagates downward through the pyramid.  Extensive experiments on two standard eye-tracking datasets show that the proposed method can achieve competitive results with state-of-the-art models.
100+ Times Faster Weighted Median Filter (WMF)
Authors: Li Xu, Jiaya Jia Qi Zhang
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Weighted median, in the form of either solver or filter, has been employed in a wide range of computer vision solutions for its beneficial properties in sparsity representation. But it is hard to be accelerated due to the spatially varying weight and the median property. We propose a few efficient schemes to reduce computation complexity from O(r^2) to O(r) where r is the kernel size. Our contribution is on a new joint-histogram representation, median tracking, and a new data structure that enables fast data access. The effectiveness of these schemes is demonstrated on optical flow estimation, stereo matching, structure-texture separation, image filtering, to name a few. The running time is largely shortened from several minutes to less than 1 second. The source code is provided in the project website.
Edge-aware Gradient Domain Optimization Framework for Image Filtering by Local Propagation
Authors: Xiaohui Bie, Minying Zhang, Wencheng Wang Miao Hua
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Gradient domain methods are popular for image processing. However, these methods even the edge-preserving ones cannot preserve edges well in some cases. In this paper, we present new constraints explicitly to better preserve edges for general gradient domain image filtering and theoretically analyse why these constraints are edge-aware. Our edge-aware constraints are easy to implement, fast to compute and can be seamlessly integrated into the general gradient domain optimization framework. The improved framework can better preserve edges while maintaining similar image filtering effects as the original image filters. We also demonstrate the strength of our edge-aware constraints on various applications such as image smoothing, image colorization and Poisson image cloning.
Super-Resolving Noisy Images
Authors: Fatih Porikli, Narendra Ahuja Abhishek Singh
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Our goal is to obtain a noise-free, high resolution (HR) image, from an observed, noisy, low resolution (LR) image. The conventional approach of preprocessing the image with a denoising algorithm, followed by applying a super-resolution (SR) algorithm, has an important limitation: Along with noise, some high frequency content of the image (particularly textural detail) is invariably lost during the denoising step. This 'denoising loss' restricts the performance of the subsequent SR step, wherein the challenge is to synthesize such textural details. In this paper, we show that high frequency content in the noisy image (which is ordinarily removed by denoising algorithms) can be effectively used to obtain the missing textural details in the HR domain. To do so, we first obtain HR versions of both the noisy and the denoised images, using a patch-similarity based SR algorithm. We then show that by taking a convex combination of orientation and frequency selective bands of the noisy and the denoised HR images, we can obtain a desired HR image where (i) some of the textural signal lost in the denoising step is effectively recovered in the HR domain, and (ii) additional textures can be easily synthesized by appropriately constraining the parameters of the convex combination. We show that this part-recovery and part-synthesis of textures through our algorithm yields HR images that are visually more pleasing than those obtained using the conventional processing pipeline. Furthermore, our results show a consistent improvement in numerical metrics, further corroborating the ability of our algorithm to recover lost signal.
Sparse Dictionary Learning for Edit Propagation of High-Resolution Images
Authors: Dongqing Zou, Jianwei Li, Xiaochun Cao, Qinping Zhao, Hao Zhang Xiaowu Chen
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: We introduce a method of sparse dictionary learning for edit propagation of high-resolution images or video. Previous approaches for edit propagation typically employ a global optimization over the whole set of image pixels, incurring a prohibitively high memory and time consumption for high-resolution images. Rather than propagating an edit pixel by pixel, we follow the principle of sparse representation to obtain a compact set of representative samples (or features) and perform edit propagation on the samples instead. The sparse set of samples provides an intrinsic basis for an input image, and the coding coefficients capture the linear relationship between all pixels and the samples. The representative set of samples is then optimized by a novel scheme which maximizes the KL-divergence between each sample pair to remove redundant samples. We show several applications of sparsity-based edit propagation including video recoloring, theme editing, and seamless cloning, operating on both color and texture features. We demonstrate that with a sample-to-pixel ratio in the order of 0.01%, signifying a significant reduction on memory consumption, our method still maintains a high-degree of visual fidelity.
Weighted Nuclear Norm Minimization with Application to Image Denoising
Authors: Lei Zhang, Wangmeng Zuo, Xiangchu Feng Shuhang Gu
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: As a convex relaxation of the low rank matrix factorization problem, the nuclear norm minimization has been attracting significant research interest in recent years. The standard nuclear norm minimization regularizes each singular value equally to pursue the convexity of the objective function. However, this greatly restricts its capability and flexibility in dealing with many practical problems (e.g., denoising), where the singular values have clear physical meanings and should be treated differently. In this paper we study the weighted nuclear norm minimization (WNNM) problem, where the singular values are assigned different weights. The solutions of the WNNM problem are analyzed under different weighting conditions. We then apply the proposed WNNM algorithm to image denoising by exploiting the image nonlocal self-similarity. Experimental results clearly show that the proposed WNNM algorithm outperforms many state-of-the-art denoising algorithms such as BM3D in terms of both quantitative measure and visual perception quality.
Using Projection Kurtosis Concentration Of Natural Images For Blind Noise Covariance Matrix Estimation
Authors: Siwei Lyu Xing Zhang
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Kurtosis of 1D projections provides important statistical characteristics of natural images. In this work, we first provide a theoretical underpinning to a recently observed phenomenon known as projection kurtosis concentration that the kurtosis of natural images over different band-pass channels tend to concentrate around a typical value.  Based on this analysis, we further describe a new method to estimate the covariance matrix of correlated Gaussian noise from a noise corrupted image using random band-pass filters. We demonstrate the effectiveness of our blind noise covariance matrix estimation method on natural images.
Blind Image Quality Assessment using Semi-supervised Rectifier Networks
Authors: Neel Joshi, Ashish Kapoor Huixuan Tang
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: It is often desirable to evaluate images quality with a perceptually relevant measure that does not require a reference image.  Recent approaches to this problem use human provided quality scores with machine learning to learn a measure. The biggest hurdles to these efforts are: 1) the difficulty of generalizing across diverse types of distortions and 2) collecting the enormity of human scored training data that is needed to learn the measure. We present a new blind image quality measure that addresses these difficulties by learning a robust, nonlinear kernel regression function using a rectifier neural network.  The method is pre-trained with unlabeled data and fine-tuned with labeled data.  It generalizes across a large set of images and distortion types without the need for a large amount of labeled data.  We evaluate our approach on two benchmark datasets and show that it not only outperforms the current state of the art in blind image quality estimation, but also outperforms the state of the art in non-blind measures.  Furthermore, we show that our semi-supervised approach is robust to using varying amounts of labeled data.
Separable Kernel for Image Deblurring
Authors: Haifeng Liu, Feng Wu, Xiaoyan Sun, Houqiang Li Lu Fang
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: In this paper, we deal with the image deblurring problem in a completely new perspective by proposing separable kernel to represent the inherent properties of the camera and scene system. Specifically, we decompose a blur kernel into three individual descriptors (trajectory, intensity and point spread function) so that they can be optimized separately. To demonstrate the advantages, we extract one-pixel-width trajectories of blur kernels and propose a random perturbation algorithm to optimize them but still keeping their continuity. For many cases, where current deblurring approaches fall into local minimum, excellent deblurred results and correct blur kernels can be obtained by individually optimizing the kernel trajectories. Our work strongly suggests that more constraints and priors should be introduced to blur kernels in solving the deblurring problem because blur kernels have lower dimensions than images.
Joint Depth Estimation and Camera Shake Removal from Single Blurry Image
Authors: Li Xu, Ming-Hsuan Yang Zhe Hu
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Camera shake during exposure time often results in spatially variant blur effect of the image. The non-uniform blur effect is not only caused by the camera motion, but also the depth variation of the scene. The objects close to the camera sensors are likely to appear more blurry than those at a distance in such cases. However, recent non-uniform deblurring methods do not explicitly consider the depth factor or assume fronto-parallel scenes with constant depth for simplicity. While single image non-uniform deblurring is a challenging problem, the blurry results in fact contain depth information which can be exploited. We propose to jointly estimate scene depth and remove non-uniform blur caused by camera motion by exploiting their underlying geometric relationships, with only single blurry image as input. To this end, we present a unified layer-based model for depth-involved deblurring. We provide a novel layer-based solution using matting to partition the layers and an expectation-maximization scheme to solve this problem. This approach largely reduces the number of unknowns and makes the problem tractable. Experiments on challenging examples demonstrate that both depth and camera shake removal can be well addressed within the unified framework.
Deblurring Text Images via L0-Regularized Intensity and Gradient Prior
Authors: Zhe Hu, Zhixun Su, Ming-Hsuan Yang Jinshan Pan
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: We propose a simple yet effective L_0-regularized prior based on intensity and gradient for text image deblurring. The proposed image prior is motivated by observing distinct properties of text images. Based on this prior, we develop an efficient optimization method to generate reliable intermediate results for kernel estimation. The proposed method does not require any complex filtering strategies to select salient edges which are critical to the state-of-the-art deblurring algorithms. We discuss the relationship with other deblurring algorithms based on edge selection and provide insight on how to select salient edges in a more principled way. In the final latent image restoration step, we develop a simple method to remove artifacts and render better deblurred images. Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art text image deblurring methods. In addition, we show that the proposed method can be effectively applied to deblur low-illumination images.
Total Variation Blind Deconvolution: The Devil is in the Details
Authors: Paolo Favaro Daniele Perrone
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: In this paper we study the problem of blind deconvolution. Our analysis is based on the algorithm of Chan and Wong [2] which popularized the use of sparse gradient priors via total variation. We use this algorithm because many methods in the literature are essentially adaptations of this framework. Such algorithm is an iterative alternating energy minimization where at each step either the sharp image or the blur function are reconstructed. Recent work of Levin et al. [14] showed that any algorithm that tries to minimize that same energy would fail, as the desired solution has a higher energy than the no-blur solution, where the sharp image is the blurry input and the blur is a Dirac delta. However, experimentally one can observe that Chan and Wong's algorithm converges to the desired solution even when initialized with the no-blur one. We provide both analysis and experiments to resolve this paradoxical conundrum. We find that both claims are right. The key to understanding how this is possible lies in the details of Chan and Wong's implementation and in how seemingly harmless choices result in dramatic effects. Our analysis reveals that the delayed scaling (normalization) in the iterative step of the blur kernel is fundamental to the convergence of the algorithm. This then results in a procedure that eludes the no-blur solution, despite it being a global minimum of the original energy. We introduce an adaptation of this algorithm and show that, in spite of its extreme simplicity, it is very robust and achieves a performance comparable to the state of the art.
Single Image Super-resolution using Deformable Patches
Authors: Yanning Zhang, Alan L. Yuille Yu Zhu
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: We proposed a deformable patches based method for single image super-resolution. By the concept of deformation, a patch is not regarded as a fixed vector but a flexible deformation flow. Via deformable patches, the dictionary can cover more patterns that do not appear, thus becoming more expressive. We present the energy function with slow, smooth and flexible prior for deformation model. During example-based super-resolution, we develop the deformation similarity based on the minimized energy function for basic patch matching. For robustness, we utilize multiple deformed patches combination for the final reconstruction. Experiments evaluate the deformation effectiveness and super-resolution performance, showing that the deformable patches help improve the representation accuracy and perform better than the state-of-art methods.
Multi-Shot Imaging: Joint Alignment, Deblurring and Resolution-Enhancement
Authors: Lawrence Carin Haichao Zhang
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: The capture of multiple images is a simple way to increase the chance of capturing a good photo with a light-weight hand-held camera, for which the camera-shake blur is typically a nuisance problem. The naive approach of selecting the  single best  captured photo as output does not take full advantage of all the observations. Conventional multi-image blind deblurring methods can take all observations as input but  usually require  the multiple images are well aligned. However, the multiple blurry images captured in the presence of camera shake are rarely free from mis-alignment. Registering multiple blurry images is a challenging task due to the presence of blur while deblurring of multiple blurry images requires accurate alignment, leading to an intrinsically  coupled problem. In this paper, we propose  a blind multi-image restoration method which can achieve  joint alignment, non-uniform  deblurring, together with resolution enhancement from multiple low-quality images. Experiments on several real-world images with comparison to some previous methods validate the effectiveness of the proposed method.
CID: Combined Image Denoising in Spatial and Frequency Domains Using Web Images
Authors: Xiaoyan Sun, Jingyu Yang, Feng Wu Huanjing Yue
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: In this paper, we propose a novel two-step scheme to filter heavy noise from images with the assistance of retrieved Web images. There are two key technical contributions in our scheme. First, for every noisy image block, we build two three dimensional (3D) data cubes by using similar blocks in retrieved Web images and similar nonlocal blocks within the noisy image, respectively. To better use their correlations, we propose different denoising strategies. The denoising in the 3D cube built upon the retrieved images is performed as median filtering in the spatial domain, whereas the denoising in the other 3D cube is performed in the frequency domain. These two denoising results are then combined in the frequency domain to produce a denoising image. Second, to handle heavy noise, we further propose using the denoising image to improve image registration of the retrieved Web images, 3D cube building, and the estimation of filtering parameters in the frequency domain. Afterwards, the proposed denoising is performed on the noisy image again to generate the final denoising result. Our experimental results show that when the noise is high, the proposed scheme is better than BM3D by more than 2 dB in PSNR and the visual quality improvement is clear to see.
Multipoint Filtering with Local Polynomial Approximation and Range Guidance
Authors: Changming Sun, Tuan D. Pham Xiao Tan
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: This paper presents a novel guided image filtering method using multipoint local polynomial approximation (LPA) with range guidance. In our method, the LPA is extended from a pointwise model into a multipoint model for reliable  filtering and better preserving image spatial variation which usually contains the essential information in the input image. In addition, we develop a scheme with constant computational complexity (invariant to the size of filtering kernel) for generating a spatial adaptive support region around a point. By using the hybrid of the local polynomial model and color/intensity based range guidance, the proposed method not only preserves edges but also does a much better job in preserving spatial variation than existing popular filtering methods. Our method proves to be effective in a number of applications: depth image upsampling, joint image denoising, details enhancement, and image abstraction. Experimental results show that our method produces better results than state-of-the-art methods and it is also computationally efficient.
Decomposable Nonlocal Tensor Dictionary Learning for Multispectral Image Denoising
Authors: Deyu Meng, Zongben Xu, Chenqiang Gao, Yi Yang, Biao Zhang Yi Peng
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: As compared to the conventional RGB or gray-scale images, multispectral images (MSI) can deliver more faithful representation for real scenes, and enhance the performance of many computer vision tasks. In practice, however, an MSI is always corrupted by various noises. In this paper we propose an effective MSI denoising approach by combinatorially considering two intrinsic characteristics underlying an MSI: the nonlocal similarity over space and the global correlation across spectrum. In specific, by explicitly considering spatial self-similarity of an MSI we construct a nonlocal tensor dictionary learning model with a group-block-sparsity constraint, which makes similar full-band patches (FBP) share the same atoms from the spatial and spectral dictionaries. Furthermore, through exploiting spectral correlation of an MSI and assuming over-redundancy of dictionaries, the constrained nonlocal MSI dictionary learning model can be decomposed into a series of unconstrained low-rank tensor approximation problems, which can be readily solved by off-the-shelf higher order statistics. Experimental results show that our method outperforms all state-of-the-art MSI denoising methods under comprehensive quantitative performance measures.
Robust 3D Features for Matching between Distorted Range Scans Captured by Moving Systems
Authors: Bo Zheng, Takeshi Masuda, Katsushi Ikeuchi Xiangqi Huang
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Laser range sensors are often demanded to mount on a moving platform for achieving the good efficiency of 3D reconstruction. However, such moving systems often suffer from the difficulty of matching the distorted range scans. In this paper, we propose novel 3D features which can be robustly extracted and matched even for the distorted 3D surface captured by a moving system. Our feature extraction employs Morse theory to construct Morse functions which capture the critical points approximately invariant to the 3D surface distortion. Then for each critical point, we extract support regions with the maximally stable region defined by extremal region or disconnectivity. Our feature description is designed as two steps: 1) we normalize the detected local regions to canonical shapes for robust matching; 2) we encode each key point with multiple vectors at different Morse function values. In experiments, we demonstrate that the proposed 3D features achieve substantially better performance for distorted surface matching than the state-of-the-art methods.
Discriminative Blur Detection Features
Authors: Li Xu, Jiaya Jia Jianping Shi
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 (Main Conference Track), 2014

Abstract: Ubiquitous image blur brings out a practically important question – what are effective features to differentiate between blurred and unblurred image regions. We address it by studying a few blur feature representations in image gradient, Fourier domain, and data-driven local filters. Unlike previous methods, which are often based on restoration mechanisms, our features are constructed to enhance discriminative power and are adaptive to various blur scales in images. To avail evaluation, we build a new blur perception dataset containing thousands of images with labeled ground-truth. Our results are applied to several applications, including blur region segmentation, deblurring, and blur magnification.