pwc-archive/papers-with-abstracts huggingface dataset:
Spectral Reconstruction From Dispersive Blur: A Novel Light Efficient Spectral Imager
Authors: Yuanyuan Zhao,  Xuemei Hu,  Hui Guo,  Zhan Ma,  Tao Yue,  Xun Cao
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Spectral Reconstruction

Developing high light efficiency imaging techniques to retrieve high dimensional optical signal is a long-term goal in computational photography. Multispectral imaging, which captures images of different wavelengths and boosting the abilities for revealing scene properties, has developed rapidly in the last few decades. From scanning method to snapshot imaging, the limit of light collection efficiency is kept being pushed which enables wider applications especially under the light-starved scenes. In this work, we propose a novel multispectral imaging technique, that could capture the multispectral images with a high light efficiency. Through investigating the dispersive blur caused by spectral dispersers and introducing the difference of blur (DoB) constraints, we propose a basic theory for capturing multispectral information from a single dispersive-blurred image and an additional spectrum of an arbitrary point in the scene. Based on the theory, we design a prototype system and develop an optimization algorithm to realize snapshot multispectral imaging. The effectiveness of the proposed method is verified on both the synthetic data and real captured images.
Quasi-Unsupervised Color Constancy
Authors: Simone Bianco,  Claudio Cusano
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Color Constancy

We present here a method for computational color constancy in which a deep convolutional neural network is trained to detect achromatic pixels in color images after they have been converted to grayscale. The method does not require any information about the illuminant in the scene and relies on the weak assumption, fulfilled by almost all images available on the web, that training images have been approximately balanced. Because of this requirement we define our method as quasi-unsupervised. After training, unbalanced images can be processed thanks to the preliminary conversion to grayscale of the input to the neural network.  The results of an extensive experimentation demonstrate that the proposed method is able to outperform the other unsupervised methods in the state of the art being, at the same time, flexible enough to be supervisedly fine-tuned to reach performance comparable with those of the best supervised methods.
Deep Defocus Map Estimation Using Domain Adaptation
Authors: Junyong Lee,  Sungkil Lee,  Sunghyun Cho,  Seungyong Lee
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Deblurring, Defocus Estimation, Domain Adaptation

In this paper, we propose the first end-to-end convolutional neural network (CNN) architecture, Defocus Map Estimation Network (DMENet), for spatially varying defocus map estimation. To train the network, we produce a novel depth-of-field (DOF) dataset, SYNDOF, where each image is synthetically blurred with a ground-truth depth map. Due to the synthetic nature of SYNDOF, the feature characteristics of images in SYNDOF can differ from those of real defocused photos. To address this gap, we use domain adaptation that transfers the features of real defocused photos into those of synthetically blurred ones. Our DMENet consists of four subnetworks: blur estimation, domain adaptation, content preservation, and sharpness calibration networks. The subnetworks are connected to each other and jointly trained with their corresponding supervisions in an end-to-end manner. Our method is evaluated on publicly available blur detection and blur estimation datasets and the results show the state-of-the-art performance.In this paper, we propose the first end-to-end convolutional neural network (CNN) architecture, Defocus Map Estimation Network (DMENet), for spatially varying defocus map estimation. To train the network, we produce a novel depth-of-field (DOF) dataset, SYNDOF, where each image is synthetically blurred with a ground-truth depth map. Due to the synthetic nature of SYNDOF, the feature characteristics of images in SYNDOF can differ from those of real defocused photos. To address this gap, we use domain adaptation that transfers the features of real defocused photos into those of synthetically blurred ones. Our DMENet consists of four subnetworks: blur estimation, domain adaptation, content preservation, and sharpness calibration networks. The subnetworks are connected to each other and jointly trained with their corresponding supervisions in an end-to-end manner. Our method is evaluated on publicly available blur detection and blur estimation datasets and the results show the state-of-the-art performance.
Using Unknown Occluders to Recover Hidden Scenes
Authors: Adam B. Yedidia,  Manel Baradad,  Christos Thrampoulidis,  William T. Freeman,  Gregory W. Wornell
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

We consider the challenging problem of inferring a hidden moving scene from faint shadows cast on a diffuse surface. Recent work in passive non-line-of-sight (NLoS) imaging has shown that the presence of occluding objects in between the scene and the diffuse surface significantly improves the conditioning of the problem. However, that work assumes that the shape of the occluder is known a priori. In this paper, we relax this often impractical assumption, extending the range of applications for passive occluder-based NLoS imaging systems.  We formulate the task of jointly recovering the unknown scene and unknown occluder as a blind deconvolution problem, for which we propose a simple but effective two-step algorithm. At the first step, the algorithm exploits motion in the scene in order to obtain an estimate of the occluder. In particular, it exploits the fact that motion in realistic scenes is typically sparse. The second step is more standard: using regularization, we deconvolve by the occluder estimate to solve for the hidden scene.  We demonstrate the effectiveness of our method with simulations and experiments in a variety of settings.
Scalable Convolutional Neural Network for Image Compressed Sensing
Authors: Wuzhen Shi,  Feng Jiang,  Shaohui Liu,  Debin Zhao
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: compressed sensing, Decoder, Deep Learning, Image Compressed Sensing

Recently, deep learning based image Compressed Sensing (CS) methods have been proposed and demonstrated superior reconstruction quality with low computational complexity. However, the existing deep learning based image CS methods need to train different models for different sampling ratios, which increases the complexity of the encoder and decoder. In this paper, we propose a scalable convolutional neural network (dubbed SCSNet) to achieve scalable sampling and scalable reconstruction with only one model. Specifically, SCSNet provides both coarse and fine granular scalability. For coarse granular scalability, SCSNet is designed as a single sampling matrix plus a hierarchical reconstruction network that contains a base layer plus multiple enhancement layers. The base layer provides the basic reconstruction quality, while the enhancement layers reference the lower reconstruction layers and gradually improve the reconstruction quality. For fine granular scalability, SCSNet achieves sampling and reconstruction at any sampling ratio by using a greedy method to select the measurement bases. Compared with the existing deep learning based image CS methods, SCSNet achieves scalable sampling and quality scalable reconstruction at any sampling ratio with only one model. Experimental results demonstrate that SCSNet has the state-of-the-art performance while maintaining a comparable running speed with the existing deep learning based image CS methods.
Event Cameras, Contrast Maximization and Reward Functions: An Analysis
Authors: Timo Stoffregen,  Lindsay Kleeman
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Event-based vision, Optical Flow Estimation, Semantic Segmentation

Event cameras asynchronously report timestamped changes in pixel intensity and offer advantages over conventional raster scan cameras in terms of low-latency, low redundancy sensing and high dynamic range. In recent years, much of research in event based vision has been focused on performing tasks such as optic flow estimation, moving object segmentation, feature tracking, camera rotation estimation and more, through contrast maximization. In contrast maximization, events are warped along motion trajectories whose parameters depend on the quantity being estimated, to some time t_ref. The parameters are then scored by some reward function of the accumulated events at t_ref. The versatility of this approach has lead to a flurry of research in recent years, but no in-depth study of the reward chosen during optimization has yet been made. In this work we examine the choice of reward used in contrast maximization, propose a classification of different rewards and show how a reward can be constructed that is more robust to noise and aperture uncertainty. We validate our work experimentally by predicting optical flow and comparing to ground-truth data.
Convolutional Neural Networks Can Be Deceived by Visual Illusions
Authors: Alexander Gomez-Villa,  Adrian Martin,  Javier Vazquez-Corral,  Marcelo Bertalmio
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Color Constancy, Deblurring, Denoising, Image Deblurring, Image Denoising

Visual illusions teach us that what we see is not always what is represented in the physical world. Their special nature make them a fascinating tool to test and validate any new vision model proposed. In general, current vision models are based on the concatenation of linear and non-linear operations. The similarity of this structure with the operations present in Convolutional Neural Networks (CNNs) has motivated us to study if CNNs trained for low-level visual tasks are deceived by visual illusions. In particular, we show that CNNs trained for image denoising, image deblurring, and computational color constancy are able to replicate the human response to visual illusions, and that the extent of this replication varies with respect to variation in architecture  and spatial pattern size. These results suggest that in order to obtain CNNs that better replicate human behaviour, we may need to start aiming for them to better replicate visual illusions.
PDE Acceleration for Active Contours
Authors: Anthony Yezzi,  Ganesh Sundaramoorthi,  Minas Benyamin
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: parameter estimation

Following the seminal work of Nesterov, accelerated optimization   methods have been used to powerfully boost the performance of   first-order, gradient-based parameter estimation in scenarios where   second-order optimization strategies are either inapplicable or   impractical.  Accelerated gradient descent converges faster and   performs a more robust local search of the parameter space by   initially overshooting then oscillating back into minimizers which   have a basis of attraction large enough to contain the overshoot.   Recent work has demonstrated how a broad class of accelerated   schemes can be cast in a variational framework leading to continuum   limit ODE's.  We extend their formulation to the PDE framework,   specifically for the infinite dimensional manifold of continuous   curves, to introduce acceleration, and its added robustness, into   the broad range of PDE based active contours.
Dichromatic Model Based Temporal Color Constancy for AC Light Sources
Authors: Jun-Sang Yoo,  Jong-Ok Kim
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Color Constancy

Existing dichromatic color constancy approach commonly requires a number of spatial pixels which have high specularity. In this paper, we propose a novel approach to estimate the illuminant chromaticity of AC light source using high-speed camera. We found that the temporal observations of an image pixel at a fixed location distribute on an identical dichromatic plane. Instead of spatial pixels with high specularity, multiple temporal samples of a pixel are exploited to determine AC pixels for dichromatic plane estimation, whose pixel intensity is sinusoidally  varying well. A dichromatic plane is calculated per each AC pixel, and illuminant chromaticity is determined by the intersection of dichromatic planes. From multiple dichromatic planes, an optimal illuminant is estimated with a novel MAP framework. It is shown that the proposed method outperforms both existing dichromatic based methods and temporal color constancy methods, irrespective of the amount of specularity.
Skin-Based Identification From Multispectral Image Data Using CNNs
Authors: Takeshi Uemori,  Atsushi Ito,  Yusuke Moriuchi,  Alexander Gatto,  Jun Murayama
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: User Identification

User identification from hand images only is still a challenging task. In this paper, we propose a new biometric identification system based solely on a skin patch from a multispectral image. The system is utilizing a novel modified 3D CNN architecture which is taking advantage of multispectral data. We demonstrate the application of our system for the example of human identification from multispectral images of hands. To the best of our knowledge, this paper is the first to describe a pose-invariant and robust to overlapping real-time human identification system using hands. Additionally, we provide a framework to optimize the required spectral bands for the given spatial resolution limitations.
PIEs: Pose Invariant Embeddings
Authors: Chih-Hui Ho,  Pedro Morgado,  Amir Persekian,  Nuno Vasconcelos
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: General Classification, Retrieval

The role of pose invariance in image recognition and retrieval is studied. A taxonomic classification of embeddings, according to their level of invariance, is introduced and used to clarify connections between existing embeddings, identify missing approaches, and propose invariant generalizations. This leads to a new family of pose invariant embeddings (PIEs), derived from existing approaches by a combination of two models, which follow from the interpretation of CNNs as estimators of class posterior probabilities: a view-to-object model and an object-to-class model. The new pose-invariant models are shown to have interesting properties, both theoretically and through experiments, where they outperform existing multiview approaches. Most notably, they achieve good performance for both 1) classification and retrieval, and 2) single and multiview inference. These are important properties for the design of real vision systems, where universal embeddings are preferable to task specific ones, and multiple images are usually not available at inference time. Finally, a new multiview dataset of real objects, imaged in the wild against complex backgrounds, is introduced. We believe that this is a much needed complement to the synthetic datasets in wide use and will contribute to the advancement of multiview recognition and retrieval.
A Simple Baseline for Audio-Visual Scene-Aware Dialog
Authors: Idan Schwartz,  Alexander G. Schwing,  Tamir Hazan
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Scene-Aware Dialogue

The recently proposed audio-visual scene-aware dialog task paves the way to a more data-driven way of learning virtual assistants, smart speakers and car navigation systems.  However, very little is known to date about how to effectively extract meaningful information from a plethora of sensors that pound the computational engine of those devices. Therefore, in this paper, we provide and carefully analyze a simple baseline for audio-visual scene-aware dialog which is trained end-to-end. Our method differentiates in a data-driven manner useful signals from distracting ones using an attention mechanism. We evaluate the proposed approach on the recently introduced and challenging audio-visual scene-aware dataset, and demonstrate the key features that permit to outperform the current state-of-the-art by more than 20% on CIDEr.
Cascaded Generative and Discriminative Learning for Microcalcification Detection in Breast Mammograms
Authors: Fandong Zhang,  Ling Luo,  Xinwei Sun,  Zhen Zhou,  Xiuli Li,  Yizhou Yu,  Yizhou Wang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Decoder, Image Reconstruction

Accurate microcalcification (mC) detection is of great importance due to its high proportion in early breast cancers. Most of the previous mC detection methods belong to discriminative models, where classifiers are exploited to distinguish mCs from other backgrounds. However, it is still challenging for these methods to tell the mCs from amounts of normal tissues because they are too tiny (at most 14 pixels). Generative methods can precisely model the normal tissues and regard the abnormal ones as outliers, while they fail to further distinguish the mCs from other anomalies, i.e. vessel calcifications. In this paper, we propose a hybrid approach by taking advantages of both generative and discriminative models. Firstly, a generative model named Anomaly Separation Network (ASN) is used to generate candidate mCs. ASN contains two major components. A deep convolutional encoder-decoder network is built to learn the image reconstruction mapping and a t-test loss function is designed to separate the distributions of the reconstruction residuals of mCs from normal tissues. Secondly, a discriminative model is cascaded to tell the mCs from the false positives. Finally, to verify the effectiveness of our method, we conduct experiments on both public and in-house datasets, which demonstrates that our approach outperforms previous state-of-the-art methods.
In Defense of Pre-Trained ImageNet Architectures for Real-Time Semantic Segmentation of Road-Driving Images
Authors: Marin Orsic,  Ivan Kreso,  Petra Bevandic,  Sinisa Segvic
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Real-Time Semantic Segmentation, Semantic Segmentation

Recent success of semantic segmentation approaches on demanding road driving datasets has spurred interest in many related application fields. Many of these applications involve real-time prediction on mobile platforms such as cars, drones and various kinds of robots. Real-time setup is challenging due to extraordinary computational complexity involved. Many previous works address the challenge with custom lightweight architectures which decrease computational complexity by reducing depth, width and layer capacity with respect to general purpose architectures. We propose an alternative approach which achieves a significantly better performance across a wide range of computing budgets. First, we rely on a light-weight general purpose architecture as the main recognition engine. Then, we leverage light-weight upsampling with lateral connections as the most cost-effective solution to restore the prediction resolution. Finally, we propose to enlarge the receptive field by fusing shared features at multiple resolutions in a novel fashion. Experiments on several road driving datasets show a substantial advantage of the proposed approach, either with ImageNet pre-trained parameters or when we learn from scratch. Our Cityscapes test submission entitled SwiftNetRN-18 delivers 75.5% MIoU and achieves 39.9 Hz on 1024x2048 images on GTX1080Ti.
Multiview 2D/3D Rigid Registration via a Point-Of-Interest Network for Tracking and Triangulation
Authors: Haofu Liao,  Wei-An Lin,  Jiarui Zhang,  Jingdan Zhang,  Jiebo Luo,  S. Kevin Zhou
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

We propose to tackle the problem of multiview 2D/3D rigid registration for intervention via a Point-Of-Interest Network for Tracking and Triangulation (POINT^2). POINT^2 learns to establish 2D point-to-point correspondences between the pre- and intra-intervention images by tracking a set of random POIs. The 3D pose of the pre-intervention volume is then estimated through a triangulation layer. In POINT^2, the unified framework of the POI tracker and the triangulation layer enables learning informative 2D features and estimating 3D pose jointly. In contrast to existing approaches, POINT^2 only requires a single forward-pass to achieve a reliable 2D/3D registration. As the POI tracker is shift-invariant, POINT^2 is more robust to the initial pose of the 3D pre-intervention image. Extensive experiments on a large-scale clinical cone-beam CT (CBCT) dataset show that the proposed POINT^2 method outperforms the existing learning-based method in terms of accuracy, robustness and running time. Furthermore, when used as an initial pose estimator, our method also improves the robustness and speed of the state-of-the-art optimization-based approaches by ten folds.
Context-Aware Spatio-Recurrent Curvilinear Structure Segmentation
Authors: Feigege Wang,  Yue Gu,  Wenxi Liu,  Yuanlong Yu,  Shengfeng He,  Jia Pan
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

Curvilinear structures are frequently observed in various images in different forms, such as blood vessels or neuronal boundaries in biomedical images. In this paper, we propose a novel curvilinear structure segmentation approach using context-aware spatio-recurrent networks. Instead of directly segmenting the whole image or densely segmenting fixed-sized local patches, our method recurrently samples patches with varied scales from the target image with learned policy and processes them locally, which is similar to the behavior of changing retinal fixations in the human visual system and it is beneficial for capturing the multi-scale or hierarchical modality of the complex curvilinear structures.  In specific, the policy of choosing local patches is attentively learned based on the contextual information of the image and the historical sampling experience.   In this way, with more patches sampled and refined, the segmentation of the whole image can be progressively improved. To validate our approach, comparison experiments on different types of image data are conducted and the sampling procedures for exemplar images are illustrated. We demonstrate that our method achieves the state-of-the-art performance in public datasets.
An Alternative Deep Feature Approach to Line Level Keyword Spotting
Authors: George Retsinas,  Georgios Louloudis,  Nikolaos Stamatopoulos,  Giorgos Sfikas,  Basilis Gatos
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Handwritten Text Recognition, HTR, Keyword Spotting

Keyword  spotting  (KWS)  is  defined  as  the  problem  of detecting  all  instances  of  a  given  word,  provided  by  the user  either  as  a  query  word  image (Query-by-Example, QbE) or a query word string (Query-by-String, QbS) in a body  of  digitized  documents.   Keyword  detection  is  typically preceded by a preprocessing step where the text is segmented into text lines (line-level KWS). Methods following this  paradigm  are  monopolized  by  test-time  computationally  expensive  handwritten  text  recognition  (HTR)-based approaches; furthermore, they typically cannot handle image  queries  (QbE). In  this  work,  we  propose  a  time  and storage-efficient, deep feature-based approach that enables both the image and textual search options. Three distinct components, all modeled as neural networks, are combined: normalization, feature extraction and representation of image  and  textual  input  into  a  common  space. These  components, even if designed on word level image representations, collaborate in order to achieve an efficient line level keyword spotting system. The experimental results indicate that the proposed system is on par with state-of-the-art KWS methods.