pwc-archive/papers-with-abstracts huggingface dataset:
Image Reassembly Combining Deep Learning and Shortest Path Problem
Authors: Marie-Morgane Paumard, David Picard, Hedi Tabia
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deep Learning, Position

This paper addresses the problem of reassembling images from disjointed fragments. More specifically, given an unordered set of fragments, we aim at reassembling one or several possibly incomplete images. The main contributions of this work are: 1) several deep neural architectures to predict the relative position of image fragments that outperform the previous state of the art; 2) casting the reassembly problem into the shortest path in a graph problem for which we provide several construction algorithms depending on available information; 3) a new dataset of images taken from the Metropolitan Museum of Art (MET) dedicated to image reassembly for which we provide a clear setup and a strong baseline.
Coded Illumination and Imaging for Fluorescence Based Classification
Authors: Yuta Asano, Misaki Meguro, Chao Wang, Antony Lam, Yinqiang Zheng, Takahiro Okabe, Imari Sato
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Classification, General Classification, Material Classification

The quick detection of specific substances in objects such as produce items via non-destructive visual cues is vital to ensuring the quality and safety of consumer products. At the same time, it is well-known that the fluorescence excitation-emission characteristics of many organic objects can serve as a kind of ``fingerprint'' for detecting the presence of specific substances in classification tasks such as determining if something is safe to consume. However, conventional capture of the fluorescence excitation-emission matrix can take on the order of minutes and can only be done for point measurements. In this paper, we propose a coded illumination approach whereby light spectra are learned such that key visual fluorescent features can be easily seen for material classification. We show that under a single coded illuminant, we can capture one RGB image and perform pixel-level classifications of materials at high accuracy. This is demonstrated through effective classification of different types of honey and alcohol using real images.
Deep Kalman Filtering Network for Video Compression Artifact Reduction
Authors: Guo Lu, Wanli Ouyang, Dong Xu, Xiaoyun Zhang, Zhiyong Gao, Ming-Ting Sun
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Video Compression

When lossy video compression algorithms are applied, compression artifacts often appear in videos, making decoded videos unpleasant for human visual systems. In this paper, we model the video artifact reduction task as a Kalman filtering procedure and restore decoded frames through a deep Kalman filtering network. Different from the existing works using the noisy previous decoded frames as the temporal information in restoration problem, we utilize the less noisy previous restored frame and build a recursive filtering scheme based on Kalman model. This strategy can provide more accurate and consistent temporal information, which produces higher quality restoration. In addition, the strong prior information of the prediction residual is also exploited for restoration through a well designed neural network. These two components are combined under the Kalman framework and optimized through the deep Kalman filtering network. Our approach can well bridge the gap between the model-based methods and learning-based methods by integrating the recursive nature of the Kalman model and highly non-linear transformation ability of deep neural network. Experimental results on the benchmark dataset demonstrate the effectiveness of our proposed method.
A Deeply-initialized Coarse-to-fine Ensemble of Regression Trees for Face Alignment
Authors: Roberto Valle, Jose M. Buenaposada, Antonio Valdes, Luis Baumela
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Face Alignment, Face Model, Facial Landmark Detection, regression

In this paper we present DCFE, a real-time facial landmark regression method based on a coarse-to-fine Ensemble of Regression Trees (ERT). We use a simple Convolutional Neural Network (CNN) to generate probability maps of landmarks location. These are further refined with the ERT regressor, which is initialized by fitting a 3D face model to the landmark maps. The coarse-to-fine structure of the ERT lets us address the combinatorial explosion of parts deformation. With the 3D model we also tackle other key problems such as robust regressor initialization, self occlusions, and simultaneous frontal and profile face analysis. In the experiments DCFE achieves the best reported result in AFLW, COFW, and 300W private and common public data sets.
BusterNet: Detecting Copy-Move Image Forgery with Source/Target Localization
Authors: Yue Wu, Wael Abd-Almageed, Prem Natarajan
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

We introduce a novel deep neural architecture for image copy-move forgery detection (CMFD), code-named BusterNet. Unlike previous eorts, BusterNet is a pure, end-to-end trainable, deep neural network solution. It features a two-branch architecture followed by a fu- sion module. The two branches localize potential manipulation regions (by looking for visual artifacts) and copy-move regions (by assessing vi- sual similarities), respectively. To the best of our knowledge, this is the rst CMFD algorithm with discernibility to localize source/target re- gions.We also propose simple schemes for synthesizing large-scale CMFD samples using out-of-domain datasets, and stage-wise strategies for eec- tive BusterNet training. Our extensive studies demonstrate that Buster- Net outperforms state-of-the-art copy-move detection algorithms by a large margin on the two publicly available datasets, CASIA and CoMo- FoD, and that it is robust against various known attacks.
Task-Aware Image Downscaling
Authors: Heewon Kim, Myungsub Choi, Bee Lim, Kyoung Mu Lee
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Colorization, Image Colorization, Super-Resolution

Image downscaling is one of the most classical problems in computer vision that aims to preserve the visual appearance of the original image when it is resized to a smaller scale. Upscaling a small image back to its original size is a difficult, ill-posed problem due to information loss that arises in the downscaling process. In this paper, we present a novel operation called task-aware image downscaling to support an upscaling task. We propose an auto-encoder-based framework that enables joint learning of the downscaling network and the upscaling network to maximize the restoration performance. Our framework is efficient, and it can be generalized to handle an arbitrary image resizing operation. Experimental results show that our task-aware downscaled image, greatly improved the super-resolution performance of the previous state-of-the-art. In addition, realistic images can be recovered by recursively applying our scaling model up to an extreme scaling factor of x128. We validate our model's generalization capability by applying it to the task of image colorization.
Multi-scale Residual Network for Image Super-Resolution
Authors: Juncheng Li, Faming Fang, Kangfu Mei, Guixu Zhang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Image Super-Resolution, Super-Resolution

Recent studies have shown that deep neural networks can significantly improve the quality of single-image super-resolution. Current researches tend to use deeper convolutional neural networks to enhance performance. However, blindly increasing the depth of the network cannot ameliorate the network effectively. Worse still, with the depth of the network increases, more problems occurred in the training process and more training tricks are needed. In this paper, we propose a novel multi-scale residual network (MSRN) to fully exploit the image features, which outperform most of the state-of-the-art methods. Based on the residual block, we introduce convolution kernels of different sizes to adaptively detect the image features in different scales. Meanwhile, we let these features interact with each other to get the most efficacious image information, we call this structure Multi-scale Residual Block (MSRB). Furthermore, the outputs of each MSRB are used as the hierarchical features for global feature fusion. Finally, all these features are sent to the reconstruction module for recovering the high-quality image.
Efficient Global Point Cloud Registration by Matching Rotation Invariant Features Through Translation Search
Authors: Yinlong Liu, Chen Wang, Zhijian Song, Manning Wang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Point Cloud Registration, Translation

Three-dimensional rigid point cloud registration has many applications in computer vision and robotics. Local methods tend to fail, causing global methods to be needed, when the relative transformation is large or the overlap ratio is small. Most existing global methods utilize BnB optimization over the 6D parameter space of SE(3). Such methods are usually very slow because the time complexity of BnB optimization is exponential in the dimensionality of the parameter space. In this paper, we decouple the optimization of translation and rotation, and we propose a fast BnB algorithm to globally optimize the 3D translation parameter first. The optimal rotation is then calculated by utilizing the global optimal translation found by the BnB algorithm. The separate optimization of translation and rotation is realized by using a newly proposed rotation invariant feature. Experiments on challenging data sets demonstrate that the proposed method outperforms state-of-the-art global methods in terms of both speed and accuracy.
DeepVS: A Deep Learning Based Video Saliency Prediction Approach
Authors: Lai Jiang, Mai Xu, Tie Liu, Minglang Qiao, Zulin Wang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deep Learning, Prediction, Saliency Prediction, Video Saliency Detection, Video Saliency Prediction

In this paper, we propose a novel deep learning based video saliency prediction method, named DeepVS. Specifically, we establish a large-scale eye-tracking database of videos (LEDOV), which includes 32 subjects' fixations on 538 videos. We find from LEDOV that human attention is more likely to be attracted by objects, particularly the moving objects or the moving parts of objects. Hence, an object-to-motion convolutional neural network (OM-CNN) is developed to predict the intra-frame saliency for DeepVS, which is composed of the objectness and motion subnets. In OM-CNN, cross-net mask and hierarchical feature normalization are proposed to combine the spatial features of the objectness subnet and the temporal features of the motion subnet. We further find from our database that there exists a temporal correlation of human attention with a smooth saliency transition across video frames. We thus propose saliency-structured convolutional long short-term memory (SS-ConvLSTM) network, using the extracted features from OM-CNN as the input. Consequently, the inter-frame saliency maps of a video can be generated, which consider both structured output with center-bias and cross-frame transitions of human attention maps. Finally, the experimental results show that DeepVS advances the state-of-the-art in video saliency prediction.
Proxy Clouds for Live RGB-D Stream Processing and Consolidation
Authors: Adrien Kaiser, Jose Alonso Ybanez Zepeda, Tamy Boubekeur
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

We propose a new multiplanar superstructure for unified real-time processing of RGB-D data. Modern RGB-D sensors are widely used for indoor 3D capture, with applications ranging from modeling to robotics, through augmented reality. Nevertheless, their use is limited by their low resolution, with frames often corrupted with noise, missing data and temporal inconsistencies. Our approach, named Proxy Clouds, consists in generating and updating through time a single set of compact local statistics parameterized over detected planar proxies, which are fed from raw RGB-D data. Proxy Clouds provide several processing primitives, which improve the quality of the RGB-D stream on-the-fly or lighten further operations. Experimental results confirm that our light weight analysis framework copes well with embedded execution as well as moderate memory and computational capabilities compared to state-of-the-art methods. Processing of RGB-D data with Proxy Clouds includes noise and temporal flickering removal, hole filling and resampling. As a substitute of the observed scene, our proxy cloud can additionally be applied to compression and scene reconstruction. We present experiments performed with our framework in indoor scenes of different natures within a recent open RGB-D dataset.
U-PC: Unsupervised Planogram Compliance
Authors: Archan Ray, Nishant Kumar, Avishek Shaw, Dipti Prasad Mukherjee
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Marketing

We present an end-to-end solution for recognizing merchandise displayed in the shelves of a supermarket. Given images of individual products, which are taken under ideal illumination for product marketing, the challenge is to find these products automatically in the images of the shelves. Note that the images of shelves are taken using hand-held camera under store level illumination. We provide a two-layer hypotheses generation and verification model. In the first layer, the model predicts a set of candidate merchandise at a specific location of the shelf while in the second layer, the hypothesis is verified by a novel graph theoretic approach. The performance of the proposed approach on two publicly available datasets is better than the competing approaches by at least 10%.
Efficient Dense Point Cloud Object Reconstruction using Deformation Vector Fields
Authors: Kejie Li, Trung Pham, Huangying Zhan, Ian Reid
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Object Reconstruction, Object, Object Reconstruction

Most existing CNN-based methods for single-view 3D object reconstruction represent a 3D object as either a 3D voxel occupancy grid or multiple depth-mask image pairs. However, these representations are inefficient since empty voxels or background pixels are wasteful. We propose a novel approach that addresses this limitation by replacing masks with ''deformation-fields''. Given a single image at an arbitrary viewpoint, a CNN predicts multiple surfaces, each in a canonical location relative to the object. Each surface comprises a depth-map and corresponding deformation-field that ensures every pixel-depth pair in the depth-map lies on the object surface. These surfaces are then fused to form the full 3D shape. During training, we use a combination of per-view and multi-view losses. The novel multi-view loss encourages the 3D points back-projected from a particular view to be consistent across views. Extensive experiments demonstrate the efficiency and efficacy of our method on single-view 3D object reconstruction.
Recurrent Tubelet Proposal and Recognition Networks for Action Detection
Authors: Dong Li, Zhaofan Qiu, Qi Dai, Ting Yao, Tao Mei
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Action Detection, Region Proposal

Detecting actions in videos is a challenging task as video is an information intensive media with complex variations. Existing approaches predominantly generate action proposals for each individual frame or fixed-length clip independently, while overlooking temporal context across them. Such temporal contextual relations are vital for action detection as an action is by nature a sequence of movements. This motivates us to leverage the localized action proposals in previous frames when determining action regions in the current one. Specifically, we present a novel deep architecture called Recurrent Tubelet Proposal and Recognition (RTPR) networks to incorporate temporal context for action detection. The proposed RTPR consists of two correlated networks, i.e., Recurrent Tubelet Proposal (RTP) networks and Recurrent Tubelet Recognition (RTR) networks. The RTP initializes action proposals of the start frame through a Region Proposal Network and then estimates the movements of proposals in next frame in a recurrent manner. The action proposals of different frames are linked to form the tubelet proposals. The RTR capitalizes on a multi-channel architecture, where in each channel, a tubelet proposal is fed into a CNN plus LSTM to recurrently recognize action in the tubelet. We conduct extensive experiments on four benchmark datasets and demonstrate superior results over state-of-the-art methods. More remarkably, we obtain mAP of 98.6%, 81.3%, 77.9% and 22.3% with gains of 2.9%, 4.3%, 0.7% and 3.9% over the best competitors on UCF-Sports, J-HMDB, UCF-101 and AVA, respectively.
Domain transfer through deep activation matching
Authors: Haoshuo Huang, Qi-Xing Huang, Philipp Krahenbuhl
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Domain Adaptation, Generative Adversarial Network, image-classification, Image Classification, Segmentation, Semantic Segmentation, Unsupervised Domain Adaptation

We introduce a layer-wise unsupervised domain adaptation approach for the task of semantic segmentation. Instead of merely matching the output distributions of the source and target domains, our approach aligns the distributions of activations of intermediate layers. This scheme exhibits two key advantages. First, matching across intermediate layers introduces more constraints for training the network in the target domain, making the optimization problem better conditioned. Second, the matched activations at each layer provide similar inputs to the next layer for both training and adaptation, and thus alleviate covariate shift. We use a Generative Adversarial Network (or GAN) to align activation distributions. Experimental results show that our approach achieves state-of-the-art results on a variety of popular domain adaptation tasks, including (1) from GTA to Cityscapes for semantic segmentation, (2) from SYNTHIA to Cityscapes for semantic segmentation,  and (3) adaptations on USPS and MNIST for image classification.
Recovering Accurate 3D Human Pose in The Wild Using IMUs and a Moving Camera
Authors: Timo von Marcard, Roberto Henschel, Michael J. Black, Bodo Rosenhahn, Gerard Pons-Moll
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Pose Estimation, Pose Estimation

In this work, we propose a method that combines a single hand-held camera and a set of Inertial Measurement Units (IMUs) attached at the body limbs to estimate accurate 3D poses in the wild. This poses many new challenges: the moving camera, heading drift, cluttered background, occlusions and many people visible in the video. We associate 2D pose detections in each image to the corresponding IMU-equipped persons by solving a novel graph based optimization problem that forces 3D to 2D coherency within a frame and across long range frames. Given associations, we jointly optimize the pose of a statistical body model, the camera pose and heading drift using a continuous optimization framework. We validated our method on the TotalCapture dataset, which provides video and IMU synchronized with ground truth. We obtain an accuracy of 26mm, which makes it accurate enough to serve as a benchmark for image-based 3D pose estimation in the wild. Using our method, we recorded 3D Poses in the Wild (3DPW ), a new dataset consisting of more than 51; 000 frames with accurate 3D pose in challenging sequences, including walking in the city, going up-stairs, having coffee or taking the bus. We make the reconstructed 3D poses, video, IMU and 3D models available for research purposes at http://virtualhumans.mpi-inf.mpg.de/3DPW.
Beyond local reasoning for stereo confidence estimation with deep learning
Authors: Fabio Tosi, Matteo Poggi, Antonio Benincasa, Stefano Mattoccia
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deep Learning

Confidence measures for stereo gained popularity in recent years due to their improved capability to detect outliers and the increasing number of applications exploiting these cues. In this field, convolutional neural networks achieved top-performance compared to other known techniques in the literature by processing local information to tell disparity assignments from outliers. Despite this outstanding achievements, all approaches rely on clues extracted with small receptive fields thus ignoring most of the overall image content. Therefore, in this paper, we propose to exploit nearby and farther clues available from image and disparity domains to obtain a more accurate confidence estimation. While local information is very effective for detecting high frequency patterns, it lacks insights from farther regions in the scene. On the other hand, enlarging the receptive field allows to include clues from farther regions but produces smoother uncertainty estimation, not particularly accurate when dealing with high frequency patterns. For these reasons, we propose in this paper a multi-stage cascaded network to combine the best of the two worlds. Extensive experiments on three datasets using three popular stereo algorithms prove that the proposed framework outperforms state-of-the-art confidence estimation techniques.
Multimodal image alignment through a multiscale chain of neural networks with application to remote sensing
Authors: Armand Zampieri, Guillaume Charpiat, Nicolas Girard, Yuliya Tarabalka
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

We tackle here the problem of multimodal image non-rigid registration, which is of prime importance in remote sensing and medical imaging. The difficulties encountered by classical registration approaches include feature design and slow optimization by gradient descent. By analyzing these methods, we note the significance of the notion of scale. We design easy-to-train, fully-convolutional neural networks able to learn scale-specific features. Once chained appropriately, they perform global registration in linear time, getting rid of gradient descent schemes by predicting directly the deformation. We show their performance in terms of quality and speed through various tasks of remote sensing multimodal image alignment. In particular, we are able to register correctly cadastral maps of buildings as well as road polylines onto RGB images, and outperform current keypoint matching methods.
Start, Follow, Read: End-to-End Full-Page Handwriting Recognition
Authors: Curtis Wigington, Chris Tensmeyer, Brian Davis, William Barrett, Brian Price, Scott Cohen
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Handwriting Recognition, Handwritten Text Recognition, Position, Region Proposal, Segmentation, Text Detection

Despite decades of research, offline handwriting recognition (HWR) of degraded historical documents remains a challenging problem, which if solved could greatly improve the searchability of online cultural heritage archives. HWR models are often limited by the accuracy of the preceding steps of text detection and segmentation. Motivated by this, we present a deep learning model that jointly learns text detection, segmentation, and recognition using mostly images without detection or segmentation annotations. Our Start, Follow, Read (SFR) model is composed of a Region Proposal Network to find the start position of text lines, a novel line follower network that incrementally follows and preprocesses lines of (perhaps curved) text into dewarped images suitable for recognition by a CNN-LSTM network. SFR exceeds the performance of the winner of the ICDAR2017 handwriting recognition competition, even when not using the provided competition region annotations.