pwc-archive/papers-with-abstracts huggingface dataset:
Dividing and Aggregating Network for Multi-view Action Recognition
Authors: Dongang Wang, Wanli Ouyang, Wen Li, Dong Xu
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Action Recognition, Temporal Action Localization

In this paper, we propose a new Dividing and Aggregating Network (DA-Net) for multi-view action recognition. In our DA-Net, we learn view-independent representations shared by all views at lower layers, while we learn one view-specific representation for each view at higher layers. We then train view-specific action classifiers based on the view-specific representation for each view and a view classifier based on the shared representation at lower layers. The view classifier is used to  predict how likely each video belongs to each view. Finally, the predicted view probabilities from multiple views are used as the weights when fusing the prediction scores of view-specific action classifiers. We also propose a new approach based on the conditional random field (CRF) formulation to pass message among view-specific representations from different branches to help each other. Comprehensive experiments on two benchmark datasets clearly demonstrate the effectiveness of our proposed DA-Net for multi-view action recognition.
Deblurring Natural Image Using Super-Gaussian Fields
Authors: Yuhang Liu, Wenyong Dong, Dong Gong, Lei Zhang, Qinfeng Shi
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Blind Image Deblurring, Deblurring, Image Deblurring

Blind image deblurring is a challenging problem due to its ill-posed nature, of which the success is closely related to a proper image prior. Although a large number of sparsity-based priors, such as the sparse gradient prior, have been successfully applied for blind image deblurring, they inherently suffer from several drawbacks, limiting their applications. Existing sparsity-based priors are usually rooted in modeling the response of images to some specific filters (e.g., image gradients), which are insufficient to capture the complicated image structures. Moreover, the traditional sparse priors or regularizations model the filter response (e.g., image gradients) independently and thus fail to depict the long range correlation among them. To address the above issues, we present a novel image prior for image deblurring based on a Super-Gaussian field model with adaptive structures. Instead of modeling the response of the fixed short-term filters, the proposed Super-Gaussian fields capture the complicated structures in natural images by integrating potentials on all cliques (e.g., centring at each pixel) into a joint probabilistic distribution. Considering that the fixed filters in different scales are impractical for the coarse-to-fine framework of image deblurring, we define each potential function as a super-Gaussian distribution. Through this definition, the partition function, the curse for traditional MRFs, can be theoretically ignored, and all model parameters of the proposed Super-Gaussian fields can be data-adaptively learned and inferred from the blurred observation with a variational framework. Extensive experiments on both blind deblurring and non-blind deblurring demonstrate the effectiveness of the proposed method.
Real-time 'Actor-Critic' Tracking
Authors: Boyu Chen, Dong Wang, Peixia Li, Shuang Wang, Huchuan Lu
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Reinforcement Learning, Visual Tracking

In this work, we propose a novel tracking algorithm with real-time performance based on the âActor-Criticâ framework. This framework consists of two major components: âActorâ and âCriticâ. The âActorâ model aims to infer the optimal choice in a continuous action space, which directly makes the tracker move the bounding box to the object location in the current frame. For ofï¬ine training,theâCriticâmodelisintroducedtoformaâActor-Criticâframeworkwith reinforcement learning and outputs a Q-value to guide the learning process of both âActorâ and âCriticâ deep networks. Then, we modify the original deep deterministic policy gradient algorithm to effectively train our âActor-Criticâ model for the tracking task. For online tracking, the âActorâ model provides a dynamic search strategy to locate the tracked object efï¬ciently and the âCriticâ model acts as a veriï¬cation module to make our tracker more robust. To the best of our knowledge, this work is the ï¬rst attempt to exploit the continuous action and âActor-Criticâ framework for visual tracking. Extensive experimental results on popular benchmarks demonstrate that the proposed tracker performs favorably against many state-of-the-art methods, with real-time performance.
Fully Motion-Aware Network for Video Object Detection
Authors: Shiyao Wang, Yucong Zhou, Junjie Yan, Zhidong Deng
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Object, object-detection, Object Detection, Video Object Detection

Video objection detection is challenging  in the presence of appearance deterioration in certain video frames. One of typical solutions is to enhance per-frame features through aggregating neighboring frames. But the features of objects are usually not spatially calibrated across frames due to motion from object and camera. In this paper, we propose an end-to-end model called fully motion-aware network (MANet), which jointly calibrates the features of objects on both pixel-level and instance-level in a unified framework. The pixel-level calibration is flexible in modeling detailed motion while the instance-level calibration captures more global motion cues in order to be robust to occlusion. To our best knowledge, MANet is the first work that can jointly train the two modules and dynamically combine them according to the motion patterns. It achieves leading performance on the large-scale ImageNet VID dataset.
Geometric Constrained Joint Lane Segmentation and Lane Boundary Detection
Authors: Jie Zhang, Yi Xu, Bingbing Ni, Zhenyu Duan
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Boundary Detection, Lane Detection

Lane detection is playing an indispensable role in advanced driver assistance systems. The existing approaches for lane detection can be categorized as lane area segmentation and lane boundary detection. Most of these methods abandon a great quantity of complementary information, such as geometric priors, when exploiting the lane area and the lane boundaries alternatively. In this paper, we establish a multiple-task learning framework to segment lane areas and detect lane boundaries simultaneously. The main contributions of the proposed frame- work are highlighted in two facets: (1) We put forward a multiple-task learning framework with mutually interlinked sub-structures between lane segmentation and lane boundary detection to improve overall performance. (2) A novel loss function is proposed with two geometric constraints considered, as assumed that the lane boundary is predicted as the outer contour of the lane area while the lane area is predicted as the area integration result within the lane boundary lines. With an end-to-end training process, these improvements extremely enhance the robustness and accuracy of our approach on several metrics. The proposed framework is evaluated on KITTI dataset, CULane dataset and RVD dataset. Compared with the state of the arts, our approach achieves the best performance on the metrics and a more robust detection in varied traffic scenes.
Goal-Oriented Visual Question Generation via Intermediate Rewards
Authors: Jun-Jie Zhang, Qi Wu, Chunhua Shen, Jian Zhang, Jianfeng Lu, Anton Van Den Hengel
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deep Reinforcement Learning, Informativeness, Question Generation, Question-Generation, Reinforcement Learning, Reinforcement Learning (RL)

Despite significant progress in a variety of vision-and-language problems, developing a method capable of asking intelligent, goal-oriented questions about images is proven to be an inscrutable challenge. Towards this end, we propose a Deep Reinforcement Learning framework based on three new intermediate rewards, namely goal-achieved, progressive and informativeness that encourage the generation of succinct questions, which in turn uncover valuable information towards the overall goal. By directly optimizing for questions that work quickly towards fulfilling the overall goal, we avoid the tendency of existing methods to generate long series of inane queries that add little value. We evaluate our model on the GuessWhat?! dataset and show that the resulting questions can help a standard `Guesser' identify a specific object in an image at a much higher success rate.
Teaching Machines to Understand Baseball Games: Large-Scale Baseball Video Database for Multiple Video Understanding Tasks
Authors: Minho Shim, Young Hwi Kim, Kyung-Min Kim, Seon Joo Kim
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Video Alignment, Video Recognition, Video Understanding

A major obstacle in teaching machines to understand videos is the lack of training data, as creating temporal annotations for long videos requires a huge amount of human effort. To this end, we introduce a new large-scale baseball video dataset called the BBDB, which is produced  semi-automatically by using play-by-play texts available online. The  BBDB contains 4200 hours of baseball game videos with 400k temporally annotated activity segments. The new dataset has several major challenging factors compared to other datasets: 1) the dataset contains a large number of visually similar segments with different labels. 2) It can be used for many video understanding tasks including video recognition, localization, text-video alignment, video highlight generation, and data imbalance problem. To observe the potential of the BBDB, we conducted extensive experiments by running many different types of video understanding algorithms on our new dataset. The database is available at https://sites.google.com/site/eccv2018bbdb/
Deep Bilinear Learning for RGB-D Action Recognition
Authors: Jian-Fang Hu, Wei-Shi Zheng, Jia-Hui Pan, Jian-Huang Lai, Jian-Guo Zhang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Action Recognition, Temporal Action Localization

In this paper, we focus on exploring modality-temporal mutual information for RGB-D action recognition. In order to learn time-varying information and multi-modal features jointly, we propose a novel deep bilinear learning framework. In the framework, we propose bilinear blocks that consist of two linear pooling layers for pooling the input cube features from both modality and temporal directions, separately. To capture rich modality-temporal information and facilitate our deep bilinear learning, a new action feature called modality-temporal cube is presented in a tensor structure for characterizing RGB-D actions from a comprehensive perspective. Our method is extensively tested on two public datasets with four different evaluation settings, and the results show that the proposed method outperforms the state-of-the-art approaches.
RelocNet: Continuous Metric Learning Relocalisation using Neural Nets
Authors: Vassileios Balntas, Shuda Li, Victor Prisacariu
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Metric Learning, Pose Retrieval, Retrieval

We propose a method of learning suitable convolutional   representations for camera pose retrieval based on nearest neighbour   matching and continuous metric learning-based feature descriptors. We   introduce information from camera frusta overlaps between pairs of   images to optimise our feature embedding network. Thus, the final   camera pose descriptor differences represent camera pose   changes. In addition, we build a pose regressor that is trained with a   geometric loss to infer finer relative poses between a query and   nearest neighbour images. Experiments show that our method is able to   generalise in a meaningful way, and outperforms related methods across   several experiments.
Generative Semantic Manipulation with Mask-Contrasting GAN
Authors: Xiaodan Liang, Hao Zhang, Liang Lin, Eric Xing 
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Image-to-Image Translation

Despite the promising results on paired/unpaired image-to-image translation achieved by Generative Adversarial Networks (GANs), prior works often only transfer the low-level information (e.g. color or texture changes), but fail to manipulate high-level semantic meanings (e.g., geometric structure or content) of different object regions. On the other hand, while some researches can synthesize compelling real-world images given a class label or caption, they cannot condition on arbitrary shapes or structures, which largely limits their application scenarios and interpretive capability of model results. In this work, we focus on a more challenging semantic manipulation task, aiming at modifying the semantic meaning of an object while preserving its own characteristics (e.g. viewpoints and shapes), such as cow$ightarrow$sheep, motor$ightarrow$ bicycle, cat$ightarrow$dog. To tackle such large semantic changes, we introduce a contrasting GAN (contrast-GAN) with a novel adversarial contrasting objective which is able to perform all types of semantic translations with one category-conditional generator. Instead of directly making the synthesized samples close to target data as previous GANs did, our adversarial contrasting objective optimizes over the distance comparisons between samples, that is, enforcing the manipulated data be semantically closer to the real data with target category than the input data. Equipped with the new contrasting objective, a novel mask-conditional contrast-GAN architecture is proposed to enable disentangle image background with object semantic changes. Extensive qualitative and quantitative experiments on several semantic manipulation tasks on ImageNet and MSCOCO dataset show considerable performance gain by our contrast-GAN over other conditional GANs.
Interpolating Convolutional Neural Networks Using Batch Normalization
Authors: Gratianus Wesley Putra Data, Kirjon Ngu, David William Murray, Victor Adrian Prisacariu
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Few-Shot Learning, Style Transfer

Perceiving a visual concept as a mixture of learned ones is natural for humans, aiding them to grasp new concepts and strengthening old ones. For all their power and recent success, deep convolutional networks do not have this ability. Inspired by recent work on universal representations for neural networks, we propose a simple emulation of this mechanism by purposing batch normalization layers to discriminate visual classes, and formulating a way to combine them to solve new tasks. We show that this can be applied for 2-way few-shot learning where we obtain between 4% and 17% better accuracy compared to straightforward full fine-tuning, and demonstrate that it can also be extended to the orthogonal application of style transfer.
An Adversarial Approach to Hard Triplet Generation
Authors: Yiru Zhao, Zhongming Jin, Guo-Jun Qi, Hongtao Lu, Xian-Sheng Hua
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Clustering, Image Retrieval, Retrieval, Triplet

While deep neural networks have demonstrated competitive results for many visual recognition and image retrieval tasks, the major challenge lies in distinguishing similar images from different categories (i.e., hard negative examples) while clustering images with large variations from the same category (i.e., hard positive examples). The current state-of-the-art is to mine the most hard triplet examples from the mini-batch to train the network. However, mining-based methods tend to look into these triplets that are hard in terms of the current estimated network, rather than deliberately generating those hard triplets that really matter in globally optimizing the network.     For this purpose, we propose an adversarial network for Hard Triplet Generation (HTG) to optimize the network ability in distinguishing similar examples of different categories as well as grouping varied examples of the same categories.    We evaluate our method on the real-world challenging datasets, such as CUB200-2011, CARS196, DeepFashion and VehicleID datasets, and show that our method outperforms the state-of-the-art methods significantly.
SphereNet: Learning Spherical Representations for Detection and Classification in Omnidirectional Images
Authors: Benjamin Coors, Alexandru Paul Condurache, Andreas Geiger
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: General Classification, image-classification, Image Classification, object-detection, Object Detection

Omnidirectional cameras offer great benefits over classical cameras wherever a wide field of view is essential, such as in virtual reality applications or in autonomous robots. Unfortunately, standard convolutional neural networks are not well suited for this scenario as the natural projection surface is a sphere which cannot be unwrapped to a plane without introducing significant distortions, particularly in the polar regions. In this work, we present SphereNet, a novel deep learning framework which encodes invariance against such distortions explicitly into convolutional neural networks. Towards this goal, SphereNet adapts the sampling locations of the convolutional filters, effectively reversing distortions, and wraps the filters around the sphere. By building on regular convolutions, SphereNet enables the transfer of existing perspective convolutional neural network models to the omnidirectional case. We demonstrate the effectiveness of our method on the tasks of image classification and object detection, exploiting two newly created semi-synthetic and real-world omnidirectional datasets.
Parallel Feature Pyramid Network for Object Detection
Authors: Seung-Wook Kim, Hyong-Keun Kook, Jee-Young Sun, Mun-Cheon Kang, Sung-Jea Ko
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Object, object-detection, Object Detection

Recently developed object detectors employ a convolutional neural network (CNN) by gradually increasing the number of feature layers with a pyramidal shape instead of using a featurized image pyramid. However, the different abstraction levels of the CNN feature layers often limit the detection performance, especially on small objects. To overcome this limitation, we propose a CNN-based object detection architecture, referred to as a parallel feature pyramid (FP) network (PFPNet), where the FP is constructed by widening the network width instead of increasing the network depth. First, we adopt spatial pyramid pooling and some additional feature transformations to generate a pool of feature maps with different sizes. In PFPNet, the additional feature transformation is performed in parallel, which yields the feature maps with similar levels of semantic abstraction across the scales.  We then resize the elements of the feature pool to a uniform size and aggregate their contextual information to generate each level of the final FP.  The experimental results confirmed that PFPNet increases the performance of the latest version of the single-shot multi-box detector (SSD) by mAP of 6.4% AP and especially, 7.8% AP_small on the MS-COCO dataset.
Person Search by Multi-Scale Matching
Authors: Xu Lan , Xiatian Zhu , Shaogang Gong 
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Benchmarking, Human Detection, Person Search

We consider the problem of person search in unconstrained scene images. Existing methods usually focus on improving the person detection accuracy to mitigate negative effects imposed by misalignment, mis-detections, and false alarms resulted from noisy people auto-detection. In contrast to previous studies, we show that sufficiently reliable person instance cropping is achievable by slightly improved state-of-the-art deep learning object detectors (e.g. Faster-RCNN), and the under-studied multi-scale matching problem in person search is a more severe barrier. In this work, we address this multi-scale person search challenge by proposing a Cross-Level Semantic Alignment (CLSA) deep learning approach capable of learning more discriminative identity feature representations in a unified end-to-end model. This is realised by exploiting the in-network feature pyramid structure of a deep neural network enhanced by a novel cross pyramid-level semantic alignment loss function. This favourably eliminates the need for constructing a computationally expensive image pyramid and a complex multi-branch network architecture. Extensive experiments show the modelling advantages and performance superiority of CLSA over the state-of-the-art person search and multi-scale matching methods on two large person search benchmarking datasets: CUHK-SYSU and PRW.
Hierarchy of Alternating Specialists for Scene Recognition
Authors: Hyo Jin Kim, Jan-Michael Frahm 
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Scene Classification, Scene Recognition

We introduce a method for improving convolutional neural networks (CNNs) for scene classification. We present a hierarchy of specialist networks, which disentangles the intra-class variation and inter-class similarity in a coarse to fine manner. Our key insight is that each subset within a class is often associated with different types of inter-class similarity. This suggests that existing network of experts approaches that organize classes into coarse categories are suboptimal. In contrast, we group images based on high-level appearance features rather than their class membership and dedicate a specialist model per group. In addition, we propose an alternating architecture with a global ordered- and a global orderless-representation to account for both the coarse layout of the scene and the transient objects. We demonstrate that it leads to better performance than using a single type of representation as well as the fused features. We also introduce a mini-batch soft k-means that allows end-to-end fine-tuning, as well as a novel routing function for assigning images to specialists. Experimental results show that the proposed approach achieves a significant improvement over baselines including the existing tree-structured CNNs with class-based grouping.
3D Scene Flow from 4D Light Field Gradients
Authors: Sizhuo Ma, Brandon M. Smith, Mohit Gupta
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Optical Flow Estimation

This paper presents novel techniques for recovering 3D dense scene flow, based on differential analysis of 4D light fields. The key enabling result is a per-ray linear equation, called the ray flow equation, that relates 3D scene flow to 4D light field gradients. The ray flow equation is invariant to 3D scene structure and applicable to a general class of scenes, but is underconstrained (3 unknowns per equation). Thus, additional constraints must be imposed to recover motion. We develop two families of scene flow algorithms by leveraging the structural similarity between ray flow and optical flow equations: local "Lucas-Kanade" ray flow and global "Horn-Schunck" ray flow, inspired by corresponding optical flow methods. We also develop a combined local-global method by utilizing the correspondence structure in the light fields. We demonstrate high precision 3D scene flow recovery for a wide range of scenarios, including rotation and non-rigid motion. We analyze the theoretical and practical performance limits of the proposed techniques via the light field structure tensor, a 3x3 matrix that encodes the local structure of light fields. We envision that the proposed analysis and algorithms will lead to design of future light-field cameras that are optimized for motion sensing, in addition to depth sensing.