pwc-archive/papers-with-abstracts huggingface dataset:
Summarizing First-Person Videos from Third Persons' Points of View
Authors: Hsuan-I Ho, Wei-Chen Chiu, Yu-Chiang Frank Wang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

Video highlight or summarization is among interesting topics in computer vision, which benefits a variety of applications like viewing, searching, or storage. However, most existing studies rely on training data of third-person videos, which cannot easily generalize to highlight the first-person ones. With the goal of deriving an effective model to summarize first-person videos, we propose a novel deep neural network architecture for describing and discriminating vital spatiotemporal information across videos with different points of view. Our proposed model is realized in a semi-supervised setting, in which fully annotated third-person videos, unlabeled first-person videos, and a small amount of annotated first-person ones are presented during training. In our experiments, qualitative and quantitative evaluations on both benchmarks and our collected first-person video datasets are presented.
Programmable Triangulation Light Curtains
Authors: Jian Wang, Joseph Bartels, William Whittaker, Aswin C. Sankaranarayanan, Srinivasa G. Narasimhan
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

A vehicle on a road or a robot in the field does not need a full-featured 3D depth sensor to detect potential collisions or monitor its blind spot. Instead, it needs to only monitor if any object comes within its near proximity which is an easier task than full depth scanning. We introduce a novel device that monitors the presence of objects on a  virtual shell near the device, which we refer to as a light curtain. Light curtains offer a light-weight, resource-efficient and programmable approach to proximity awareness for obstacle avoidance and navigation. They also have additional benefits in terms of improving visibility in fog as well as flexibility in handling light fall-off. Our prototype for generating light curtains works by rapidly rotating a line sensor and a line laser, in synchrony. The device is capable of generating light curtains of various shapes with a range of 20-30m in sunlight (40m under cloudy skies and 50m indoors) and adapts dynamically to the demands of the task. We analyze properties of light curtains and various approaches to optimize their thickness as well as power requirements. We showcase the potential of light curtains using a range of real-world scenarios.
Learning Single-View 3D Reconstruction with Limited Pose Supervision
Authors: Guandao Yang, Yin Cui, Serge Belongie, Bharath Hariharan
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Reconstruction, Single-View 3D Reconstruction, Transfer Learning

It is expensive to label images with 3D structure or precise camera pose. Yet, this is precisely the kind of annotation required to train single-view 3D reconstruction models. In contrast, unlabeled images or images with just category labels are easy to acquire, but few current models can use this weak supervision. We present a unified framework that can combine both types of supervision: a small amount of camera pose annotations are used to enforce pose-invariance and view-point consistency, and unlabeled images combined with an adversarial loss are used to enforce the realism of rendered, generated models. We use this unified framework to measure the impact of each form of supervision in three paradigms: semi-supervised, multi-task, and transfer learning. We show that with a combination of these ideas, we can train single-view reconstruction models that improve up to 7 points in performance (AP) when using only 1% pose annotated training data.
Snap Angle Prediction for 360Â° Panoramas
Authors: Bo Xiong, Kristen Grauman
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Prediction, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

360Â° panoramas are a rich medium, yet notoriously difficult to visualize in the 2D image plane.  We explore how intelligent rotations of a spherical image may enable content-aware projection with fewer perceptible distortions.  Whereas existing approaches assume the viewpoint is fixed, intuitively some viewing angles within the sphere preserve high-level objects better than others.  To discover the relationship between these optimal emph{snap angles} and the spherical panorama's content, we develop a reinforcement learning approach for the cubemap projection model.  Implemented as a deep recurrent neural network, our method selects a sequence of rotation actions and receives reward for avoiding cube boundaries that overlap with important foreground objects.  Our results demonstrate the impact both qualitatively and quantitatively.
K-convexity shape priors for segmentation
Authors: Hossam Isack, Lena Gorelick, Karin Ng, Olga Veksler, Yuri Boykov
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Descriptive, Object, Segmentation

This work extends popular star-convexity and other more general forms of convexity priors. We represent an object as a union of "convex'' overlappable subsets. Since an arbitrary shape can always be divided into convex parts, our regularization model restricts the number of such parts. Previous k-part shape priors are limited to disjoint parts. For example, one approach segments an object via optimizing its $k$-coverage by disjoint convex parts, which we show is highly sensitive to local minima. In contrast, our shape model allows the convex parts to overlap, which both relaxes and simplifies the coverage problem, e.g. fewer parts are needed to represent any object. As shown in the paper, for many forms of convexity our regularization model is significantly more descriptive for any given k. Our shape prior is useful in practice, e.g. in biomedical applications, and its optimization is robust to local minima.
Incremental Multi-graph Matching via Diversity and Randomness based Graph Clustering
Authors: Tianshu Yu, Junchi Yan, Wei Liu, Baoxin Li
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: All, Clustering, Diversity, Graph Clustering, Graph Matching

Multi-graph matching refers to finding correspondences across graphs, which are traditionally solved by matching all the graphs in a single batch. However in real-world applications, graphs are often collected incrementally, rather than once for all. In this paper, we present an incremental multi-graph matching approach, which deals with the arriving graph utilizing the previous matching results under the global consistency constraint. When a new graph arrives, rather than re-optimizing over all graphs, we propose to partition graphs into subsets with certain topological structure and conduct optimization within each subset. The partitioning procedure is guided by the diversity within partitions and randomness over iterations, and we present an interpretation showing why these two factors are essential. The final matching results are calculated over all subsets via an intersection graph. Extensive experimental results on synthetic and real image datasets show that our algorithm notably improves the efficiency without sacrificing the accuracy.
End-to-End Joint Semantic Segmentation of Actors and Actions in Video
Authors: Jingwei Ji, Shyamal Buch, Alvaro Soto, Juan Carlos Niebles
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Action Recognition, Segmentation, Semantic Segmentation, Temporal Action Localization, Video Understanding, Zero-Shot Learning

Traditional video understanding tasks include human action recognition and actor/object semantic segmentation. However, the combined task of providing semantic segmentation for different actor classes simultaneously with their action class remains a challenging but necessary task for many applications. In this work, we propose a new end-to-end architecture for tackling this task in videos. Our model effectively leverages multiple input modalities, contextual information, and multitask learning in the video to directly output semantic segmentations in a single unified framework. We train and benchmark our model on the Actor-Action Dataset (A2D) for joint actor-action semantic segmentation, and demonstrate state-of-the-art performance for both segmentation and detection. We also perform experiments verifying our approach improves performance for zero-shot recognition, indicating generalizability of our jointly learned feature space.
Scalable Exemplar-based Subspace Clustering on Class-Imbalanced Data
Authors: Chong You, Chi Li, Daniel P. Robinson, Rene Vidal
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Clustering, image-classification, Image Classification

Subspace clustering methods based on expressing each data point as a linear combination of a few other data points (e.g., sparse subspace clustering) have become a popular tool for unsupervised learning due to their empirical success and theoretical guarantees. However, their performance can be affected by imbalanced data distributions and large-scale datasets. This paper presents an exemplar-based subspace clustering method to tackle the problem of imbalanced and large-scale datasets. The proposed method searches for a subset of the data that best represents all data points as measured by the $ell_1$-norm of the representation coefficients. To solve our model efficiently, we introduce a farthest first search algorithm which iteratively selects the least well-represented point as an exemplar. When data comes from a union of subspaces, we prove that the computed subset contains enough exemplars from each subspace for expressing all data points even if the data are imbalanced. Our experiments demonstrate that the proposed method outperforms state-of-the-art subspace clustering methods in two large-scale image datasets that are imbalanced. We also demonstrate the effectiveness of our method on unsupervised data subset selection for a face image classification task.
RCAA: Relational Context-Aware Agents for Person Search
Authors: Xiaojun Chang, Po-Yao Huang, Yi-Dong Shen, Xiaodan Liang, Yi Yang, Alexander G. Hauptmann
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Person Search

We aim to search for a target person from a gallery of whole scene images for which the annotations of pedestrian bounding boxes are unavailable. Previous approaches to this problem have relied on a pedestrian proposal net, which may generate redundant proposals and increase the computational burden. In this paper, we address this problem by training relational context-aware agents which learn the actions to localize the target person from the gallery of whole scene images. We incorporate the relational spatial and temporal contexts into the framework. Specifically, we propose to use the target person as the query in the query-dependent relational network. The agent determines the best action to take at each time step by simultaneously considering the local visual information, the relational and temporal contexts, together with the target person. To validate the performance of our approach, we conduct extensive experiments on the large-scale Person Search benchmark dataset and achieve significant improvements over the compared approaches. It is also worth noting that the proposed model even performs better than traditional methods with perfect pedestrian detectors.
Product Quantization Network for Fast Image Retrieval
Authors: Tan Yu, Junsong Yuan, Chen Fang, Hailin Jin 
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Image Retrieval, Quantization, Retrieval, Triplet

Product quantization has been widely used in fast image retrieval due to its effectiveness of coding high-dimensional visual features. By extending the hard assignment to soft assignment, we make it feasible to incorporate the product quantization as a layer of a convolutional neural network and propose our product quantization network. Meanwhile, we come up with a novel asymmetric triplet loss, which effectively boosts the retrieval accuracy of the proposed product quantization network based on asymmetric similarity.  Through the proposed product quantization network, we can obtain a discriminative and compact image representation in an end-to-end manner, which further enables a fast and accurate image retrieval. Comprehensive experiments conducted on public benchmark datasets demonstrate the state-of-the-art performance of the proposed product quantization network.
Learning to Blend Photos
Authors: Wei-Chih Hung, Jianming Zhang, Xiaohui Shen, Zhe Lin, Joon-Young Lee, Ming-Hsuan Yang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

Photo blending is a common technique to create aesthetically pleasing artworks by combining multiple photos. However, the process of photo blending is usually time-consuming, and care must be taken in the process of blending, filtering, positioning, and masking each of the source photos. To make photo blending accessible to general public, we propose an efficient approach for automatic photo blending via deep learning. Specifically, given a foreground image and a background image, our proposed method automatically generates a set of blending photos with scores that indicate the aesthetics quality with the proposed quality network and policy network. Experimental results show that the proposed approach can effectively generate high quality blending photos with efficiency.
Deeply Learned Compositional Models for Human Pose Estimation
Authors: Wei Tang, Pei Yu, Ying Wu
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Pose Estimation

Compositional models represent patterns with hierarchies of meaningful parts and subparts. Their ability to characterize high-order relationships among body parts helps resolve low-level ambiguities in human pose estimation (HPE). However, prior compositional models make unrealistic assumptions on subpart-part relationships, making them incapable to characterize complex compositional patterns. Moreover, state spaces of their higher-level parts can be exponentially large, complicating both inference and learning. To address these issues, this paper introduces a novel framework, termed as Deeply Learned Compositional Model (DLCM), for HPE. It exploits deep neural networks to learn the compositionality of human bodies. This results in a network with a hierarchical compositional architecture and bottom-up/top-down inference stages. In addition, we propose a novel bone-based part representation. It not only compactly encodes orientations, scales and shapes of parts, but also avoids their potentially large state spaces. With significantly lower complexities, our approach outperforms state-of-the-art methods on three benchmark datasets.
Unsupervised Video Object Segmentation with Motion-based Bilateral Networks
Authors: Siyang Li, Bryan Seybold, Alexey Vorobyov, Xuejing Lei, C. -C. Jay Kuo
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Object, Segmentation, Semantic Segmentation, Unsupervised Video Object Segmentation, Video Object Segmentation, Video Salient Object Detection, Video Semantic Segmentation

In this work, we study the unsupervised video object segmentation problem where moving objects are segmented without prior knowledge of these objects. First, we propose a motion-based bilateral network to estimate the background based on the motion pattern of non-object regions. The bilateral network reduces false positive regions by accurately identifying background objects. Then, we integrate the background estimate from the bilateral network with instance embeddings into a graph, which allows multiple frame reasoning with graph edges linking pixels from different frames. We classify graph nodes by defining and minimizing a cost function, and segment the video frames based on the node labels. The proposed method outperforms previous state-of-the-art unsupervised video object segmentation methods against the DAVIS 2016 and the FBMS-59 datasets.
Generalizing A Person Retrieval Model Hetero- and Homogeneously
Authors: Zhun Zhong, Liang Zheng, Shaozi Li, Yi Yang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Domain Adaptation, model, Person Re-Identification, Person Retrieval, Retrieval, Unsupervised Domain Adaptation

Person re-identification (re-ID) poses unique challenges for unsupervised domain adaptation (UDA) in that classes in the source and target sets (domains) are entirely different and that image variations are largely caused by cameras. Given a labeled source training set and an unlabeled target training set, we aim to improve the generalization ability of re-ID models on the target testing set. To this end, we introduce a Hetero-Homogeneous Learning (HHL) method. Our method enforces two properties simultaneously: 1) camera invariance, learned via positive pairs formed by unlabeled target images and their camera style transferred counterparts; 2) domain connectedness, by regarding source / target images as negative matching pairs to the target / source images. The first property is implemented by homogeneous learning because training pairs are collected from the same domain. The second property is achieved by heterogeneous learning because we sample training pairs from both the source and target domains. On Market-1501, DukeMTMC-reID and CUHK03, we show that the two properties contribute indispensably and that very competitive re-ID UDA accuracy is achieved. Code is available at: https://github.com/zhunzhong07/HHL
Cross-Modal Hamming Hashing
Authors: Yue Cao , Bin Liu, Mingsheng Long, Jian-Min Wang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Cross-Modal Retrieval, Retrieval

Cross-modal hashing enables similarity retrieval across different content modalities, such as searching relevant images in response to text queries. It provides with the advantages of computation efficiency and retrieval quality for multimedia retrieval. Hamming space retrieval enables efficient constant-time search that returns data items within a given Hamming radius to each query, by hash lookups instead of linear scan. However, Hamming space retrieval is ineffective in existing cross-modal hashing methods, subject to their weak capability of concentrating the relevant items to be within a small Hamming ball, while worse still, the Hamming distances between hash codes from different modalities are inevitably large due to the large heterogeneity across different modalities. This work presents Cross-Modal Hamming Hashing (CMHH), a novel deep cross-modal hashing approach that generates compact and highly concentrated hash codes to enable efficient and effective Hamming space retrieval. The main idea is to penalize significantly on similar cross-modal pairs with Hamming distance larger than the Hamming radius threshold, by designing a pairwise focal loss based on the exponential distribution. Extensive experiments demonstrate that CMHH can generate highly concentrated hash codes and achieve state-of-the-art cross-modal retrieval performance for both hash lookups and linear scan scenarios on three benchmark datasets, NUS-WIDE, MIRFlickr-25K, and IAPR TC-12.
Propagating LSTM: 3D Pose Estimation based on Joint Interdependency
Authors: Kyoungoh Lee, Inwoong Lee, Sang-Hoon Lee
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Human Pose Estimation, 3D Pose Estimation, Pose Estimation

We present a novel 3D pose estimation method based on joint interdependency (JI) for acquiring 3D joints from the human pose of an RGB image. The JI incorporates the body part based structural connectivity of joints to learn the high spatial correlation of human posture on our method. Towards this goal, we propose a new long short-term memory (LSTM)-based deep learning architecture named propagating LSTM networks (p-LSTMs), where each LSTM is connected sequentially to reconstruct 3D depth from the centroid to edge joints through learning the intrinsic JI. In the first LSTM, the seed joints of 3D pose are created and reconstructed into the whole-body joints through the connected LSTMs. Utilizing the p-LSTMs, we achieve the higher accuracy of about 11.2% than state-of-the-art methods on the largest publicly available database. Importantly, we demonstrate that the JI drastically reduces the structural errors at body edges, thereby leads to a significant improvement.
Deep Video Quality Assessor: From Spatio-temporal Visual Sensitivity to A Convolutional Neural Aggregation Network
Authors: Woojae Kim, Jongyoo Kim, Sewoong Ahn, Jinwoo Kim, Sang-Hoon Lee
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Sensitivity, Video Quality Assessment, Visual Question Answering (VQA)

Incorporating spatio-temporal human visual perception into video quality assessment (VQA) remains a formidable issue. Previous statistical or computational models of spatio-temporal perception have limitations to be applied to the general VQA algorithms. In this paper, we propose a novel full-reference (FR) VQA framework named Deep Video Quality Assessor (DeepVQA) to quantify the spatio-temporal visual perception via a convolutional neural network (CNN) and a convolutional neural aggregation network (CNAN). Our framework enables to figure out the spatio-temporal sensitivity behavior through learning in accordance with the subjective score. In addition, to manipulate the temporal variation of distortions, we propose a novel temporal pooling method using an attention model. In the experiment, we show DeepVQA remarkably achieves the state-of-the-art prediction accuracy of more than 0.9 correlation, which is ~5% higher than those of conventional methods on the LIVE and CSIQ video databases.
Face Recognition with Contrastive Convolution
Authors: Chunrui Han, Shiguang Shan, Meina Kan, Shuzhe Wu, Xilin Chen
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Face Recognition, Face Verification

In current face recognition approaches with convolutional neural network (CNN), a pair of faces to compare are independently fed into the CNN for feature extraction. For both faces the same kernels are applied and hence the representation of a face stays fixed regardless of who it is compared with. As for us humans, however, one generally focuses on varied characteristics of a face when comparing it with distinct persons. Inspired, we propose a novel CNN structure with what we referred to as contrastive convolution, which specifically focuses on the distinct characteristics between the two faces to compare, i.e., those contrastive characteristics. Extensive experiments on the challenging LFW, and IJB-A show that our proposed contrastive convolution significantly improves the vanilla CNN and achieves quite promising performance in face verification task.
Monocular Depth Estimation with Affinity, Vertical Pooling, and Label Enhancement
Authors: Yukang Gan, Xiangyu Xu, Wenxiu Sun, Liang Lin
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Depth Estimation, Monocular Depth Estimation, Stereo Matching, Stereo Matching Hand

While significant progress has been made in monocular depth estimation with Convolutional Neural Networks (CNNs) extracting absolute features, such as edges and textures, the depth constraint of neighboring pixels, namely relative features, has been mostly ignored by recent methods. To overcome this limitation, we explicitly model the relationships of different image locations with an affinity layer and combine absolute and relative features in an end-to-end network. In addition, we also consider another prior knowledge that major depth changes in images lie in the vertical direction, and thus, it is beneficial to capture local vertical features for refined depth estimation. In the proposed algorithm we introduce vertical pooling to aggregate image features vertically to improve the depth accuracy.Furthermore, since the Lidar depth ground truth is quite sparse, we enhance the depth labels by generating high-quality dense depth maps with off-the-shelf stereo matching method which takes left-right image pairs as input.We also integrate multi-scale structures in our network to obtain global understanding the image depth and exploit residual learning to help depth refinement.We demonstrate that the proposed algorithm performs favorably against state-of-the-art methods both qualitatively and quantitatively on the KITTI driving dataset.