pwc-archive/papers-with-abstracts huggingface dataset:
Integrating Dashcam Views Through Inter-Video Mapping
Authors: Hsin-I Chen, Yi-Ling Chen, Wei-Tse Lee, Fan Wang, Bing-Yu Chen
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Motion Estimation

In this paper, an inter-video mapping approach is proposed to integrate video footages from two dashcams installed on a preceding and its following vehicle to provide the illusion that the driver of the following vehicle can see-through the preceding one. The key challenge is to adapt the perspectives of the two videos based on a small number of common features since a large portion of the common region in the video captured by the following vehicle is occluded by the preceding one. Inspired by the observation that images with the most similar viewpoints yield dense and high-quality matches, the proposed inter-video mapping estimates spatially-varying motions across two videos utilizing images of very similar contents. Specifically, we estimate frame-to-frame motions of each two consecutive images and incrementally add new views into a merged representation. In this way, long-rang motion estimation is achieved, and the observed perspective discrepancy between the two videos can be well approximated our motion estimation. Once the inter-video mapping is established, the correspondences can be updated incrementally, so the proposed method is suitable for on-line applications. Our experiments demonstrate the effectiveness of our approach on real-world challenging videos.
Visual Tracking With Fully Convolutional Networks
Authors: Lijun Wang, Wanli Ouyang, Xiaogang Wang, Huchuan Lu
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Object Tracking, Visual Tracking

We propose a new approach for general object tracking with fully convolutional neural network. Instead of treating convolutional neural network (CNN) as a black-box feature extractor, we conduct in-depth study on the properties of CNN features offline pre-trained on massive image data and classification task on ImageNet. The discoveries motivate the design of our tracking system. It is found that convolutional layers in different levels characterize the target from different perspectives. A top layer encodes more semantic features and serves as a category detector, while a lower layer carries more discriminative information and can better separate the target from distracters with similar appearance. Both layers are jointly used with a switch mechanism during tracking. It is also found that for a tracking target, only a subset of neurons are relevant. A feature map selection method is developed to remove noisy and irrelevant feature maps, which can reduce computation redundancy and improve tracking accuracy. Extensive evaluation on the widely used tracking benchmark shows that the proposed tacker outperforms the state-of-the-art significantly.
Multiple Feature Fusion via Weighted Entropy for Visual Tracking
Authors: Lin Ma, Jiwen Lu, Jianjiang Feng, Jie zhou
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Object, Object Tracking, Visual Object Tracking, Visual Tracking

It is desirable to combine multiple feature descriptors to improve the visual tracking performance because different features can provide complementary information to describe objects of interest. However, how to effectively fuse multiple features remains a challenging problem in visual tracking, especially in a data-driven manner. In this paper, we propose a new data-adaptive visual tracking approach by using multiple feature fusion via weighted entropy. Unlike existing visual trackers which simply concatenate multiple feature vectors together for object representation, we employ the weighted entropy to evaluate the dissimilarity between the object state and the background state, and seek the optimal feature combination by minimizing the weighted entropy, so that more complementary information can be exploited for object representation. Experimental results demonstrate the effectiveness of our approach in tackling various challenges for visual object tracking.
Pedestrian Travel Time Estimation in Crowded Scenes
Authors: Shuai Yi, Hongsheng Li, Xiaogang Wang
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Blocking, Scene Understanding, Travel Time Estimation

In this paper, we target on the problem of estimating the statistic of pedestrian travel time within a period from an entrance to a destination in a crowded scene. Such estimation is based on the global distributions of crowd densities and velocities instead of complete trajectories of pedestrians, which cannot be obtained in crowded scenes. The proposed method is motivated by our statistical investigation into the correlations between travel time and global properties of crowded scenes. Active regions are created for each source-destination pair to model the probable walking regions over the corresponding source-destination traffic flow. Two sets of scene features are specially designed for modeling moving and stationary persons inside the active regions and their influences on pedestrian travel time. The estimation of pedestrian travel time provides valuable  information for both crowd scene understanding and pedestrian behavior analysis, but was not sufficiently studied in literature. The effectiveness of the proposed pedestrian travel time estimation model is demonstrated through several surveillance applications, including dynamic scene monitoring,  localization of regions blocking traffics, and detection of abnormal pedestrian behaviors. Many more valuable applications based on our method are to be explored in the future.
Unsupervised Synchrony Discovery in Human Interaction
Authors: Wen-Sheng Chu, Jiabei Zeng, Fernando de la Torre, Jeffrey F. Cohn, Daniel S. Messinger
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Computational Efficiency

People are inherently social. Social interaction plays an important and natural role in human behavior. Most computational methods focus on individuals alone rather than in social context. They also require labelled training data. We present an unsupervised approach to discover interpersonal synchrony, referred as to two or more persons preforming common actions in overlapping video frames or segments. For computational efficiency, we develop a branch-and-bound (B&B) approach that affords exhaustive search while guaranteeing a globally optimal solution. The proposed method is entirely general. It takes from two or more videos any multi-dimensional signal that can be represented as a histogram. We derive three novel bounding functions and provide efficient extensions, including multi-synchrony detection and accelerated search, using a warm-start strategy and parallelism. We evaluate the effectiveness of our approach in multiple databases, including human actions using the CMU Mocap dataset, spontaneous facial behaviors using group-formation task dataset and parent-infant interaction dataset.
Efficient Video Segmentation Using Parametric Graph Partitioning
Authors: Chen-Ping Yu, Hieu Le, Gregory Zelinsky, Dimitris Samaras
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Clustering, Computational Efficiency, graph partitioning, Segmentation, Video Segmentation, Video Semantic Segmentation

Video segmentation is the task of grouping similar pixels in the spatio-temporal domain, and has become an important preprocessing step for subsequent video analysis. Most video segmentation and supervoxel methods output a hierarchy of segmentations, but while this provides useful multiscale information, it also adds difficulty in selecting the appropriate level for a task. In this work, we propose an efficient and robust video segmentation framework based on parametric graph partitioning (PGP), a fast, almost parameter free graph partitioning method that identifies and removes between-cluster edges to form node clusters. Apart from its computational efficiency, PGP performs clustering of the spatio-temporal volume without requiring a pre-specified cluster number or bandwidth parameters, thus making video segmentation more practical to use in applications. The PGP framework also allows processing sub-volumes, which further improves performance,  contrary to other streaming video segmentation methods where sub-volume processing reduces performance. We evaluate the PGP method using the  SegTrack v2 and Chen Xiph.org datasets, and show that it outperforms related state-of-the-art algorithms in 3D segmentation metrics and running time.
Activity Auto-Completion: Predicting Human Activities From Partial Videos
Authors: Zhen Xu, Laiyun Qing, Jun Miao
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Activity Prediction, Information Retrieval, Learning-To-Rank, Retrieval

In this paper, we propose an activity auto-completion (AAC) model for human activity prediction by formulating activity prediction as a query auto-completion (QAC) problem in information retrieval. First, we extract discriminative patches in frames of videos. A video is represented based on these patches and divided into a collection of segments, each of which is regarded as a character typed in the search box. Then a partially observed video is considered as an activity prefix, consisting of one or more characters. Finally, the missing observation of an activity is predicted as the activity candidates provided by the auto-completion model. The candidates are matched against the activity prefix on-the-fly and ranked by a learning-to-rank algorithm. We validate our method on UT-Interaction Set #1 and Set #2 [19]. The experimental results show that the proposed activity auto-completion model achieves promising performance.
Adaptive Exponential Smoothing for Online Filtering of Pixel Prediction Maps
Authors: Kang Dang, Jiong Yang, Junsong Yuan
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Prediction, Saliency Detection, Scene Parsing

We propose an efficient online video filtering method, called adaptive exponential filtering (AES) to refine pixel prediction maps. Assuming each pixel is associated with a discriminative prediction score, the proposed AES applies exponentially decreasing weights over time to smooth the prediction score of each pixel, similar to classic exponential smoothing. However, instead of fixing the spatial pixel location to perform temporal filtering, we trace each pixel in the past frames by finding the optimal path that can bring the maximum exponential smoothing score, thus performing adaptive and non-linear filtering. Thanks to the pixel tracing, AES can better address object movements and avoid over-smoothing. To enable real-time filtering, we propose a linear-complexity dynamic programming scheme that can trace all pixels simultaneously. We apply the proposed filtering method to improve both saliency detection maps and scene parsing maps. The comparisons with average and exponential filtering, as well as state-of-the-art methods, validate that our AES can effectively refine the pixel prediction maps, without using the original video again.
Fully Connected Object Proposals for Video Segmentation
Authors: Federico Perazzi, Oliver Wang, Markus Gross, Alexander Sorkine-Hornung
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Object, Segmentation, Semi-Supervised Video Object Segmentation, Video Segmentation, Video Semantic Segmentation

We present a novel approach to video segmentation using multiple object proposals. The problem is formulated as a minimization of a novel energy function defined over a fully connected graph of object proposals. Our model combines appearance with long-range point tracks, which is key to ensure robustness with respect to fast motion and occlusions over longer video sequences. As opposed to previous approaches based on object proposals, we do not seek the best per-frame object hypotheses to perform the segmentation. Instead, we combine multiple, potentially imperfect proposals to improve overall segmentation accuracy and ensure robustness to outliers. Overall, the basic algorithm consists of three steps. First, we generate a very large number of object proposals for each video frame using existing techniques. Next, we perform an SVM-based pruning step to retain only high quality proposals with sufficiently discriminative power. Finally, we determine the fore- and background classification by solving for the maximum a posteriori of a fully connected conditional random field, defined using our novel energy function. Experimental results on a well established dataset demonstrate that our method compares favorably to several recent state-of-the-art approaches.
Video Segmentation With Just a Few Strokes
Authors: Naveen Shankar Nagaraja, Frank R. Schmidt, Thomas Brox
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Motion Segmentation, Segmentation, Video Segmentation, Video Semantic Segmentation

As the use of videos is becoming more popular in computer vision, the need for annotated video datasets increases.  Such datasets are required either as training data or simply as ground truth for benchmark datasets. A particular challenge in video segmentation is due to disocclusions, which hamper frame-to-frame propagation, in conjunction with non-moving objects. We show that a combination of motion from point trajectories, as known from motion segmentation, along with minimal supervision can largely help solve this problem. Moreover, we integrate a new constraint that enforces consistency of the color distribution in successive frames. We quantify user interaction effort with respect to segmentation quality on challenging ego motion videos.  We compare our approach to a diverse set of algorithms in terms of user effort and in terms of performance on common video segmentation benchmarks.
Actionness-Assisted Recognition of Actions
Authors: Ye Luo, Loong-Fah Cheong, An Tran
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Action Detection, Action Recognition, Temporal Action Localization

We elicit from a fundamental definition of action low-level attributes that can reveal agency and intentionality. These descriptors are mainly trajectory-based, measuring sudden changes, temporal synchrony, and repetitiveness. The actionness map can be used to localize actions in a way that is generic across action and agent types. Furthermore, it also groups interacting regions into a useful unit of analysis, which is crucial for recognition of actions involving interactions. We then implement an actionness-driven pooling scheme to improve action recognition performance. Experimental results on three datasets show the advantages of our method on both action detection and action recognition comparing with other state-of-the-art methods.
COUNT Forest: CO-Voting Uncertain Number of Targets Using Random Forest for Crowd Density Estimation
Authors: Viet-Quoc Pham, Tatsuo Kozakaya, Osamu Yamaguchi, Ryuzo Okada
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Density Estimation, Scene Understanding

This paper presents a patch-based approach for crowd density estimation in public scenes. We formulate the problem of estimating density in a structured learning framework applied to random decision forests. Our approach learns the mapping between patch features and relative locations of all objects inside each patch, which contribute to generate the patch density map through Gaussian kernel density estimation. We build the forest in a coarse-to-fine manner with two split node layers, and further propose a crowdedness prior and an effective forest reduction method to improve the estimation accuracy and speed. Moreover, we introduce a semi-automatic training method to learn the estimator for a specific scene. We achieved state-of-the-art results on the public Mall dataset and UCSD dataset, and also proposed two potential applications in traffic counts and scene understanding with promising results.
Motion Trajectory Segmentation via Minimum Cost Multicuts
Authors: Margret Keuper, Bjoern Andres, Thomas Brox
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Clustering, Segmentation, Unsupervised Video Object Segmentation

For the segmentation of moving objects in videos, the analysis of long-term point trajectories has been very popular recently. In this paper, we formulate the segmentation of a video sequence based on point trajectories as a minimum cost multicut problem. Unlike the commonly used spectral clustering formulation, the minimum cost multicut formulation gives natural rise to optimize not only for a cluster assignment but also for the number of clusters while allowing for varying cluster sizes. In this setup, we provide a method to create a long-term point trajectory graph with attractive and repulsive binary terms and outperform state-of-the-art methods based on spectral clustering on the FBMS-59 dataset and on the motion subtask of the VSB100 dataset.
Action Localization in Videos Through Context Walk
Authors: Khurram Soomro, Haroon Idrees, Mubarak Shah
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Action Localization

This paper presents an efficient approach for localizing actions by learning contextual relations, in the form of relative locations between different video regions. We begin by over-segmenting the videos into supervoxels, which have the ability to preserve action boundaries and also reduce the complexity of the problem. Context relations are learned during training which capture displacements from all the supervoxels in a video to those belonging to foreground actions. Then, given a testing video, we select a supervoxel randomly and use the context information acquired during training to estimate the probability of each supervoxel belonging to the foreground action. The walk proceeds to a new supervoxel and the process is repeated for a few steps. This ``context walk'' generates a conditional distribution of an action over all the supervoxels. A Conditional Random Field is then used to find action proposals in the video, whose confidences are obtained using SVMs. We validated the proposed approach on several datasets and show that context in the form of relative displacements between supervoxels can be extremely useful for action localization. This also results in significantly fewer evaluations of the classifier, in sharp contrast to the alternate sliding window approaches.
RGB-W: When Vision Meets Wireless
Authors: Alexandre Alahi, Albert Haque, Li Fei-Fei
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12

Inspired by the recent success of RGB-D cameras, we propose the enrichment of RGB data with an additional "quasi-free" modality, namely, the wireless signal (e.g., wifi or Bluetooth) emitted by individuals' cell phones, referred to as RGB-W. The received signal strength acts as a rough proxy for depth and a reliable cue on their identity. Although the measured signals are highly noisy (more than 2m average localization error), we demonstrate that the combination of visual and wireless data significantly improves the localization accuracy.   We introduce a novel image-driven representation of wireless data which embeds all received signals onto a single image. We then indicate the ability of this additional data to (i) locate persons within a sparsity-driven framework and to (ii) track individuals with a new confidence measure on the data association problem. Our solution outperforms existing localization methods by a significant margin. It can be applied to the millions of currently installed RGB cameras to better analyze human behavior and offer the next generation of high-accuracy location-based services.
Action Detection by Implicit Intentional Motion Clustering
Authors: Wei Chen, Jason J. Corso
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Action Detection, Action Recognition, Clustering, Human Detection, Pose Estimation, Temporal Action Localization, Trajectory Clustering

Explicitly using human detection and pose estimation has found limited success in action recognition problems.  This may be due to the complexity in the articulated motion human exhibit.  Yet, we know that action requires an actor and intention.  This paper hence seeks to understand the spatiotemporal properties of intentional movement and how to capture such intentional movement without relying on challenging human detection and tracking.  We conduct a quantitative analysis of intentional movement, and our findings motivate a new approach for implicit intentional movement extraction that is based on  spatiotemporal trajectory clustering by leveraging the properties of intentional movement.  The intentional movement clusters are then used as action proposals for detection.  Our results on three action detection benchmarks indicate the  relevance of focusing on intentional movement for action detection; our method significantly outperforms the state of the art on the challenging MSR-II multi-action video benchmark.
Simultaneous Foreground Detection and Classification With Hybrid Features
Authors: Jaemyun Kim, Adin Ramirez Rivera, Byungyong Ryu, Oksam Chae
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Classification, General Classification

In this paper, we propose a hybrid background model that relies on edge and non-edge features of the image to produce the model. We encode these features into a coding scheme, that we called Local Hybrid Pattern (LHP), that selectively models edges and non-edges features of each pixel. Furthermore, we model each pixel with an adaptive code dictionary to represent the background dynamism, and update it by adding stable codes and discarding unstable ones. We weight each code in the dictionary to enhance its description of the pixel it models. The foreground is detected as the incoming codes that deviate from the dictionary. We can detect (as foreground or background) and classify (as edge or inner region) each pixel simultaneously. We tested our proposed method in existing databases with promising results.
Opening the Black Box: Hierarchical Sampling Optimization for Estimating Human Hand Pose
Authors: Danhang Tang, Jonathan Taylor, Pushmeet Kohli, Cem Keskin, Tae-Kyun Kim, Jamie Shotton
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Hand Pose Estimation, Image Generation, Pose Estimation

We address the problem of hand pose estimation, formulated as an inverse problem.  Typical approaches optimize an energy function over pose parameters using a `black box' image generation procedure. This procedure knows little about either the relationships between the parameters or the form of the energy function. In this paper, we show that we can significantly improving upon black box optimization by exploiting high-level knowledge of the structure of the parameters and using a local surrogate energy function. Our new framework, hierarchical sampling optimization, consists of a sequence of predictors organized into a kinematic hierarchy.  Each predictor is conditioned on its ancestors, and generates a set of samples over a subset of the pose parameters.  The highly-efficient surrogate energy is used to select among samples.  Having evaluated the full hierarchy, the partial pose samples are concatenated to generate a full-pose hypothesis.  Several hypotheses are generated using the same procedure, and finally the original full energy function selects the best result. Experimental evaluation on three publically available datasets show that our method is particularly impressive in low-compute scenarios where it significantly outperforms all other state-of-the-art methods.