mtybilly/Instruct2DS huggingface dataset:
Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition
Authors: Sihaeng Lee, Junho Yim, Sunjeong Park, Junmo Kim Heechul Jung
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Temporal information has useful features for recognizing facial expressions. However, to manually design useful features requires a lot of effort. In this paper, to reduce this effort, a deep learning technique, which is regarded as a tool to automatically extract useful features from raw data, is adopted. Our deep network is based on two different models. The first deep network extracts temporal appearance features from image sequences, while the other deep network extracts temporal geometry features from temporal facial landmark points. These two models are combined using a new integration method in order to boost the performance of the facial expression recognition. Through several experiments, we show that the two models cooperate with each other. As a result, we achieve superior performance to other state-of-the-art methods in the CK+ and Oulu-CASIA databases. Furthermore, we show that our new integration method gives more accurate results than traditional methods, such as a weighted summation and a feature concatenation method.
Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression
Authors: Michael Maire, Stella X. Yu Takuya Narihira
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We introduce a new approach to intrinsic image decomposition, the task of decomposing a single image into albedo and shading components.  Our strategy, which we term direct intrinsics, is to learn a convolutional neural network (CNN) that directly predicts output albedo and shading channels from an input RGB image patch.  Direct intrinsics is a departure from classical techniques for intrinsic image decomposition, which typically rely on physically-motivated priors and graph-based inference algorithms.  The large-scale synthetic ground-truth of the MPI Sintel dataset plays the key role in training direct intrinsics.  We demonstrate results on both the synthetic images of Sintel and the real images of the classic MIT intrinsic image dataset.  On Sintel, direct intrinsics, using only RGB input, outperforms all prior work, including methods that rely on RGB+Depth input.  Direct intrinsics also generalizes across modalities; our Sintel-trained CNN produces quite reasonable decompositions on the real images of the MIT dataset.  Our results indicate that the marriage of CNNs with synthetic training data may be a powerful new technique for tackling classic problems in computer vision.
Face Flow
Authors: Anastasios Roussos, Yannis Panagakis, Stefanos Zafeiriou Patrick Snape
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In this paper, we propose a method for the robust and efficient computation of  multi-frame optical flow in an expressive sequence of facial images. We formulate a novel energy minimisation problem for establishing dense  correspondences between a neutral template and every frame of a sequence. We exploit the highly correlated nature of human expressions by representing dense facial motion using a deformation basis. Furthermore, we exploit the even higher correlation between deformations in  a given input sequence by imposing a low-rank prior on the coefficients of the  deformation basis, yielding temporally consistent optical flow. Our proposed model-based formulation, in conjunction with the inverse  compositional strategy and low-rank matrix optimisation that we adopt, leads to  a highly efficient algorithm for calculating facial flow.  As experimental evaluation, we show quantitative experiments on a challenging  novel benchmark of face sequences, with dense ground truth optical flow provided by motion  capture data. We also provide qualitative results on a real sequence displaying fast motion and occlusions. Extensive quantitative and qualitative comparisons demonstrate that the proposed method  outperforms state-of-the-art optical flow and dense non-rigid registration techniques,  whilst running an order of magnitude faster.
Discriminative Low-Rank Tracking
Authors: Yafei Tang, Li Zhang Yao Sui
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Good tracking performance is in general attributed to accurate representation over previously obtained targets or reliable discrimination between the target and the surrounding background. In this work, we exploit the advantages of the both approaches to achieve a robust tracker. We construct a subspace to represent the target and the neighboring background, and simultaneously propagate their class labels via the learned subspace. Moreover, we propose a novel criterion to identify the target from numerous target candidates on each frame, which takes into account both discrimination reliability and representation accuracy. In addition, with the proposed criterion, the ambiguity in the class labels of the neighboring background samples, which often influences the reliability of discriminative tracking model, is effectively alleviated, while the training set is still kept small. Extensive experiments demonstrate that our tracker performs favourably against many other state-of-the-art trackers.
SOWP: Spatially Ordered and Weighted Patch Descriptor for Visual Tracking
Authors: Dae-Youn Lee, Jae-Young Sim, Chang-Su Kim Han-Ul Kim
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: A simple yet effective object descriptor for visual tracking is proposed in this paper. We first decompose the bounding box of a target object into multiple patches, which are described by color and gradient histograms. Then, we concatenate the features of the spatially ordered patches to represent the object appearance. Moreover, to alleviate the impacts of background information possibly included in the bounding box, we determine patch weights using random walk with restart (RWR) simulations. The patch weights represent the importance of each patch in the description of foreground information, and are used to construct an object descriptor, called spatially ordered and weighted patch (SOWP) descriptor. We incorporate the proposed SOWP descriptor into the structured output tracking framework. Experimental results demonstrate that the proposed algorithm yields significantly better performance than the state-of-the-art trackers on a recent benchmark dataset, and also excels in another recent benchmark dataset.
Live Repetition Counting
Authors: Lior Wolf Ofir Levy
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: The task of counting the number of repetitions of approximately the same action in an input video sequence is addressed. The proposed method runs online and not on the complete pre-captured video. It analyzes sequentially blocks of 20 non-consecutive frames. The cycle length within each block is evaluated using a convolutional neural network and the information is then integrated over time. The entropy of the network's predictions is used in order to automatically start and stop the repetition counter and to select the appropriate time scale. Coupled with a region of interest detection mechanism, the method is robust enough to handle real world videos, even when the camera is moving. A unique property of our method is that it is shown to successfully train on entirely unrealistic data created by synthesizing moving random patches.
Near-Online Multi-Target Tracking With Aggregated Local Flow Descriptor
Authors: Wongun Choi
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In this paper, we tackle two key aspects of multiple target tracking problem: 1) designing an accurate affinity measure to associate detections and 2) implementing an efficient and accurate (near) online multiple target tracking algorithm. As for the first contribution, we introduce a novel Aggregated Local Flow Descriptor (ALFD) that encodes the relative motion pattern between a pair of temporally distant detections using long term interest point trajectories (IPTs). Leveraging on the IPTs, the ALFD provides a robust affinity measure for estimating the likelihood of matching detections regardless of the application scenarios. As for another contribution, we present a Near-Online Multi-target Tracking (NOMT) algorithm. The tracking problem is formulated as a data-association between targets and detections in a temporal window, that is performed repeatedly at every frame. While being efficient, NOMT achieves robustness via integrating multiple cues including ALFD metric, target dynamics, appearance similarity, and long term trajectory regularization into the model. Our ablative analysis verifies the superiority of the ALFD metric over the other conventional affinity metrics. We run a comprehensive experimental evaluation on two challenging tracking datasets, KITTI and MOT datasets. The NOMT method combined with ALFD metric achieves the best accuracy in both datasets with significant margins (about 10% higher MOTA) over the state-of-the-art.
Multi-Kernel Correlation Filter for Visual Tracking
Authors: Jiayi Feng Ming Tang
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Correlation filter based trackers are ranked top in terms of performances. Nevertheless, they only employ a single kernel at a time. In this paper, we will derive a multi-kernel correlation filter (MKCF) based tracker which fully takes advantage of the invariance-discriminative power spectrums of various features to further improve the performance. Moreover, it may easily introduce location and representation errors to search several discrete scales for the proper one of the object bounding box, because normally the discrete candidate scales are determined and the corresponding feature pyramid are generated ahead of searching. In this paper, we will propose a novel and efficient scale estimation method based on optimal bisection search and fast evaluation of features. Our scale estimation method is the first one that uses the truly minimal number of layers of feature pyramid and avoids constructing the pyramid before searching for proper scales.
Joint Probabilistic Data Association Revisited
Authors: Anton Milan, Zhen Zhang, Qinfeng Shi, Anthony Dick, Ian Reid Seyed Hamid Rezatofighi
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In this paper, we revisit the joint probabilistic data association (JPDA) technique and propose a novel solution based on recent developments in finding the m-best solutions to an integer linear program. The key advantage of this approach is that it makes JPDA computationally tractable in applications with high target and/or clutter density, such as spot tracking in fluorescence microscopy sequences and pedestrian tracking in surveillance footage. We also show that our JPDA algorithm embedded in a simple tracking framework is surprisingly competitive with state-of-the-art global tracking methods in these two applications, while needing considerably less processing time.
Tracking-by-Segmentation With Online Gradient Boosting Decision Tree
Authors: Ilchae Jung, Kayoung Park, Bohyung Han Jeany Son
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We propose an online tracking algorithm that adaptively models target appearances based on an online gradient boosting decision tree. Our algorithm is particularly useful for non-rigid and/or articulated objects since it handles various deformations of the target effectively by integrating a classifier operating on individual patches and provides segmentation masks of the target as final results. The posterior of the target state is propagated over time by particle filtering, where the likelihood is computed based mainly on patch-level confidence map associated with a latent target state corresponding to each sample. Once tracking is completed in each frame, our gradient boosting decision tree is updated to adapt new data in a recursive manner. For effective evaluation of segmentation-based tracking algorithms, we construct a new ground-truth that contains pixel-level annotation of segmentation mask. We evaluate the performance of our tracking algorithm based on the measures for segmentation masks, where our algorithm illustrates superior accuracy compared to the state-of-the-art segmentation-based tracking methods.
Exploring Causal Relationships in Visual Object Tracking
Authors: Simon Hadfield, Richard Bowden Karel Lebeda
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Causal relationships can often be found in visual object tracking between the motions of the camera and that of the tracked object. This object motion may be an effect of the camera motion, e.g. an unsteady handheld camera. But it may also be the cause, e.g. the cameraman framing the ob- ject. In this paper we explore these relationships, and pro- vide statistical tools to detect and quantify them; these are based on transfer entropy and stem from information the- ory. The relationships are then exploited to make predic- tions about the object location. The approach is shown to be an excellent measure for describing such relationships. On the VOT2013 dataset the prediction accuracy is increased by 62 % over the best non-causal predictor. We show that the location predictions are robust to camera shake and sud- den motion, which is invaluable for any tracking algorithm and demonstrate this by applying causal prediction to two state-of-the-art trackers. Both of them benefit, Struck gain- ing a 7 % accuracy and 22 % robustness increase on the VTB1.1 benchmark, becoming the new state-of-the-art.
Hierarchical Convolutional Features for Visual Tracking
Authors: Jia-Bin Huang, Xiaokang Yang, Ming-Hsuan Yang Chao Ma
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Visual object tracking is challenging as target objects often undergo significant appearance changes caused by deformation, abrupt motion, background clutter and occlusion. In this paper, we exploit features extracted from deep convolutional neural networks trained on object recognition datasets to improve tracking accuracy and robustness. The outputs of the last convolutional layers encode the semantic information of targets and such representations are robust to significant appearance variations. However, their spatial resolution is too coarse to precisely localize targets. In contrast, earlier convolutional layers provide more precise localization but are less invariant to appearance changes. We interpret the hierarchies of convolutional layers as a nonlinear counterpart of an image pyramid representation and exploit these multiple levels of abstraction for visual tracking. Specifically, we adaptively learn correlation filters on each convolutional layer to encode the target appearance. We hierarchically infer the maximum response of each layer to locate targets. Extensive experimental results on a largescale benchmark dataset show that the proposed algorithm performs favorably against state-of-the-art methods.
Robust Non-Rigid Motion Tracking and Surface Reconstruction Using L0 Regularization
Authors: Feng Xu, Yangang Wang, Yebin Liu, Qionghai Dai Kaiwen Guo
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We present a new motion tracking method to robustly reconstruct non-rigid geometries and motions from single view depth inputs captured by a consumer depth sensor. The idea comes from the observation of the existence of intrinsic articulated subspace in most of non-rigid motions. To take advantage of this characteristic, we propose a novel L0 based motion regularizer with an iterative optimization solver that can implicitly constrain local deformation only on joints with articulated motions, leading to reduced solution space and physical plausible deformations. The L0 strategy is integrated into the available non-rigid motion tracking pipeline, forming the proposed L0-L2 non-rigid motion tracking method that can adaptively stop the tracking error propagation. Extensive experiments over complex human body motions with occlusions, face and hand motions demonstrate that our approach substantially improves tracking robustness and surface reconstruction accuracy.
Online Object Tracking With Proposal Selection
Authors: Karteek Alahari, Cordelia Schmid Yang Hua
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Tracking-by-detection approaches are some of the most successful object trackers in recent years. Their success is largely determined by the detector model they learn initially and then update over time. However, under challenging conditions where an object can undergo transformations, e.g., severe rotation, these methods are found to be lacking. In this paper, we address this problem by formulating it as a proposal selection task and making two contributions. The first one is introducing novel proposals estimated from the geometric transformations undergone by the object, and building a rich candidate set for predicting the object location. The second one is devising a novel selection strategy using multiple cues, i.e., detection score and edgeness score computed from state-of-the-art object edges and motion boundaries. We extensively evaluate our approach on the visual object tracking 2014 challenge and online tracking benchmark datasets, and show the best performance.
Understanding and Diagnosing Visual Tracking Systems
Authors: Jianping Shi, Dit-Yan Yeung, Jiaya Jia Naiyan Wang
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Several benchmark datasets for visual tracking research have been created in recent years.  Despite their usefulness, whether they are sufficient for understanding and diagnosing the strengths and weaknesses of different trackers remains questionable.  To address this issue, we propose a framework by breaking a tracker down into five constituent parts, namely, motion model, feature extractor, observation model, model updater, and ensemble post-processor.  We then conduct ablative experiments on each component to study how it affects the overall result.  Surprisingly, our findings are discrepant with some common beliefs in the visual tracking research community.  We find that the feature extractor plays the most important role in a tracker.  On the other hand, although the observation model is the focus of many studies, we find that it often brings no significant improvement.  Moreover, the motion model and model updater contain many details that could affect the result.  Also, the ensemble post-processor can improve the result substantially when the constituent trackers have high diversity.  Based on our findings, we put together some very elementary building blocks to give a basic tracker which is competitive in performance to the state-of-the-art trackers.  We believe our framework can provide a solid baseline when conducting controlled experiments for visual tracking research.
Integrating Dashcam Views Through Inter-Video Mapping
Authors: Yi-Ling Chen, Wei-Tse Lee, Fan Wang, Bing-Yu Chen Hsin-I Chen
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In this paper, an inter-video mapping approach is proposed to integrate video footages from two dashcams installed on a preceding and its following vehicle to provide the illusion that the driver of the following vehicle can see-through the preceding one. The key challenge is to adapt the perspectives of the two videos based on a small number of common features since a large portion of the common region in the video captured by the following vehicle is occluded by the preceding one. Inspired by the observation that images with the most similar viewpoints yield dense and high-quality matches, the proposed inter-video mapping estimates spatially-varying motions across two videos utilizing images of very similar contents. Specifically, we estimate frame-to-frame motions of each two consecutive images and incrementally add new views into a merged representation. In this way, long-rang motion estimation is achieved, and the observed perspective discrepancy between the two videos can be well approximated our motion estimation. Once the inter-video mapping is established, the correspondences can be updated incrementally, so the proposed method is suitable for on-line applications. Our experiments demonstrate the effectiveness of our approach on real-world challenging videos.
Visual Tracking With Fully Convolutional Networks
Authors: Wanli Ouyang, Xiaogang Wang, Huchuan Lu Lijun Wang
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We propose a new approach for general object tracking with fully convolutional neural network. Instead of treating convolutional neural network (CNN) as a black-box feature extractor, we conduct in-depth study on the properties of CNN features offline pre-trained on massive image data and classification task on ImageNet. The discoveries motivate the design of our tracking system. It is found that convolutional layers in different levels characterize the target from different perspectives. A top layer encodes more semantic features and serves as a category detector, while a lower layer carries more discriminative information and can better separate the target from distracters with similar appearance. Both layers are jointly used with a switch mechanism during tracking. It is also found that for a tracking target, only a subset of neurons are relevant. A feature map selection method is developed to remove noisy and irrelevant feature maps, which can reduce computation redundancy and improve tracking accuracy. Extensive evaluation on the widely used tracking benchmark shows that the proposed tacker outperforms the state-of-the-art significantly.
Multiple Feature Fusion via Weighted Entropy for Visual Tracking
Authors: Jiwen Lu, Jianjiang Feng, Jie Zhou Lin Ma
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: It is desirable to combine multiple feature descriptors to improve the visual tracking performance because different features can provide complementary information to describe objects of interest. However, how to effectively fuse multiple features remains a challenging problem in visual tracking, especially in a data-driven manner. In this paper, we propose a new data-adaptive visual tracking approach by using multiple feature fusion via weighted entropy. Unlike existing visual trackers which simply concatenate multiple feature vectors together for object representation, we employ the weighted entropy to evaluate the dissimilarity between the object state and the background state, and seek the optimal feature combination by minimizing the weighted entropy, so that more complementary information can be exploited for object representation. Experimental results demonstrate the effectiveness of our approach in tackling various challenges for visual object tracking.
Pedestrian Travel Time Estimation in Crowded Scenes
Authors: Hongsheng Li, Xiaogang Wang Shuai Yi
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In this paper, we target on the problem of estimating the statistic of pedestrian travel time within a period from an entrance to a destination in a crowded scene. Such estimation is based on the global distributions of crowd densities and velocities instead of complete trajectories of pedestrians, which cannot be obtained in crowded scenes. The proposed method is motivated by our statistical investigation into the correlations between travel time and global properties of crowded scenes. Active regions are created for each source-destination pair to model the probable walking regions over the corresponding source-destination traffic flow. Two sets of scene features are specially designed for modeling moving and stationary persons inside the active regions and their influences on pedestrian travel time. The estimation of pedestrian travel time provides valuable  information for both crowd scene understanding and pedestrian behavior analysis, but was not sufficiently studied in literature. The effectiveness of the proposed pedestrian travel time estimation model is demonstrated through several surveillance applications, including dynamic scene monitoring,  localization of regions blocking traffics, and detection of abnormal pedestrian behaviors. Many more valuable applications based on our method are to be explored in the future.
Unsupervised Synchrony Discovery in Human Interaction
Authors: Jiabei Zeng, Fernando De la Torre, Jeffrey F. Cohn, Daniel S. Messinger Wen-Sheng Chu
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: People are inherently social. Social interaction plays an important and natural role in human behavior. Most computational methods focus on individuals alone rather than in social context. They also require labelled training data. We present an unsupervised approach to discover interpersonal synchrony, referred as to two or more persons preforming common actions in overlapping video frames or segments. For computational efficiency, we develop a branch-and-bound (B&B) approach that affords exhaustive search while guaranteeing a globally optimal solution. The proposed method is entirely general. It takes from two or more videos any multi-dimensional signal that can be represented as a histogram. We derive three novel bounding functions and provide efficient extensions, including multi-synchrony detection and accelerated search, using a warm-start strategy and parallelism. We evaluate the effectiveness of our approach in multiple databases, including human actions using the CMU Mocap dataset, spontaneous facial behaviors using group-formation task dataset and parent-infant interaction dataset.
Efficient Video Segmentation Using Parametric Graph Partitioning
Authors: Hieu Le, Gregory Zelinsky, Dimitris Samaras Chen-Ping Yu
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Video segmentation is the task of grouping similar pixels in the spatio-temporal domain, and has become an important preprocessing step for subsequent video analysis. Most video segmentation and supervoxel methods output a hierarchy of segmentations, but while this provides useful multiscale information, it also adds difficulty in selecting the appropriate level for a task. In this work, we propose an efficient and robust video segmentation framework based on parametric graph partitioning (PGP), a fast, almost parameter free graph partitioning method that identifies and removes between-cluster edges to form node clusters. Apart from its computational efficiency, PGP performs clustering of the spatio-temporal volume without requiring a pre-specified cluster number or bandwidth parameters, thus making video segmentation more practical to use in applications. The PGP framework also allows processing sub-volumes, which further improves performance,  contrary to other streaming video segmentation methods where sub-volume processing reduces performance. We evaluate the PGP method using the  SegTrack v2 and Chen Xiph.org datasets, and show that it outperforms related state-of-the-art algorithms in 3D segmentation metrics and running time.
Learning to Track for Spatio-Temporal Action Localization
Authors: Zaid Harchaoui, Cordelia Schmid Philippe Weinzaepfel
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We propose an effective approach for spatio-temporal action localization in realistic videos. The approach first detects proposals at the frame-level and scores them with a combination of static and motion CNN features. It then tracks high-scoring proposals throughout the video using a tracking-by-detection approach. Our tracker relies simultaneously on instance-level and class-level detectors. The tracks are scored using a spatio-temporal motion histogram, a descriptor at the track level, in combination with the CNN features. Finally, we perform temporal localization of the action  using a sliding-window approach at the track level. We present experimental results for spatio-temporal localization on the UCF-Sports, J-HMDB and UCF-101 action localization datasets, where our approach outperforms the state of the art with a margin of 15%, 7% and 12% respectively in mAP.
Unsupervised Object Discovery and Tracking in Video Collections
Authors: Minsu Cho, Ivan Laptev, Jean Ponce, Cordelia Schmid Suha Kwak
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: This paper addresses the problem of automatically localizing dominant objects as spatio-temporal tubes in a noisy collection of videos with minimal or even no supervision. We formulate the problem as a combination of two complementary processes: discovery and tracking. The first one establishes correspondences between prominent regions across videos, and the second one associates similar object regions within the same video. Interestingly, our algorithm also discovers the implicit topology of frames associated with instances of the same object class across different videos, a role normally left to supervisory information in the form of class labels in conventional image and video understanding methods. Indeed, as demonstrated by our experiments, our method can handle video collections featuring multiple object classes, and substantially outperforms the state of the art in colocalization, even though it tackles a broader problem with much less supervision.
Car That Knows Before You Do: Anticipating Maneuvers via Learning Temporal Driving Models
Authors: Hema S. Koppula, Bharad Raghavan, Shane Soh, Ashutosh Saxena Ashesh Jain
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Advanced Driver Assistance Systems (ADAS) have made driving safer over the last decade. They prepare vehicles for  unsafe road conditions and alert drivers if they perform a dangerous maneuver.  However, many accidents are unavoidable because by the time drivers are alerted, it is already too late.  Anticipating maneuvers beforehand can alert drivers before they perform the  maneuver and also give ADAS more time to avoid or prepare for the danger.  In this work we anticipate driving maneuvers a few seconds before they occur. For this purpose we equip a car with cameras and a computing device to capture the driving context from both inside and outside of the car. We propose an Autoregressive Input-Output HMM to model the contextual information alongwith the maneuvers. We evaluate our approach on a  diverse data set with 1180 miles of natural freeway and city driving and show that we can anticipate  maneuvers 3.5 seconds before they occur with over 80% F1-score in real-time.
Activity Auto-Completion: Predicting Human Activities From Partial Videos
Authors: Laiyun Qing, Jun Miao Zhen Xu
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In this paper, we propose an activity auto-completion (AAC) model for human activity prediction by formulating activity prediction as a query auto-completion (QAC) problem in information retrieval. First, we extract discriminative patches in frames of videos. A video is represented based on these patches and divided into a collection of segments, each of which is regarded as a character typed in the search box. Then a partially observed video is considered as an activity prefix, consisting of one or more characters. Finally, the missing observation of an activity is predicted as the activity candidates provided by the auto-completion model. The candidates are matched against the activity prefix on-the-fly and ranked by a learning-to-rank algorithm. We validate our method on UT-Interaction Set #1 and Set #2 [19]. The experimental results show that the proposed activity auto-completion model achieves promising performance.
Person Re-Identification With Correspondence Structure Learning
Authors: Weiyao Lin, Junchi Yan, Mingliang Xu, Jianxin Wu, Jingdong Wang Yang Shen
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: This paper addresses the problem of handling spatial misalignments due to camera-view changes or human-pose variations in person re-identification. We first introduce a boosting-based approach to learn a correspondence structure which indicates the patch-wise matching probabilities between images from a target camera pair. The learned correspondence structure can not only capture the spatial correspondence pattern between cameras but also handle the viewpoint or human-pose variation in individual images. We further introduce a global-based matching process. It integrates a global matching constraint over the learned correspondence structure to exclude cross-view misalignments during the image patch matching process, hence achieving a more reliable matching score between images. Experimental results on various datasets demonstrate the effectiveness of our approach.
Adaptive Exponential Smoothing for Online Filtering of Pixel Prediction Maps
Authors: Jiong Yang, Junsong Yuan Kang Dang
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We propose an efficient online video filtering method, called adaptive exponential filtering (AES) to refine pixel prediction maps. Assuming each pixel is associated with a discriminative prediction score, the proposed AES applies exponentially decreasing weights over time to smooth the prediction score of each pixel, similar to classic exponential smoothing. However, instead of fixing the spatial pixel location to perform temporal filtering, we trace each pixel in the past frames by finding the optimal path that can bring the maximum exponential smoothing score, thus performing adaptive and non-linear filtering. Thanks to the pixel tracing, AES can better address object movements and avoid over-smoothing. To enable real-time filtering, we propose a linear-complexity dynamic programming scheme that can trace all pixels simultaneously. We apply the proposed filtering method to improve both saliency detection maps and scene parsing maps. The comparisons with average and exponential filtering, as well as state-of-the-art methods, validate that our AES can effectively refine the pixel prediction maps, without using the original video again.
P-CNN: Pose-Based CNN Features for Action Recognition
Authors: Ivan Laptev, Cordelia Schmid Guilhem Cheron
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: This work targets human action recognition in video. While recent methods typically represent actions by statistics of local video features, here we argue for the importance of a representation derived from human pose. To this end we propose a new Pose-based Convolutional Neural Network descriptor (P-CNN) for action recognition. The descriptor aggregates motion and appearance information along tracks of human body parts. We investigate different schemes of temporal aggregation and experiment with P-CNN features obtained both for automatically estimated and manually annotated human poses. We evaluate our method on the recent and challenging JHMDB and MPII Cooking datasets. For both datasets our method shows consistent improvement over the state of the art.
Fully Connected Object Proposals for Video Segmentation
Authors: Oliver Wang, Markus Gross, Alexander Sorkine-Hornung Federico Perazzi
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We present a novel approach to video segmentation using multiple object proposals. The problem is formulated as a minimization of a novel energy function defined over a fully connected graph of object proposals. Our model combines appearance with long-range point tracks, which is key to ensure robustness with respect to fast motion and occlusions over longer video sequences. As opposed to previous approaches based on object proposals, we do not seek the best per-frame object hypotheses to perform the segmentation. Instead, we combine multiple, potentially imperfect proposals to improve overall segmentation accuracy and ensure robustness to outliers. Overall, the basic algorithm consists of three steps. First, we generate a very large number of object proposals for each video frame using existing techniques. Next, we perform an SVM-based pruning step to retain only high quality proposals with sufficiently discriminative power. Finally, we determine the fore- and background classification by solving for the maximum a posteriori of a fully connected conditional random field, defined using our novel energy function. Experimental results on a well established dataset demonstrate that our method compares favorably to several recent state-of-the-art approaches.
Video Segmentation With Just a Few Strokes
Authors: Frank R. Schmidt, Thomas Brox Naveen Shankar Nagaraja
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: As the use of videos is becoming more popular in computer vision, the need for annotated video datasets increases.  Such datasets are required either as training data or simply as ground truth for benchmark datasets. A particular challenge in video segmentation is due to disocclusions, which hamper frame-to-frame propagation, in conjunction with non-moving objects. We show that a combination of motion from point trajectories, as known from motion segmentation, along with minimal supervision can largely help solve this problem. Moreover, we integrate a new constraint that enforces consistency of the color distribution in successive frames. We quantify user interaction effort with respect to segmentation quality on challenging ego motion videos.  We compare our approach to a diverse set of algorithms in terms of user effort and in terms of performance on common video segmentation benchmarks.
Actionness-Assisted Recognition of Actions
Authors: Loong-Fah Cheong, An Tran Ye Luo
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We elicit from a fundamental definition of action low-level attributes that can reveal agency and intentionality. These descriptors are mainly trajectory-based, measuring sudden changes, temporal synchrony, and repetitiveness. The actionness map can be used to localize actions in a way that is generic across action and agent types. Furthermore, it also groups interacting regions into a useful unit of analysis, which is crucial for recognition of actions involving interactions. We then implement an actionness-driven pooling scheme to improve action recognition performance. Experimental results on three datasets show the advantages of our method on both action detection and action recognition comparing with other state-of-the-art methods.
COUNT Forest: CO-Voting Uncertain Number of Targets Using Random Forest for Crowd Density Estimation
Authors: Tatsuo Kozakaya, Osamu Yamaguchi, Ryuzo Okada Viet-Quoc Pham
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: This paper presents a patch-based approach for crowd density estimation in public scenes. We formulate the problem of estimating density in a structured learning framework applied to random decision forests. Our approach learns the mapping between patch features and relative locations of all objects inside each patch, which contribute to generate the patch density map through Gaussian kernel density estimation. We build the forest in a coarse-to-fine manner with two split node layers, and further propose a crowdedness prior and an effective forest reduction method to improve the estimation accuracy and speed. Moreover, we introduce a semi-automatic training method to learn the estimator for a specific scene. We achieved state-of-the-art results on the public Mall dataset and UCSD dataset, and also proposed two potential applications in traffic counts and scene understanding with promising results.
Multi-Cue Structure Preserving MRF for Unconstrained Video Segmentation
Authors: Vladimir Pavlovic Saehoon Yi
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Video segmentation is a stepping stone to understanding video context.  Video segmentation enables one to represent a video by decomposing it into coherent regions which comprise whole or parts of objects.  However, the challenge originates from the fact that most of the video segmentation algorithms are based on unsupervised learning due to expensive cost of pixelwise video annotation and intra-class variability within similar unconstrained video classes. We propose a Markov Random Field model for unconstrained video segmentation that relies on tight integration of multiple cues: vertices are defined from contour based superpixels, unary potentials from temporally smooth label likelihood and pairwise potentials from global structure of a video.  Multi-cue structure is a breakthrough to extracting coherent object regions for unconstrained videos in absence of supervision.  Our experiments on VSB100 dataset show that the proposed model significantly outperforms competing state-of-the-art algorithms.  Qualitative analysis illustrates that video segmentation result of the proposed model is consistent with human perception of objects.
Motion Trajectory Segmentation via Minimum Cost Multicuts
Authors: Bjoern Andres, Thomas Brox Margret Keuper
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: For the segmentation of moving objects in videos, the analysis of long-term point trajectories has been very popular recently. In this paper, we formulate the segmentation of a video sequence based on point trajectories as a minimum cost multicut problem. Unlike the commonly used spectral clustering formulation, the minimum cost multicut formulation gives natural rise to optimize not only for a cluster assignment but also for the number of clusters while allowing for varying cluster sizes. In this setup, we provide a method to create a long-term point trajectory graph with attractive and repulsive binary terms and outperform state-of-the-art methods based on spectral clustering on the FBMS-59 dataset and on the motion subtask of the VSB100 dataset.
Action Localization in Videos Through Context Walk
Authors: Haroon Idrees, Mubarak Shah Khurram Soomro
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: This paper presents an efficient approach for localizing actions by learning contextual relations, in the form of relative locations between different video regions. We begin by over-segmenting the videos into supervoxels, which have the ability to preserve action boundaries and also reduce the complexity of the problem. Context relations are learned during training which capture displacements from all the supervoxels in a video to those belonging to foreground actions. Then, given a testing video, we select a supervoxel randomly and use the context information acquired during training to estimate the probability of each supervoxel belonging to the foreground action. The walk proceeds to a new supervoxel and the process is repeated for a few steps. This ``context walk'' generates a conditional distribution of an action over all the supervoxels. A Conditional Random Field is then used to find action proposals in the video, whose confidences are obtained using SVMs. We validated the proposed approach on several datasets and show that context in the form of relative displacements between supervoxels can be extremely useful for action localization. This also results in significantly fewer evaluations of the classifier, in sharp contrast to the alternate sliding window approaches.
RGB-W: When Vision Meets Wireless
Authors: Albert Haque, Li Fei-Fei Alexandre Alahi
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Inspired by the recent success of RGB-D cameras, we propose the enrichment of RGB data with an additional "quasi-free" modality, namely, the wireless signal (e.g., wifi or Bluetooth) emitted by individuals' cell phones, referred to as RGB-W. The received signal strength acts as a rough proxy for depth and a reliable cue on their identity. Although the measured signals are highly noisy (more than 2m average localization error), we demonstrate that the combination of visual and wireless data significantly improves the localization accuracy.   We introduce a novel image-driven representation of wireless data which embeds all received signals onto a single image. We then indicate the ability of this additional data to (i) locate persons within a sparsity-driven framework and to (ii) track individuals with a new confidence measure on the data association problem. Our solution outperforms existing localization methods by a significant margin. It can be applied to the millions of currently installed RGB cameras to better analyze human behavior and offer the next generation of high-accuracy location-based services.
Action Detection by Implicit Intentional Motion Clustering
Authors: Jason J. Corso Wei Chen
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Explicitly using human detection and pose estimation has found limited success in action recognition problems.  This may be due to the complexity in the articulated motion human exhibit.  Yet, we know that action requires an actor and intention.  This paper hence seeks to understand the spatiotemporal properties of intentional movement and how to capture such intentional movement without relying on challenging human detection and tracking.  We conduct a quantitative analysis of intentional movement, and our findings motivate a new approach for implicit intentional movement extraction that is based on  spatiotemporal trajectory clustering by leveraging the properties of intentional movement.  The intentional movement clusters are then used as action proposals for detection.  Our results on three action detection benchmarks indicate the  relevance of focusing on intentional movement for action detection; our method significantly outperforms the state of the art on the challenging MSR-II multi-action video benchmark.
Simultaneous Foreground Detection and Classification With Hybrid Features
Authors: Adin Ramirez Rivera, Byungyong Ryu, Oksam Chae Jaemyun Kim
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In this paper, we propose a hybrid background model that relies on edge and non-edge features of the image to produce the model. We encode these features into a coding scheme, that we called Local Hybrid Pattern (LHP), that selectively models edges and non-edges features of each pixel. Furthermore, we model each pixel with an adaptive code dictionary to represent the background dynamism, and update it by adding stable codes and discarding unstable ones. We weight each code in the dictionary to enhance its description of the pixel it models. The foreground is detected as the incoming codes that deviate from the dictionary. We can detect (as foreground or background) and classify (as edge or inner region) each pixel simultaneously. We tested our proposed method in existing databases with promising results.