OpenRaiser/Intern-Atlas huggingface dataset:
Unsupervised metric learning for face identification in TV video
Authors: Ramazan Gokberk Cinbis, Jakob Verbeek, Cordelia Schmid
Venue: ICCV
Year: 2011

The goal of face identification is to decide whether two faces depict the same person or not. This paper addresses the identification problem for face-tracks that are automatically collected from uncontrolled TV video data. Face-track identification is an important component in systems that automatically label characters in TV series or movies based on subtitles and/or scripts: it enables effective transfer of the sparse text-based supervision to other faces. We show that, without manually labeling any examples, metric learning can be effectively used to address this problem. This is possible by using pairs of faces within a track as positive examples, while negative training examples can be generated from pairs of face tracks of different people that appear together in a video frame. In this manner we can learn a cast-specific metric, adapted to the people appearing in a particular video, without using any supervision. Identification performance can be further improved using semi-supervised learning where we also include labels for some of the face tracks. We show that our cast-specific metrics not only improve identification, but also recognition and clustering.
A FACS valid 3D dynamic action unit database with applications to 3D dynamic morphable facial modeling
Authors: Darren Cosker, Eva Krumhuber, Adrian Hilton 0001
Venue: ICCV
Year: 2011

This paper presents the first dynamic 3D FACS data set for facial expression research, containing 10 subjects performing between 19 and 97 different AUs both individually and in combination. In total the corpus contains 519 AU sequences. The peak expression frame of each sequence has been manually FACS coded by certified FACS experts. This provides a ground truth for 3D FACS based AU recognition systems. In order to use this data, we describe the first framework for building dynamic 3D morphable models. This includes a novel Active Appearance Model (AAM) based 3D facial registration and mesh correspondence scheme. The approach overcomes limitations in existing methods that require facial markers or are prone to optical flow drift. We provide the first quantitative assessment of such 3D facial mesh registration techniques and show how our proposed method provides more reliable correspondence.
Linear stereo matching
Authors: Leonardo De-Maeztu, Stefano Mattoccia, Arantxa Villanueva, Rafael Cabeza
Venue: ICCV
Year: 2011

Recent local stereo matching algorithms based on an adaptive-weight strategy achieve accuracy similar to global approaches. One of the major problems of these algorithms is that they are computationally expensive and this complexity increases proportionally to the window size. This paper proposes a novel cost aggregation step with complexity independent of the window size (i.e. O(1)) that outperforms state-of-the-art O(1) methods. Moreover, compared to other O(1) approaches, our method does not rely on integral histograms enabling aggregation using colour images instead of grayscale ones. Finally, to improve the results of the proposed algorithm a disparity refinement pipeline is also proposed. The overall algorithm produces results comparable to those of state-of-the-art stereo matching algorithms.
Source camera identification using Auto-White Balance approximation
Authors: Zhonghai Deng, Arjan Gijsenij, Jingyuan Zhang
Venue: ICCV
Year: 2011

Source camera identification finds many applications in real world. Although many identification methods have been proposed, they work with only a small set of cameras, and are weak at identifying cameras of the same model. Based on the observation that a digital image would not change if the same Auto-White Balance (AWB) algorithm is applied for the second time, this paper proposes to identify the source camera by approximating the AWB algorithm used inside the camera. To the best of our knowledge, this is the first time that a source camera identification method based on AWB has been reported. Experiments show near perfect accuracy in identifying cameras of different brands and models. Besides, proposed method performances quite well in distinguishing among camera devices of the same model, as AWB is done at the end of imaging pipeline, any small differences induced earlier will lead to different types of AWB output. Furthermore, the performance remains stable as the number of cameras grows large.
Dynamic fluid surface acquisition using a camera array
Authors: Yuanyuan Ding, Feng Li 0005, Yu Ji 0001, Jingyi Yu 0001
Venue: ICCV
Year: 2011

Acquiring dynamic 3D fluid surfaces is a challenging problem in computer vision. Single or stereo camera based solutions are sensitive to refraction distortions, fast fluid motions, and calibration errors. In this paper, we present a multi-view based solution for robustly capturing fast evolving fluid wavefronts. We first construct a portable, 3×3 camera array system as the main acquisition device. We elaborately design the system to allow high-resolution and high-speed capture. To recover fluid surfaces, we place a known pattern beneath the surface and position the camera array on top to observe the pattern. By tracking the distorted feature points over time and across cameras, we obtain spatial-temporal correspondence maps and we use them for specular carving to reconstruct the time-varying surface. In case one of the cameras loses track due to distortions or blurs, we use the rest of the cameras to construct the surface and then apply multi-perspective warping to locate the lost-track feature points so that we can continue using the camera in later frames. Our experiments on synthetic and real data demonstrate that our multi-view framework is robust and reliable.
Centralized sparse representation for image restoration
Authors: Weisheng Dong, Lei Zhang 0006, Guangming Shi
Venue: ICCV
Year: 2011

This paper proposes a novel sparse representation model called centralized sparse representation (CSR) for image restoration tasks. In order for faithful image reconstruction, it is expected that the sparse coding coefficients of the degraded image should be as close as possible to those of the unknown original image with the given dictionary. However, since the available data are the degraded (noisy, blurred and/or down-sampled) versions of the original image, the sparse coding coefficients are often not accurate enough if only the local sparsity of the image is considered, as in many existing sparse representation models. To make the sparse coding more accurate, a centralized sparsity constraint is introduced by exploiting the nonlocal image statistics. The local sparsity and the nonlocal sparsity constraints are unified into a variational framework for optimization. Extensive experiments on image restoration validated that our CSR model achieves convincing improvement over previous state-of-the-art methods.
Tasting families of features for image classification
Authors: Charles Dubout, François Fleuret
Venue: ICCV
Year: 2011

Using multiple families of image features is a very efficient strategy to improve performance in object detection or recognition. However, such a strategy induces multiple challenges for machine learning methods, both from a computational and a statistical perspective. The main contribution of this paper is a novel feature sampling procedure dubbed "Tasting" to improve the efficiency of Boosting in such a context. Instead of sampling features in a uniform manner, Tasting continuously estimates the expected loss reduction for each family from a limited set of features sampled prior to the learning, and biases the sampling accordingly. We evaluate the performance of this procedure with tens of families of features on four image classification and object detection data-sets. We show that Tasting, which does not require the tuning of any meta-parameter, outperforms systematically variants of uniform sampling and state-of- the-art approaches based on bandit strategies.
Towards accurate and efficient representation of image irradiance of convex-Lambertian objects under unknown near lighting
Authors: Shireen Y. Elhabian, Ham M. Rara, Aly A. Farag
Venue: ICCV
Year: 2011

Surface irradiance signals are turned into outgoing radiance through the surface reflectance function, which can be significantly perturbed by the illumination conditions. Due to their low-frequency nature, irradiance signals can be represented using low-order basis functions, where spherical harmonics (SH) have been extensively used to provide such basis. When capturing image irradiance from a single viewpoint, the visible part of the object's surface constructs the upper hemisphere of the surface normals where the SH are no longer orthonormal. This reduced domain paves the way for even lower-dimensional approximation since full spherical representation is not needed. While harmonic basis are known to be optimal under distant light, light coming from near-by objects and indoor environments are common near light scenarios; it is essential to relax distant light assumption. Considering light source(s) distributed uniformly over the upper hemisphere, we propose the use of hemispherical harmonics (HSH) to model image irradiance of convex Lambertian objects perceived from single viewpoint under unknown near illumination. We prove analytically, and experimentally validated, that the Lambertian kernel has a more compact harmonic expansion in the hemispherical domain when compared to its spherical counterpart. We illustrate that HSH provide an efficient and accurate low-dimensional representation of image irradiance of Lambertian objects under near lighting conditions in contrast to SH.
Learning nonlinear distance functions using neural network for regression with application to robust human age estimation
Authors: Na Fan
Venue: ICCV
Year: 2011

In this paper, a robust regression method is proposed for human age estimation, in which, outlier samples are corrected by their neighbors, through asymptotically increasing the correlation coefficients between the desired distances and the distances of sample labels. As another extension, we adopt a nonlinear distance function and approximate it by neural network. For fair comparison, we also experiment on the regression problem of age estimation from face images, and the results are very competitive among the state of the art.
A graph cut algorithm for higher-order Markov Random Fields
Authors: Alexander Fix, Aritanan Gruber, Endre Boros, Ramin Zabih
Venue: ICCV
Year: 2011

Higher-order Markov Random Fields, which can capture important properties of natural images, have become increasingly important in computer vision. While graph cuts work well for first-order MRF's, until recently they have rarely been effective for higher-order MRF's. Ishikawa's graph cut technique [8, 9] shows great promise for many higher-order MRF's. His method transforms an arbitrary higher-order MRF with binary labels into a first-order one with the same minima. If all the terms are submodular the exact solution can be easily found; otherwise, pseudo-boolean optimization techniques can produce an optimal labeling for a subset of the variables. We present a new transformation with better performance than [8, 9], both theoretically and experimentally. While [8, 9] transforms each higher-order term independently, we transform a group of terms at once. For n binary variables, each of which appears in terms with k other variables, at worst we produce n non-submodular terms, while [8, 9] produces O(nk). We identify a local completeness property that makes our method perform even better, and show that under certain assumptions several important vision problems (including common variants of fusion moves) have this property. Running on the same field of experts dataset used in [8, 9] we optimally label significantly more variables (96% versus 80%) and converge more rapidly to a lower energy. Preliminary experiments suggest that some other higher-order MRF's used in stereo [20] and segmentation [1] are also locally complete and would thus benefit from our work.
Spatio-temporal clustering of probabilistic region trajectories
Authors: Fabio Galasso, Masahiro Iwasaki, Kunio Nobori, Roberto Cipolla
Venue: ICCV
Year: 2011

We propose a novel model for the spatio-temporal clustering of trajectories based on motion, which applies to challenging street-view video sequences of pedestrians captured by a mobile camera. A key contribution of our work is the introduction of novel probabilistic region trajectories, motivated by the non-repeatability of segmentation of frames in a video sequence. Hierarchical image segments are obtained by using a state-of-the-art hierarchical segmentation algorithm, and connected from adjacent frames in a directed acyclic graph. The region trajectories and measures of confidence are extracted from this graph using a dynamic programming-based optimisation. Our second main contribution is a Bayesian framework with a twofold goal: to learn the optimal, in a maximum likelihood sense, Random Forests classifier of motion patterns based on video features, and construct a unique graph from region trajectories of different frames, lengths and hierarchical levels. Finally, we demonstrate the use of Isomap for effective spatio-temporal clustering of the region trajectories of pedestrians. We support our claims with experimental results on new and existing challenging video sequences.
Discriminative learning of relaxed hierarchy for large-scale visual recognition
Authors: Tianshi Gao, Daphne Koller
Venue: ICCV
Year: 2011

In the real visual world, the number of categories a classifier needs to discriminate is on the order of hundreds or thousands. For example, the SUN dataset [24] contains 899 scene categories and ImageNet [6] has 15,589 synsets. Designing a multiclass classifier that is both accurate and fast at test time is an extremely important problem in both machine learning and computer vision communities. To achieve a good trade-off between accuracy and speed, we adopt the relaxed hierarchy structure from [15], where a set of binary classifiers are organized in a tree or DAG (directed acyclic graph) structure. At each node, classes are colored into positive and negative groups which are separated by a binary classifier while a subset of confusing classes is ignored. We color the classes and learn the induced binary classifier simultaneously using a unified and principled max-margin optimization. We provide an analysis on generalization error to justify our design. Our method has been tested on both Caltech-256 (object recognition) [9] and the SUN dataset (scene classification) [24], and shows significant improvement over existing methods.
A "string of feature graphs" model for recognition of complex activities in natural videos
Authors: Utkarsh Gaur, Yingying Zhu 0002, Bi Song, Amit K. Roy-Chowdhury
Venue: ICCV
Year: 2011

Videos usually consist of activities involving interactions between multiple actors, sometimes referred to as complex activities. Recognition of such activities requires modeling the spatio-temporal relationships between the actors and their individual variabilities. In this paper, we consider the problem of recognition of complex activities in a video given a query example. We propose a new feature model based on a string representation of the video which respects the spatio-temporal ordering. This ordered arrangement of local collections of features (e.g., cuboids, STIP), which are the characters in the string, are initially matched using graph-based spectral techniques. Final recognition is obtained by matching the string representations of the query and the test videos in a dynamic programming framework which allows for variability in sampling rates and speed of activity execution. The method does not require tracking or recognition of body parts, is able to identify the region of interest in a cluttered scene, and gives reasonable performance with even a single query example. We test our approach in an example-based video retrieval framework with two publicly available complex activity datasets and provide comparisons against other methods that have studied this problem.
iGroup: Weakly supervised image and video grouping
Authors: Andrew Gilbert, Richard Bowden
Venue: ICCV
Year: 2011

We present a generic, efficient and iterative algorithm for interactively clustering classes of images and videos. The approach moves away from the use of large hand labelled training datasets, instead allowing the user to find natural groups of similar content based upon a handful of “seed” examples. Two efficient data mining tools originally developed for text analysis; min-Hash and APriori are used and extended to achieve both speed and scalability on large image and video datasets. Inspired by the Bag-of-Words (BoW) architecture, the idea of an image signature is introduced as a simple descriptor on which nearest neighbour classification can be performed. The image signature is then dynamically expanded to identify common features amongst samples of the same class. The iterative approach uses APriori to identify common and distinctive elements of a small set of labelled true and false positive signatures. These elements are then accentuated in the signature to increase similarity between examples and “pull” positive classes together. By repeating this process, the accuracy of similarity increases dramatically despite only a few training examples, only 10% of the labelled groundtruth is needed, compared to other approaches. It is tested on two image datasets including the caltech101 [9] dataset and on three state-of-the-art action recognition datasets. On the YouTube [18] video dataset the accuracy increases from 72% to 97% using only 44 labelled examples from a dataset of over 1200 videos. The approach is both scalable and efficient, with an iteration on the full YouTube dataset taking around 1 minute on a standard desktop machine.
Efficient regression of general-activity human poses from depth images
Authors: Ross B. Girshick, Jamie Shotton, Pushmeet Kohli, Antonio Criminisi, Andrew W. Fitzgibbon
Venue: ICCV
Year: 2011

We present a new approach to general-activity human pose estimation from depth images, building on Hough forests. We extend existing techniques in several ways: real time prediction of multiple 3D joints, explicit learning of voting weights, vote compression to allow larger training sets, and a comparison of several decision-tree training objectives. Key aspects of our work include: regression directly from the raw depth image, without the use of an arbitrary intermediate representation; applicability to general motions (not constrained to particular activities) and the ability to localize occluded as well as visible body joints. Experimental results demonstrate that our method produces state of the art results on several data sets including the challenging MSRC-5000 pose estimation test set, at a speed of about 200 frames per second. Results on silhouettes suggest broader applicability to other imaging modalities.
Viewpoint-aware object detection and pose estimation
Authors: Daniel Glasner, Meirav Galun, Sharon Alpert, Ronen Basri, Gregory Shakhnarovich
Venue: ICCV
Year: 2011

We describe an approach to category-level detection and viewpoint estimation for rigid 3D objects from single 2D images. In contrast to many existing methods, we directly integrate 3D reasoning with an appearance-based voting architecture. Our method relies on a nonparametric representation of a joint distribution of shape and appearance of the object class. Our voting method employs a novel parametrization of joint detection and viewpoint hypothesis space, allowing efficient accumulation of evidence. We combine this with a re-scoring and refinement mechanism, using an ensemble of view-specific Support Vector Machines. We evaluate the performance of our approach in detection and pose estimation of cars on a number of benchmark datasets.
Hough-based tracking of non-rigid objects
Authors: Martin Godec, Peter M. Roth, Horst Bischof
Venue: ICCV
Year: 2011

Online learning has shown to be successful in tracking of previously unknown objects. However, most approaches are limited to a bounding-box representation with fixed aspect ratio. Thus, they provide a less accurate fore- ground/background separation and cannot handle highly non-rigid and articulated objects. This, in turn, increases the amount of noise introduced during online self-training. In this paper, we present a novel tracking-by-detection approach to overcome this limitation based on the generalized Hough-transform. We extend the idea of Hough Forests to the online domain and couple the voting- based detection and back-projection with a rough segmentation based on GrabCut. This significantly reduces the amount of noisy training samples during online learning and thus effectively prevents the tracker from drifting. In the experiments, we demonstrate that our method successfully tracks a variety of previously unknown objects even under heavy non-rigid transformations, partial occlusions, scale changes and rotations. Moreover, we compare our tracker to state-of-the-art methods (both bounding-box- based as well as part-based) and show robust and accurate tracking results on various challenging sequences.
Introducing total curvature for image processing
Authors: Bastian Goldlücke, Daniel Cremers
Venue: ICCV
Year: 2011

We introduce the novel continuous regularizer total curvature (TC) for images u: Ω → ℝ. It is defined as the Menger-Melnikov curvature of the Radon measure |Du|, which can be understood as a measure theoretic formulation of curvature mathematically related to mean curvature. The functional is not convex, therefore we define a convex relaxation which yields a close approximation. Similar to the total variation, the relaxation can be written as the support functional of a convex set, which means that there are stable and efficient minimization algorithms available when it is used as a regularizer in image processing problems. Our current implementation can handle general inverse problems, inpainting and segmentation. We demonstrate in experiments and comparisons how the regularizer performs in practice.
Multi-hypothesis motion planning for visual object tracking
Authors: Haifeng Gong, Jack Sim, Maxim Likhachev, Jianbo Shi
Venue: ICCV
Year: 2011

In this paper, we propose a long-term motion model for visual object tracking. In crowded street scenes, persistent occlusions are a frequent challenge for tracking algorithm and a robust, long-term motion model could help in these situations. Motivated by progresses in robot motion planning, we propose to construct a set of `plausible' plans for each person, which are composed of multiple long-term motion prediction hypotheses that do not include redundancies, unnecessary loops or collisions with other objects. Constructing plausible plan is the key step in utilizing motion planning in object tracking, which has not been fully investigate in robot motion planning. We propose a novel method of efficiently constructing disjoint plans in different homotopy classes, based on winding numbers and winding angles of planned paths around all obstacles. As the goals can be specified by winding numbers and winding angles, we can avoid redundant plans in the same homotopy class and multiple whirls or loops around a single obstacle. We test our algorithm on a challenging, real-world dataset, and compare our algorithm with Linear Trajectory Avoidance and a simplified linear planning model. We find that our algorithm outperforms both algorithms in most sequences.