OpenRaiser/Intern-Atlas huggingface dataset:
Learning to associate: Hybridboosted multi-target tracker for crowded scene
Authors: Yuan Li 0022, Chang Huang, Ram Nevatia
Venue: Conference CVPR 2009
Year: 2009

We propose a learning-based hierarchical approach of multi-target tracking from a single camera by progressively associating detection responses into longer and longer track fragments (tracklets) and finally the desired target trajectories. To define tracklet affinity for association, most previous work relies on heuristically selected parametric models; while our approach is able to automatically select among various features and corresponding non-parametric models, and combine them to maximize the discriminative power on training data by virtue of a HybridBoost algorithm. A hybrid loss function is used in this algorithm because the association of tracklet is formulated as a joint problem of ranking and classification: the ranking part aims to rank correct tracklet associations higher than other alternatives; the classification part is responsible to reject wrong associations when no further association should be done. Experiments are carried out by tracking pedestrians in challenging datasets. We compare our approach with state-of-the-art algorithms to show its improvement in terms of tracking accuracy.
Untangling cycles for contour grouping
Authors: Qihui Zhu, Gang Song, Jianbo Shi
Venue: Conference ICCV 2007
Year: 2007

We introduce a novel topological formulation for contour grouping. Our grouping criterion, called untangling cycles, exploits the inherent topological 1D structure of salient contours to extract them from the otherwise 2D image clutter. To define a measure for topological classification robust to clutter and broken edges, we use a graph formulation instead of the standard computational topology. The key insight is that a pronounced ID contour should have a clear ordering of edges, to which all graph edges adhere, and no long range entanglements persist. Finding the contour grouping by optimizing these topological criteria is challenging. We introduce a novel concept of circular embedding to encode this combinatorial task. Our solution leads to computing the dominant complex eigenvectors/eigenvalues of the random walk matrix of the contour grouping graph. We demonstrate major improvements over state-of-the-art approaches on challenging real images.
Coarse-to- Fine Low-Rank Structure-From-Motion
Authors: Adrien Bartoli, Vincent Gay-Bellile, Umberto Castellani, Julien Peyras, Søren I. Olsen, Patrick Sayd
Venue: Conference CVPR 2008
Year: 2008

We address the problem of deformable shape and motion recovery from point correspondences in multiple perspective images. We use the low-rank shape model, i.e. the 3D shape is represented as a linear combination of unknown shape bases. We propose a new way of looking at the low-rank shape model. Instead of considering it as a whole, we assume a coarse-to-fine ordering of the deformation modes, which can be seen as a model prior. This has several advantages. First, the high level of ambiguity of the original low-rank shape model is drastically reduced since the shape bases can not anymore be arbitrarily re-combined. Second, this allows us to propose a coarse-to-fine reconstruction algorithm which starts by computing the mean shape and iteratively adds deformation modes. It directly gives the sought after metric model, thereby avoiding the difficult upgrading step required by most of the other methods. Third, this makes it possible to automatically select the number of deformation modes as the reconstruction algorithm proceeds. We propose to incorporate two other priors, accounting for temporal and spatial smoothness, which are shown to improve the quality of the recovered model parameters. The proposed model and reconstruction algorithm are successfully demonstrated on several videos and are shown to outperform the previously proposed algorithms.
Web-Scale Responsive Visual Search at Bing
Authors: Houdong Hu, Yan Wang, Linjun Yang, Pavel Komlev, Li Huang, Xi Stephen Chen, Jiapei Huang, Ye Wu, Meenaz Merchant, Arun Sacheti
Venue: Conference KDD 2018
Year: 2018

In this paper, we introduce a web-scale general visual search system deployed in Microsoft Bing. The system accommodates tens of billions of images in the index, with thousands of features for each image, and can respond in less than 200 ms. In order to overcome the challenges in relevance, latency, and scalability in such large scale of data, we employ a cascaded learning-to-rank framework based on various latest deep learning visual features, and deploy in a distributed heterogeneous computing platform. Quantitative and qualitative experiments show that our system is able to support various applications on Bing website and apps.
Soft edge smoothness prior for alpha channel super resolution
Authors: Shengyang Dai, Mei Han, Wei Xu 0007, Ying Wu 0001, Yihong Gong
Venue: Conference CVPR 2007
Year: 2007

Effective image prior is necessary for image super resolution, due to its severely under-determined nature. Although the edge smoothness prior can be effective, it is generally difficult to have analytical forms to evaluate the edge smoothness, especially for soft edges that exhibit gradual intensity transitions. This paper finds the connection between the soft edge smoothness and a soft cut metric on an image grid by generalizing the Geocuts method (Y. Boykov and V. Kolmogorov, 2003), and proves that the soft edge smoothness measure approximates the average length of all level lines in an intensity image. This new finding not only leads to an analytical characterization of the soft edge smoothness prior, but also gives an intuitive geometric explanation. Regularizing the super resolution problem by this new form of prior can simultaneously minimize the length of all level lines, and thus resulting in visually appealing results. In addition, this paper presents a novel combination of this soft edge smoothness prior and the alpha matting technique for color image super resolution, by normalizing edge segments with their alpha channel description, to achieve a unified treatment of edges with different contrast and scale.
View-invariance in action recognition
Authors: Cen Rao, Mubarak Shah
Venue: Conference CVPR 2001
Year: 2001

Automatically understanding human actions using motion trajectories derived from video sequences is a very challenging problem. Since an action takes place in 3-D, and is projected on 2-D image, depending on the viewpoint of the camera, the projected 2-D trajectory may vary. Therefore, the same action may have very different trajectories, and trajectories of different actions may look the same. This may create a problem in interpretation of trajectories at the higher level. However, if the representation of actions only captures characteristics, which are view-invariant, then the higher level interpretation can proceed without any ambiguity. In most of the current work on action recognition, the issue of view invariance has been ignored. Therefore, proposed methods do not succeed in more general situations. In this paper, we first present a view-invariant representation of action consisting of dynamic instants and intervals, which is computed using spatiotemporal curvature of a trajectory. Then this representation is used by our system to learn human actions without any training. The system is able to incrementally learn different actions starting with no model. It can discover instances of the same action performed by different people, and in different viewpoints.
View-invariant action recognition using fundamental ratios
Authors: Yuping Shen, Hassan Foroosh
Venue: Conference CVPR 2008
Year: 2008

A moving plane observed by a fixed camera induces a fundamental matrix F across multiple frames, where the ratios among the elements in the upper left 2×2 submatrix are herein referred to as the Fundamental Ratios. We show that fundamental ratios are invariant to camera parameters, and hence can be used to identify similar plane motions from varying viewpoints. For action recognition, we decompose a body posture into a set of point triplets (planes). The similarity between two actions is then determined by the motion of point triplets and hence by their associated fundamental ratios, providing thus view-invariant recognition of actions. Results evaluated over 255 semi-synthetic video data with 100 independent trials at a wide range of noise levels, and also on 56 real videos of 8 different classes of actions, confirm that our method can recognize actions under substantial amount of noise, even when they have dynamic timeline maps, and the viewpoints and camera parameters are unknown and totally different.
Alpha estimation in natural images
Authors: Mark A. Ruzon, Carlo Tomasi
Venue: Conference CVPR 2000
Year: 2000

Many boundaries between objects in the world project onto curves in an image. However, boundaries involving natural objects (e.g., trees, hair, water, smoke) are often unworkable under this model because many pixels receive light from more than one object. We propose a technique for estimating alpha, the proportion in which two colors mix to produce a color at the boundary. The technique extends blue screen matting to backgrounds that have almost arbitrary color distributions, though coarse knowledge of the boundary's location is required. Results show a number of different objects moved from one image to another while maintaining naturalism.
Projective factorization of multiple rigid-body motions
Authors: Ting Li, Vinutha Kallem, Dheeraj Singaraju, René Vidal
Venue: Conference CVPR 2007
Year: 2007

Given point correspondences in multiple perspective views of a scene containing multiple rigid-body motions, we present an algorithm for segmenting the correspondences according to the multiple motions. We exploit the fact that when the depths of the points are known, the point trajectories associated with a single motion live in a subspace of dimension at most four. Thus motion segmentation with known depths can be achieved by methods of subspace separation, such as GPCA or LSA. When the depths are unknown, we proceed iteratively. Given the segmentation, we compute the depths using standard techniques. Given the depths, we use GPCA or LSA to segment the scene into multiple motions. Experiments on the Hopkins 155 motion segmentation database show that our method compares favorably against existing affine motion segmentation methods in terms of segmentation error and execution time.
Metapix: Few-shot video retargeting
Authors: Jessica Lee, Deva Ramanan, Rohit Girdhar
Venue: Conference ICLR 2020
Year: 2020

We address the task of unsupervised retargeting of human actions from one video to another. We consider the challenging setting where only a few frames of the target is available. The core of our approach is a conditional generative model that can transcode input skeletal poses (automatically extracted with an off-the-shelf pose estimator) to output target frames. However, it is challenging to build a universal transcoder because humans can appear wildly different due to clothing and background scene geometry. Instead, we learn to adapt - or personalize - a universal generator to the particular human and background in the target. To do so, we make use of meta-learning to discover effective strategies for on-the-fly personalization. One significant benefit of meta-learning is that the personalized transcoder naturally enforces temporal coherence across its generated frames; all frames contain consistent clothing and background geometry of the target. We experiment on in-the-wild internet videos and images and show our approach improves over widely-used baselines for the task.
Real-time indoor scene understanding using bayesian filtering with motion cues
Authors: Grace Tsai, Changhai Xu, Jingen Liu, Benjamin Kuipers
Venue: Conference ICCV 2011
Year: 2011

We present a method whereby an embodied agent using visual perception can efficiently create a model of a local indoor environment from its experience of moving within it. Our method uses motion cues to compute likelihoods of indoor structure hypotheses, based on simple, generic geometric knowledge about points, lines, planes, and motion. We present a single-image analysis, not to attempt to identify a single accurate model, but to propose a set of plausible hypotheses about the structure of the environment from an initial frame. We then use data from subsequent frames to update a Bayesian posterior probability distribution over the set of hypotheses. The likelihood function is efficiently computable by comparing the predicted location of point features on the environment model to their actual tracked locations in the image stream. Our method runs in real-time, and it avoids the need of extensive prior training and the Manhattan-world assumption, which makes it more practical and efficient for an intelligent robot to understand its surroundings compared to most previous scene understanding methods. Experimental results on a collection of indoor videos suggest that our method is capable of an unprecedented combination of accuracy and efficiency.
Probabilistic image segmentation with closedness constraints
Authors: Björn Andres, Jörg H. Kappes, Thorsten Beier, Ullrich Köthe, Fred A. Hamprecht
Venue: Conference ICCV 2011
Year: 2011

We propose a novel graphical model for probabilistic image segmentation that contributes both to aspects of perceptual grouping in connection with image segmentation, and to globally optimal inference with higher-order graphical models. We represent image partitions in terms of cellular complexes in order to make the duality between connected regions and their contours explicit. This allows us to formulate a graphical model with higher-order factors that represent the requirement that all contours must be closed. The model induces a probability measure on the space of all partitions, concentrated on perceptually meaningful segmentations. We give a complete polyhedral characterization of the resulting global inference problem in terms of the multicut polytope and efficiently compute global optima by a cutting plane method. Competitive results for the Berkeley segmentation benchmark confirm the consistency of our approach.
Scalable clustering of signed networks using balance normalized cut
Authors: Kai-Yang Chiang, Joyce Jiyoung Whang, Inderjit S. Dhillon
Venue: Conference CIKM 2012
Year: 2012

We consider the general $k$-way clustering problem in signed social networks where relationships between entities can be either positive or negative. Motivated by social balance theory, the clustering problem in signed networks aims to find mutually antagonistic groups such that entities within the same group are friends with each other. A recent method proposed in [13] extended the spectral clustering algorithm to the signed network setting by considering the signed graph Laplacian. This has been shown to be equivalent to finding clusters that minimize the 2-way signed ratio cut. In this paper, we show that there is a fundamental weakness when we directly extend the signed Laplacian to the k-way clustering problem. To overcome this weakness, we formulate new k-way objectives for signed networks. In particular, we propose a criterion that is analogous to the normalized cut, called balance normalized cut, which is not only theoretically sound but also experimentally effective in k-way clustering. In addition, we prove that these objectives are equivalent to weighted kernel k-means objectives by choosing an appropriate kernel matrix. Employing this equivalence, we develop a multilevel clustering framework for signed networks. In this framework, we coarsen the graph level by level and refine the clustering results at each level via a k-means based algorithm so that the signed clustering objectives are optimized. This approach gives good quality clustering results, and is also highly efficient and scalable. In experiments, we see that our multilevel approach is competitive to other state-of-the-art methods, while it is much faster and more scalable. In particular, the largest graph we have considered in our experiments contains 1 million nodes and 100 million edges --- this graph can be clustered in less than four hundred seconds using our algorithm.
Semi-supervised hierarchical models for 3d human pose reconstruction
Authors: Atul Kanaujia, Cristian Sminchisescu, Dimitris N. Metaxas
Venue: Conference CVPR 2007
Year: 2007

Recent research in visual inference from monocular images has shown that discriminatively trained image-based predictors can provide fast, automatic qualitative 3D reconstructions of human body pose or scene structure in real-world environments. However, the stability of existing image representations tends to be perturbed by deformations and misalignments in the training set, which, in turn, degrade the quality of learning and generalization. In this paper we advocate the semi-supervised learning of hierarchical image descriptions in order to better tolerate variability at multiple levels of detail. We combine multilevel encodings with improved stability to geometric transformations, with metric learning and semi-supervised manifold regularization methods in order to further profile them for task-invariance -resistance to background clutter and within the same human pose class differences. We quantitatively analyze the effectiveness of both descriptors and learning methods and show that each one can contribute, sometimes substantially, to more reliable 3D human pose estimates in cluttered images.
Ransac for (quasi-) degenerate data (qdegsac
Authors: Jan-Michael Frahm, Marc Pollefeys
Venue: Conference CVPR 2006
Year: 2006

The computation of relations from a number of potential matches is a major task in computer vision. Often RANSAC is employed for the robust computation of relations such as the fundamental matrix. For (quasi-)degenerate data however, it often fails to compute the correct relation. The computed relation is always consistent with the data but RANSAC does not verify that it is unique. The paper proposes a framework that estimates the correct relation with the same robustness as RANSAC even for (quasi-)degenerate data. The approach is based on a hierarchical RANSAC over the number of constraints provided by the data. In contrast to all previously presented algorithms for (quasi-)degenerate data our technique does not require problem specific tests or models to deal with degenerate configurations. Accordingly it can be applied for the estimation of any relation on any data and is not limited to a special type of relation as previous approaches. The results are equivalent to the results achieved by state of the art approaches that employ knowledge about degeneracies.
Robust structured light coding for 3d reconstruction
Authors: Chadi Albitar, Pierre Graebling, Christophe Doignon
Venue: Conference ICCV 2007
Year: 2007

In this paper we present a new monochromatic pattern for a robust structured light coding based on the spatial neighborhood scheme using the M-array approach. The proposed pattern is robust as it allows a high error rate characterized by an average Hamming distance higher than 6. We tackle the design problem with the definition of a small set of symbols associated to simple geometrical primitives. One of these primitives embeds the local orientation of the pattern. This is helpful while performing the search for the relevant neighborhood during the decoding process. The aim of this work is to use this pattern for the real-time 3-D reconstruction of dynamic scenes, particularly in endoscopic surgery, with fast and reliable detection and decoding stages. Ongoing results are presented to assess both the capabilities of the proposed pattern and the reliable decoding algorithm with projections onto simple 3-D scenes and onto internal structures of a pig abdomen.
Robust focus ranging
Authors: Hari N. Nair, Charles V. Stewart
Venue: Conference CVPR 1992
Year: 1992

Depth maps obtained from focus ranging can have numerous errors and distortions due to edge bleeding, feature shifts, image noise, and field curvature. An improved algorithm that examines an initial high depth-of-field image of the scene to identify regions susceptible to edge bleeding and image noise is given. Focus evaluation windows are adapted to local image content and optimize the tradeoff between spatial resolution and noise sensitivity. An elliptical paraboloid field curvature model is used to reduce range distortion in peripheral image areas. Spatio-temporal tracking compensates for image feature shifts. The result is a sparse but reliable depth map.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Proximate sensing: Inferring what-is-where from georeferenced photo collections
Authors: Daniel Leung, Shawn D. Newsam
Venue: Conference CVPR 2010
Year: 2010

The primary and novel contribution of this work is the conjecture that large collections of georeferenced photo collections can be used to derive maps of what-is-where on the surface of the earth. We investigate the application of what we term “proximate sensing” to the problem of land cover classification for a large geographic region. We show that our approach is able to achieve almost 75% classification accuracy in a binary land cover labelling problem using images from a photo sharing site in a completely automated fashion. We also investigate 1) how existing geographic knowledge can be used to provide labelled training data in a weakly-supervised manner; 2) the effect of the photographer's intent when he or she captures the photograph; and 3) a method for filtering out non-informative images.
A band-weighted landuse classification method for multispectral images
Authors: Chunhong Pan, Gang Wu 0017, Véronique Prinet, Qing Yang 0002, Songde Ma
Venue: Conference CVPR 2005
Year: 2005

Landuse classification is an important problem in the remote sensing field. It can be used in a wide range of applications. In this paper, we propose a hybrid method fusing edges and regions information for the landuse classification of multispectral images. It mainly includes the steps of image pre-processing, initial segmentation and region merging. Especially, a novel spatial mean shift procedure is proposed so that some information can be extracted and used in the successive steps. Aiming at the multispectral images processing, we also design a band weighting strategy that give a proper weight to each band adoptively according to the region to be processed. Experimental results on the Landsat TM and ETM+ images validate the performance of the proposed method.
Fast articulated motion tracking using a sums of Gaussians body model
Authors: Carsten Stoll, Nils Hasler, Juergen Gall, Hans-Peter Seidel, Christian Theobalt
Venue: Conference ICCV 2011
Year: 2011

We present an approach for modeling the human body by Sums of spatial Gaussians (SoG), allowing us to perform fast and high-quality markerless motion capture from multi-view video sequences. The SoG model is equipped with a color model to represent the shape and appearance of the human and can be reconstructed from a sparse set of images. Similar to the human body, we also represent the image domain as SoG that models color consistent image blobs. Based on the SoG models of the image and the human body, we introduce a novel continuous and differentiable model-to-image similarity measure that can be used to estimate the skeletal motion of a human at 5-15 frames per second even for many camera views. In our experiments, we show that our method, which does not rely on silhouettes or training data, offers an good balance between accuracy and computational cost.