waashk/dblp huggingface dataset:
improving people search using query expansions. in this paper we are interested in finding images of people on the web, and more specifically within large databases of captioned news images. it has recently been shown that visual analysis of the faces in images returned on a text-based query over captions can significantly improve search results. the underlying idea to improve the text-based results is that although this initial result is imperfect, it will render the queried person to be relatively frequent as compared to other people, so we can search for a large group of highly similar faces. the performance of such methods depends strongly on this assumption: for people whose face appears in less than about 40% of the initial text-based result, the performance may be very poor. the contribution of this paper is to improve search results by exploiting faces of other people that co-occur frequently with the queried person. we refer to this process as `query expansion'. in the face analysis we use the query expansion to provide a query-specific relevant set of `negative' examples which should be separated from the potentially positive examples in the text-based result set. we apply this idea to a recently-proposed method which filters the initial result set using a gaussian mixture model, and apply the same idea using a logistic discriminant model. we experimentally evaluate the methods using a set of 23 queries on a database of 15.000 captioned news stories from yahoo! news. the results show that (i) query expansion improves both methods, (ii) that our discriminative models outperform the generative ones, and (iii) our best results surpass the state-of-the-art results by 10% precision on average.
interactive tracking of 2d generic objects with spacetime optimization. we present a continuous optimization framework for interactive tracking of 2d generic objects in a single video stream. the user begins with specifying the locations of a target object in a small set of keyframes; the system then automatically tracks locations of the objects by combining user constraints with visual measurements across the entire sequence. we formulate the problem in a spacetime optimization framework that optimizes over the whole sequence simultaneously. the resulting solution is consistent with visual measurements across the entire sequence while satisfying user constraints. we also introduce prior terms to reduce tracking ambiguity. we demonstrate the power of our algorithm on tracking objects with significant occlusions, scale and orientation changes, illumination changes, sudden movement of objects, and also simultaneous tracking of multiple objects. we compare the performance of our algorithm with alternative methods.
image feature extraction using gradient local auto-correlations. in this paper, we propose a method for extracting image features which utilizes 2nd order statistics, i.e., spatial and orientational auto-correlations of local gradients. it enables us to extract richer information from images and to obtain more discriminative power than standard histogram based methods. the image gradients are sparsely described in terms of magnitude and orientation. in addition, normal vectors on the image surface are derived from the gradients and these could also be utilized instead of the gradients. from a geometrical viewpoint, the method extracts information about not only the gradients but also the curvatures of the image surface. experimental results for pedestrian detection and image patch matching demonstrate the effectiveness of the proposed method compared with other methods, such as hog and sift.
discriminative locality alignment. fisher's linear discriminant analysis (lda), one of the most popular dimensionality reduction algorithms for classification, has three particular problems: it fails to find the nonlinear structure hidden in the high dimensional data; it assumes all samples contribute equivalently to reduce dimension for classification; and it suffers from the matrix singularity problem. in this paper, we propose a new algorithm, termed discriminative locality alignment (dla), to deal with these problems. the algorithm operates in the following three stages: first, in part optimization, discriminative information is imposed over patches, each of which is associated with one sample and its neighbors; then, in sample weighting, each part optimization is weighted by the margin degree, a measure of the importance of a given sample; and finally, in whole alignment, the alignment trick is used to align all weighted part optimizations to the whole optimization. furthermore, dla is extended to the semi-supervised case, i.e., semi-supervised dla (sdla), which utilizes unlabeled samples to improve the classification performance. thorough empirical studies on the face recognition demonstrate the effectiveness of both dla and sdla.
semidefinite programming heuristics for surface reconstruction ambiguities. we consider the problem of reconstructing a smooth surface under constraints that have discrete ambiguities. these problems arise in areas such as shape from texture, shape from shading, photometric stereo and shape from defocus. while the problem is computationally hard, heuristics based on semidefinite programming may reveal the shape of the surface.
tracking of abrupt motion using wang-landau monte carlo estimation. we propose a novel tracking algorithm based on the wang-landau monte carlo sampling method which efficiently deals with the abrupt motions. abrupt motions could cause conventional tracking methods to fail since they violate the motion smoothness constraint. to address this problem, we introduce the wang-landau algorithm that has been recently proposed in statistical physics, and integrate this algorithm into the markov chain monte carlo based tracking method. our tracking method alleviates the motion smoothness constraint utilizing both the likelihood term and the density of states term, which is estimated by the wang-landau algorithm. the likelihood term helps to improve the accuracy in tracking smooth motions, while the density of states term captures abrupt motions robustly. experimental results reveal that our approach efficiently samples the object's states even in a whole state space without loss of time. therefore, it tracks the object of which motion is drastically changing, accurately and robustly.
perspective nonrigid shape and motion recovery. we present a closed form solution to the nonrigid shape and motion (nrsm) problem from point correspondences in multiple perspective uncalibrated views. under the assumption that the nonrigid object deforms as a linear combination of k rigid shapes, we show that the nrsm problem can be viewed as a reconstruction problem from multiple projections from ℙ3k to ℙ2. therefore, one can linearly solve for the projection matrices by factorizing a multifocal tensor. however, this projective reconstruction in ℙ3k does not satisfy the constraints of the nrsm problem, because it is computed only up to a projective transformation in ℙ3k . our key contribution is to show that, by exploiting algebraic dependencies among the entries of the projection matrices, one can upgrade the projective reconstruction to determine the affine configuration of the points in ℝ3, and the motion of the camera relative to their centroid. moreover, if k &ge; 2, then either by using calibrated cameras, or by assuming a camera with fixed internal parameters, it is possible to compute the euclidean structure by a closed form method.
local statistic based region segmentation with automatic scale selection. recently, new segmentation models based on local information have emerged. they combine local statistics of the regions along the contour (inside and outside) to drive the segmentation procedure. since they are based on local decisions, these models are more robust to local variations of the regions of interest (contrast, noise, blur, ...). they nonetheless also introduce some new difficulties which are inherent to the fact of basing a global property (the segmentation) on pure local decisions. this papers explores some of those difficulties and proposes some possible corrections. results on both 2d and 3d data are compared to those obtained without these corrections.
asn: image keypoint detection from adaptive shape neighborhood. we describe an accurate keypoint detector that is stable under viewpoint change. in this paper, keypoints correspond to actual junctions in the image. the principle of asn differs fundamentally from other keypoint detectors. at each position in the image and before any detection, it systematically estimates the position of a potential junction from the local gradient field. keypoints then appear where multiple position estimates are attracted. this approach allows the detector to adapt in shape and size to the image content. one further obtains the area where the keypoint has attracted solutions. comparative results with other detectors show the improved accuracy and stability with viewpoint change.
shadows in three-source photometric stereo. shadows are one of the most significant difficulties of the photometric stereo method. when four or more images are available, local surface orientation is overdetermined and the shadowed pixels can be discarded. in this paper we look at the challenging case when only three images under three different illuminations are available. in this case, when one of the three pixel intensity constraints is missing due to shadow, a 1 dof ambiguity per pixel arises. we show that using integrability one can resolve this ambiguity and use the remaining two constraints to reconstruct the geometry in the shadow regions. as the problem becomes ill-posed in the presence of noise, we describe a regularization scheme that improves the numerical performance of the algorithm while preserving data. we propose a simple mrf optimization scheme to identify and segment shadow regions in the image. finally the paper describes how this theory applies in the framework of color photometric stereo where one is restricted to only three images. experiments on synthetic and real image sequences are presented.
learning to recognize activities from the wrong view point. appearance features are good at discriminating activities in a fixed view, but behave poorly when aspect is changed. we describe a method to build features that are highly stable under change of aspect. it is not necessary to have multiple views to extract our features. our features make it possible to learn a discriminative model of activity in one view, and spot that activity in another view, for which one might poses no labeled examples at all. our construction uses labeled examples to build activity models, and unlabeled, but corresponding, examples to build an implicit model of how appearance changes with aspect. we demonstrate our method with challenging sequences of real human motion, where discriminative methods built on appearance alone fail badly.
discriminative learning for deformable shape segmentation: a comparative study. we present a comparative study on how to use discriminative learning methods such as classification, regression, and ranking to address deformable shape segmentation. traditional generative models and energy minimization methods suffer from local minima. by casting the segmentation into a discriminative framework, the target fitting function can be steered to possess a desired shape for ease of optimization yet better characterize the relationship between shape and appearance. to address the high-dimensional learning challenge present in the learning framework, we use a multi-level approach to learning discriminative models. our experimental results on left ventricle segmentation from ultrasound images and facial feature point localization demonstrate that the discriminative models outperform generative models and energy minimization methods by a large margin.
a perceptual comparison of distance measures for color constancy algorithms. color constancy is the ability to measure image features independent of the color of the scene illuminant and is an important topic in color and computer vision. as many color constancy algorithms exist, different distance measures are used to compute their accuracy. in general, these distances measures are based on mathematical principles such as the angular error and euclidean distance. however, it is unknown to what extent these distance measures correlate to human vision.therefore, in this paper, a taxonomy of different distance measures for color constancy algorithms is presented. the main goal is to analyze the correlation between the observed quality of the output images and the different distance measures for illuminant estimates. the output images are the resulting color corrected images using the illuminant estimates of the color constancy algorithms, and the quality of these images is determined by human observers. distance measures are analyzed how they mimic differences in color naturalness of images as obtained by humans.based on the theoretical and experimental results on spectral and real-world data sets, it can be concluded that the perceptual euclidean distance (ped) with weight-coefficients (w r = 0.26, w g = 0.70, w b = 0.04) finds its roots in human vision and correlates significantly higher than all other distance measures including the angular error and euclidean distance.
edge-preserving smoothing and mean-shift segmentation of video streams. video streams are ubiquitous in applications such as surveillance, games, and live broadcast. processing and analyzing these data is challenging because algorithms have to be efficient in order to process the data on the fly. from a theoretical standpoint, video streams have their own specificities --- they mix spatial and temporal dimensions, and compared to standard video sequences, half of the information is missing, i.e. the future is unknown. the theoretical part of our work is motivated by the ubiquitous use of the gaussian kernel in tools such as bilateral filtering and mean-shift segmentation. we formally derive its equivalent for video streams as well as a dedicated expression of isotropic diffusion. building upon this theoretical ground, we adapt a number of classical algorithms to video streams: bilateral filtering, mean-shift segmentation, and anisotropic diffusion.
robust 3d pose estimation and efficient 2d region-based segmentation from a 3d shape prior. in this work, we present an approach to jointly segment a rigid object in a 2d image and estimate its 3d pose, using the knowledge of a 3d model. we naturally couple the two processes together into a unique energy functional that is minimized through a variational approach. our methodology differs from the standard monocular 3d pose estimation algorithms since it does not rely on local image features. instead, we use global image statistics to drive the pose estimation process. this confers a satisfying level of robustness to noise and initialization for our algorithm, and bypasses the need to establish correspondences between image and object features. moreover, our methodology possesses the typical qualities of region-based active contour techniques with shape priors, such as robustness to occlusions or missing information, without the need to evolve an infinite dimensional curve. another novelty of the proposed contribution is to use a unique 3d model surface of the object, instead of learning a large collection of 2d shapes to accommodate for the diverse aspects that a 3d object can take when imaged by a camera. experimental results on both synthetic and real images are provided, which highlight the robust performance of the technique on challenging tracking and segmentation applications.
fast and accurate rotation estimation on the 2-sphere without correspondences. we present a refined method for rotation estimation of signals on the 2-sphere. our approach utilizes a fast correlation in the harmonic domain to estimate rotation angles of arbitrary size and resolution. the method is able to achieve great accuracy even for very low spherical harmonic expansions of the input signals without using correspondences or any other kind of a priori information. the rotation parameters are computed analytically without additional iterative post-processing or "fine tuning".the theoretical advances presented in this paper can be applied to a wide range of practical problems such as: shape description and shape retrieval, 3d rigid registration, robot positioning with omni-directional cameras or 3d invariant feature design.
the naked truth: estimating body shape under clothing. we propose a method to estimate the detailed 3d shape of a person from images of that person wearing clothing. the approach exploits a model of human body shapes that is learned from a database of over 2000 range scans. we show that the parameters of this shape model can be recovered independently of body pose. we further propose a generalization of the visual hull to account for the fact that observed silhouettes of clothed people do not provide a tight bound on the true 3d shape. with clothed subjects, different poses provide different constraints on the possible underlying 3d body shape. we consequently combine constraints across pose to more accurately estimate 3d body shape in the presence of occluding clothing. finally we use the recovered 3d shape to estimate the gender of subjects and then employ gender-specific body models to refine our shape estimates. results on a novel database of thousands of images of clothed and "naked" subjects, as well as sequences from the humaneva dataset, suggest the method may be accurate enough for biometric shape analysis in video.
robust real-time visual tracking using pixel-wise posteriors. we derive a probabilistic framework for robust, real-time, visual tracking of previously unseen objects from a moving camera. the tracking problem is handled using a bag-of-pixels representation and comprises a rigid registration between frames, a segmentation and online appearance learning. the registration compensates for rigid motion, segmentation models any residual shape deformation and the online appearance learning provides continual refinement of both the object and background appearance models. the key to the success of our method is the use of pixel-wise posteriors, as opposed to likelihoods. we demonstrate the superior performance of our tracker by comparing cost function statistics against those commonly used in the visual tracking literature. our comparison method provides a way of summarising tracking performance using lots of data from a variety of different sequences.
contour context selection for object detection: a set-to-set contour matching approach. we introduce a shape detection framework called contour context selection for detecting objects in cluttered images using only one exemplar. shape based detection is invariant to changes of object appearance, and can reason with geometrical abstraction of the object. our approach uses salient contours as integral tokens for shape matching. we seek a maximal, holistic matching of shapes, which checks shape features from a large spatial extent, as well as long-range contextual relationships among object parts. this amounts to finding the correct figure/ground contour labeling, and optimal correspondences between control points on/around contours. this removes accidental alignments and does not hallucinate objects in background clutter, without negative training examples. we formulate this task as a set-to-set contour matching problem. naive methods would require searching over 'exponentially' many figure/ground contour labelings. we simplify this task by encoding the shape descriptor algebraically in a linear form of contour figure/ground variables. this allows us to use the reliable optimization technique of linear programming. we demonstrate our approach on the challenging task of detecting bottles, swans and other objects in cluttered images.
a graph based subspace semi-supervised learning framework for dimensionality reduction. the key to the graph based semi-supervised learning algorithms for classification problems is how to construct the weight matrix of the p-nearest neighbor graph. a new method to construct the weight matrix is proposed and a graph based subspace semi-supervised learning framework (sslf) is developed. the framework aims to find an embedding transformation which respects the discriminant structure inferred from the labeled data, as well as the intrinsic geometrical structure inferred from both the labeled and unlabeled data. by utilizing this framework as a tool, we drive three semi-supervised dimensionality reduction algorithms: subspace semi-supervised linear discriminant analysis (sslda), subspace semi-supervised locality preserving projection (sslpp), and subspace semi-supervised marginal fisher analysis (ssmfa). the experimental results on face recognition demonstrate our subspace semi-supervised algorithms are able to use unlabeled samples effectively.
temporal surface tracking using mesh evolution. in this paper, we address the problem of surface tracking in multiple camera environments and over time sequences. in order to fully track a surface undergoing significant deformations, we cast the problem as a mesh evolution over time. such an evolution is driven by 3d displacement fields estimated between meshes recovered independently at different time frames. geometric and photometric information is used to identify a robust set of matching vertices. this provides a sparse displacement field that is densified over the mesh by laplacian diffusion. in contrast to existing approaches that evolve meshes, we do not assume a known model or a fixed topology. the contribution is a novel mesh evolution based framework that allows to fully track, over long sequences, an unknown surface encountering deformations, including topological changes. results on very challenging and publicly available image based 3d mesh sequences demonstrate the ability of our framework to efficiently recover surface motions .
2d image analysis by generalized hilbert transforms in conformal space. this work presents a novel rotational invariant quadrature filter approach - called the conformal monogenic signal - for analyzing i(ntrinsic)1d and i2d local features of any curved 2d signal such as lines, edges, corners and junctions without the use of steering. the conformal monogenic signal contains the monogenic signal as a special case for i1d signals and combines monogenic scale space, phase, direction/orientation, energy and curvature in one unified algebraic framework. the conformal monogenic signal will be theoretically illustrated and motivated in detail by the relation of the 3d radon transform and the generalized hilbert transform on the sphere. the main idea is to lift up 2d signals to the higher dimensional conformal space where the signal features can be analyzed with more degrees of freedom. results of this work are the low computational time complexity, the easy implementation into existing computer vision applications and the numerical robustness of determining curvature without the need of any derivatives.