waashk/dblp huggingface dataset:
pose priors for simultaneously solving alignment and correspondence. estimating a camera pose given a set of 3d-object and 2d-image feature points is a well understood problem when correspondences are given. however, when such correspondences cannot be established a priori, one must simultaneously compute them along with the pose. most current approaches to solving this problem are too computationally intensive to be practical. an interesting exception is the softposit algorithm, that looks for the solution as the minimum of a suitable objective function. it is arguably one of the best algorithms but its iterative nature means it can fail in the presence of clutter, occlusions, or repetitive patterns. in this paper, we propose an approach that overcomes this limitation by taking advantage of the fact that, in practice, some prior on the camera pose is often available. we model it as a gaussian mixture model that we progressively refine by hypothesizing new correspondences. this rapidly reduces the number of potential matches for each 3d point and lets us explore the pose space more thoroughly than softposit at a similar computational cost. we will demonstrate the superior performance of our approach on both synthetic and real data.
facial expression recognition based on 3d dynamic range model sequences. traditionally, facial expression recognition (fer) issues have been studied mostly based on modalities of 2d images, 2d videos, and 3d static models. in this paper, we propose a spatio-temporal expression analysis approach based on a new modality, 3d dynamic geometric facial model sequences, to tackle the fer problems. our approach integrates a 3d facial surface descriptor and hidden markov models (hmm) to recognize facial expressions. to study the dynamics of 3d dynamic models for fer, we investigated three types of hmms: temporal 1d-hmm, pseudo 2d-hmm (a combination of a spatial hmm and a temporal hmm), and real 2d-hmm. we also created a new dynamic 3d facial expression database for the research community. the results show that our approach achieves a 90.44% person-independent recognition rate for distinguishing six prototypic facial expressions. the advantage of our method is demonstrated as compared to methods based on 2d texture images, 2d/3d motion units, and 3d static range models. further experimental evaluations also verify the benefits of our approach with respect to partial facial surface occlusion, expression intensity changes, and 3d model resolution variations.
real time feature based 3-d deformable face tracking. in this paper, we develop a novel framework for 3d tracking of the non-rigid face deformation from a single camera. the difficulty of the problem lies in the fact that 3d deformation parameter estimation becomes unstable when there are few reliable facial features correspondences. unfortunately, this often occurs in real tracking scenario when there is significant illumination change, motion blur or large pose variation. in order to extract more information of feature correspondences, the proposed framework integrates three types of features which discriminate face deformation across different views: 1) the semantic features which provide constant correspondences between 3d model points and major facial features; 2) the silhouette features which provide dynamic correspondences between 3d model points and facial silhouette under varying views; 3) the online tracking features that provide redundant correspondences between 3d model points and salient image features. the integration of these complementary features is important for robust estimation of the 3d parameters. in order to estimate the high dimensional 3d deformation parameters, we develop a hierarchical parameter estimation algorithm to robustly estimate both rigid and non-rigid 3d parameters. we show the importance of both features fusion and hierarchical parameter estimation for reliable tracking 3d face deformation. experiments demonstrate the robustness and accuracy of the proposed algorithm especially in the cases of agile head motion, drastic illumination change, and large pose change up to profile view.
learning crfs using graph cuts. many computer vision problems are naturally formulated as random fields, specifically mrfs or crfs. the introduction of graph cuts has enabled efficient and optimal inference in associative random fields, greatly advancing applications such as segmentation, stereo reconstruction and many others. however, while fast inference is now widespread, parameter learning in random fields has remained an intractable problem. this paper shows how to apply fast inference algorithms, in particular graph cuts, to learn parameters of random fields with similar efficiency. we find optimal parameter values under standard regularized objective functions that ensure good generalization. our algorithm enables learning of many parameters in reasonable time, and we explore further speedup techniques. we also discuss extensions to non-associative and multi-class problems. we evaluate the method on image segmentation and geometry recognition.
active contour based segmentation of 3d surfaces. algorithms incorporating 3d information have proven to be superior to purely 2d approaches in many areas of computer vision including face biometrics and recognition. still, the range of methods for feature extraction from 3d surfaces is limited. very popular in 2d image analysis, active contours have been generalized to curved surfaces only recently. current implementations require a global surface parametrisation. we show that a balloon force cannot be included properly in existing methods, making them unsuitable for applications with noisy data. to overcome this drawback we propose a new algorithm for evolving geodesic active contours on implicit surfaces. we also introduce a new narrowband scheme which results in linear computational complexity. the performance of our model is illustrated on various real and synthetic 3d surfaces.
online tracking and reacquisition using co-trained generative and discriminative trackers. visual tracking is a challenging problem, as an object may change its appearance due to viewpoint variations, illumination changes, and occlusion. also, an object may leave the field of view and then reappear. in order to track and reacquire an unknown object with limited labeling data, we propose to learn these changes online and build a model that describes all seen appearance while tracking. to address this semi-supervised learning problem, we propose a co-training based approach to continuously label incoming data and online update a hybrid discriminative generative model. the generative model uses a number of low dimension linear subspaces to describe the appearance of the object. in order to reacquire an object, the generative model encodes all the appearance variations that have been seen. a discriminative classifier is implemented as an online support vector machine, which is trained to focus on recent appearance variations. the online co-training of this hybrid approach accounts for appearance changes and allows reacquisition of an object after total occlusion. we demonstrate that under challenging situations, this method has strong reacquisition ability and robustness to distracters in background.
shape-based retrieval of heart sounds for disease similarity detection. retrieval of similar heart sounds from a sound database has applications in physician training, diagnostic screening, and decision support. in this paper, we exploit a visual rendering of heart sounds and model the morphological variations of audio envelopes through a constrained non-rigid translation transform. similar heart sounds are then retrieved by recovering the corresponding alignment transform using a variant of shape-based dynamic time warping. results of similar heart sound retrieval are demonstrated for various diseases on a large database of heart sounds.
compressive sensing for background subtraction. compressive sensing (cs) is an emerging field that provides a framework for image recovery using sub-nyquist sampling rates. the cs theory shows that a signal can be reconstructed from a small set of random projections, provided that the signal is sparse in some basis, e.g., wavelets. in this paper, we describe a method to directly recover background subtracted images using cs and discuss its applications in some communication constrained multi-camera computer vision problems. we show how to apply the cs theory to recover object silhouettes (binary background subtracted images) when the objects of interest occupy a small portion of the camera view, i.e., when they are sparse in the spatial domain. we cast the background subtraction as a sparse approximation problem and provide different solutions based on convex optimization and total variation. in our method, as opposed to learning the background, we learn and adapt a low dimensional compressed representation of it, which is sufficient to determine spatial innovations; object silhouettes are then estimated directly using the compressive samples without any auxiliary image reconstruction. we also discuss simultaneous appearance recovery of the objects using compressive measurements. in this case, we show that it may be necessary to reconstruct one auxiliary image. to demonstrate the performance of the proposed algorithm, we provide results on data captured using a compressive single-pixel camera. we also illustrate that our approach is suitable for image coding in communication constrained problems by using data captured by multiple conventional cameras to provide 2d tracking and 3d shape reconstruction results with compressive measurements.
latent pose estimator for continuous action recognition. recently, models based on conditional random fields (crf) have produced promising results on labeling sequential data in several scientific fields. however, in the vision task of continuous action recognition, the observations of visual features have dimensions as high as hundreds or even thousands. this might pose severe difficulties on parameter estimation and even degrade the performance. to bridge the gap between the high dimensional observations and the random fields, we propose a novel model that replace the observation layer of a traditional random fields model with a latent pose estimator. in training stage, the human pose is not observed in the action data, and the latent pose estimator is learned under the supervision of the labeled action data, instead of image-to-pose data. the advantage of this model is twofold. first, it learns to convert the high dimensional observations into more compact and informative representations. second, it enables transfer learning to fully utilize the existing knowledge and data on image-to-pose relationship. the parameters of the latent pose estimator and the random fields are jointly optimized through a gradient ascent algorithm. our approach is tested on humaneva [1] --- a publicly available dataset. the experiments show that our approach can improve recognition accuracy over standard crf model and its variations. the performance can be further significantly improved by using additional image-to-pose data for training. our experiments also show that the model trained on humaneva can generalize to different environment and human subjects.
event modeling and recognition using markov logic networks. we address the problem of visual event recognition in surveillance where noise and missing observations are serious problems. common sense domain knowledge is exploited to overcome them. the knowledge is represented as first-order logic production rules with associated weights to indicate their confidence. these rules are used in combination with a relaxed deduction algorithm to construct a network of grounded atoms, the markov logic network. the network is used to perform probabilistic inference for input queries about events of interest. the system's performance is demonstrated on a number of videos from a parking lot domain that contains complex interactions of people and vehicles.
unsupervised structure learning: hierarchical recursive composition, suspicious coincidence and competitive exclusion. we describe a new method for unsupervised structure learning of a hierarchical compositional model (hcm) for deformable objects. the learning is unsupervised in the sense that we are given a training dataset of images containing the object in cluttered backgrounds but we do not know the position or boundary of the object. the structure learning is performed by a bottom-up and top-down process. the bottom-up process is a novel form of hierarchical clustering which recursively composes proposals for simple structures to generate proposals for more complex structures. we combine standard clustering with the suspicious coincidence principle and the competitive exclusion principle to prune the number of proposals to a practical number and avoid an exponential explosion of possible structures. the hierarchical clustering stops automatically, when it fails to generate new proposals, and outputs a proposal for the object model. the top-down process validates the proposals and fills in missing elements. we tested our approach by using it to learn a hierarchical compositional model for parsing and segmenting horses on weizmann dataset. we show that the resulting model is comparable with (or better than) alternative methods. the versatility of our approach is demonstrated by learning models for other objects (e.g., faces, pianos, butterflies, monitors, etc.). it is worth noting that the low-levels of the object hierarchies automatically learn generic image features while the higher levels learn object specific features.
determining patch saliency using low-level context. the increased use of context for high level reasoning has been popular in recent works to increase recognition accuracy. in this paper, we consider an orthogonal application of context. we explore the use of context to determine which low-level appearance cues in an image are salient or representative of an image's contents. existing classes of low-level saliency measures for image patches include those based on interest points, as well as supervised discriminative measures. we propose a new class of unsupervised contextual saliency measures based on co-occurrence and spatial information between image patches. for recognition, image patches are sampled using a weighted random sampling based on saliency, or using a sequential approach based on maximizing the likelihoods of the image patches. we compare the different classes of saliency measures, along with a baseline uniform measure, for the task of scene and object recognition using the bag-of-features paradigm. in our results, the contextual saliency measures achieve improved accuracies over the previous methods. moreover, our highest accuracy is achieved using a sparse sampling of the image, unlike previous approaches who's performance increases with the sampling density.
window annealing over square lattice markov random field. monte carlo methods and their subsequent simulated annealing are able to minimize general energy functions. however, the slow convergence of simulated annealing compared with more recent deterministic algorithms such as graph cuts and belief propagation hinders its popularity over the large dimensional markov random field (mrf). in this paper, we propose a new efficient sampling-based optimization algorithm called wa (window annealing) over squared lattice mrf, in which cluster sampling and annealing concepts are combined together. unlike the conventional annealing process in which only the temperature variable is scheduled, we design a series of artificial "guiding" (auxiliary) probability distributions based on the general sequential monte carlo framework. these auxiliary distributions lead to the maximum a posteriori (map) state by scheduling both the temperature and the proposed maximum size of the windows (rectangular cluster) variable. this new annealing scheme greatly enhances the mixing rate and consequently reduces convergence time. moreover, by adopting the integral image technique for computation of the proposal probability of a sampled window, we can achieve a dramatic reduction in overall computations. the proposed wa is compared with several existing monte carlo based optimization techniques as well as state-of-the-art deterministic methods including graph cut (gc) and sequential tree re-weighted belief propagation (trw-s) in the pairwise mrf stereo problem. the experimental results demonstrate that the proposed wa method is comparable with gc in both speed and obtained energy level.
articulated multi-body tracking under egomotion. in this paper, we address the problem of 3d articulated multi-person tracking in busy street scenes from a moving, human-level observer. in order to handle the complexity of multi-person interactions, we propose to pursue a two-stage strategy. a multi-body detection-based tracker first analyzes the scene and recovers individual pedestrian trajectories, bridging sensor gaps and resolving temporary occlusions. a specialized articulated tracker is then applied to each recovered pedestrian trajectory in parallel to estimate the tracked person's precise body pose over time. this articulated tracker is implemented in a gaussian process framework and operates on global pedestrian silhouettes using a learned statistical representation of human body dynamics. we interface the two tracking levels through a guided segmentation stage, which combines traditional bottom-up cues with top-down information from a human detector and the articulated tracker's shape prediction. we show the proposed approach's viability and demonstrate its performance for articulated multi-person tracking on several challenging video sequences of a busy inner-city scenario.
learning visual shape lexicon for document image content recognition. developing effective content recognition methods for diverse imagery continues to challenge computer vision researchers. we present a new approach for document image content categorization using a lexicon of shape features. each lexical word corresponds to a scale and rotation invariant shape feature that is generic enough to be detected repeatably and segmentation free. we learn a concise, structurally indexed shape lexicon from training by clustering and partitioning feature types through graph cuts. we demonstrate our approach on two challenging document image content recognition problems: 1) the classification of 4,500 web images crawled from google image search into three content categories -- pure image, image with text, and document image, and 2) language identification of 8 languages (arabic, chinese, english, hindi, japanese, korean, russian, and thai) on a 1,512 complex document image database composed of mixed machine printed text and handwriting. our approach is capable to handle high intra-class variability and shows results that exceed other state-of-the-art approaches, allowing it to be used as a content recognizer in image indexing and retrieval systems.
face alignment via component-based discriminative search. in this paper, we propose a component-based discriminative approach for face alignment without requiring initialization. unlike many approaches which locally optimize in a small range, our approach searches the face shape in a large range at the component level by a discriminative search algorithm. specifically, a set of direction classifiers guide the search of the configurations of facial components among multiple detected modes of facial components. the direction classifiers are learned using a large number of aligned local patches and misaligned local patches from the training data. the discriminative search is extremely effective and able to find very good alignment results only in a few (2~3) search iterations. as the new approach gives excellent alignment results on the commonly used datasets (e.g., ar [18], feret [21]) created under-controlled conditions, we evaluate our approach on a more challenging dataset containing over 1,700 well-labeled facial images with a large range of variations in pose, lighting, expression, and background. the experimental results show the superiority of our approach on both accuracy and efficiency.
improving the agility of keyframe-based slam. the ability to localise a camera moving in a previously unknown environment is desirable for a wide range of applications. in computer vision this problem is studied as monocular slam. recent years have seen improvements to the usability and scalability of monocular slam systems to the point that they may soon find uses outside of laboratory conditions. however, the robustness of these systems to rapid camera motions (we refer to this quality as agility) still lags behind that of tracking systems which use known object models. in this paper we attempt to remedy this. we present two approaches to improving the agility of a keyframe-based slam system: firstly, we add edge features to the map and exploit their resilience to motion blur to improve tracking under fast motion. secondly, we implement a very simple inter-frame rotation estimator to aid tracking when the camera is rapidly panning --- and demonstrate that this method also enables a trivially simple yet effective relocalisation method. results show that a slam system combining points, edge features and motion initialisation allows highly agile tracking at a moderate increase in processing time.
anisotropic geodesics for perceptual grouping and domain meshing. this paper shows how voronoi diagrams and their dual delaunay complexes, defined with geodesic distances over 2d reimannian manifolds, can be used to solve two important problems encountered in computer vision and graphics. the first problem studied is perceptual grouping which is a curve reconstruction problem where one should complete in a meaningful way a sparse set of noisy curves. from this latter curves, our grouping algorithm first designs an anisotropic tensor field that corresponds to a reimannian metric. then, according to this metric, the delaunay graph is constructed and pruned in order to correctly link together salient features. the second problem studied is planar domain meshing, where one should build a good quality triangulation of a given domain. our meshing algorithm is a geodesic delaunay refinement method that exploits an anisotropic tensor field in order to locally impose the orientation and aspect ratio of the created triangles.
relevant feature selection for human pose estimation and localization in cluttered images. we address the problem of estimating human body pose from a single image with cluttered background. we train multiple local linear regressors for estimating the 3d pose from a feature vector of gradient orientation histograms. each linear regressor is capable of selecting relevant components of the feature vector depending on pose by training it on a pose cluster which is a subset of the training samples with similar pose. for discriminating the pose clusters, we use kernel support vector machines (svm) with pose-dependent feature selection. we achieve feature selection for kernel svms by estimating scale parameters of rbf kernel through minimization of the radius/margin bound, which is an upper bound of the expected generalization error, with efficient gradient descent. human detection is also possible with these svms. quantitative experiments show the effectiveness of pose-dependent feature selection to both human detection and pose estimation.