waashk/dblp huggingface dataset:
brdf invariant stereo using light transport constancy. nearly all existing methods for stereo reconstruction assume that scene reflectance is lambertian, and make use of color constancy as a matching invariant. we introduce a new invariant for stereo reconstruction called light transport constancy, which allows completely arbitrary scene reflectance (brdfs). this invariant can be used to formulate a rank constraint on multiview stereo matching when the scene is observed in several lighting configurations. in addition, we show that this multiview constraint can be used with as few as two cameras and two lighting configurations. unlikely previous methods for brdf invariant stereo, light transport constancy does not require precisely configured or calibrated light sources, nor calibration objects in the scene. importantly, the new constraint can be used to provide brdf invariance to any existing stereo method, whenever appropriate lighting variation is available.
real-time simultaneous localisation and mapping with a single camera. ego-motion estimation for an agile single camera movingthrough general, unknown scenes becomes a much morechallenging problem when real-time performance is requiredrather than under the off-line processing conditionsunder which most successful structure from motion workhas been achieved. this task of estimating camera motionfrom measurements of a continuously expanding set of self-mappedvisual features is one of a class of problems knownas simultaneous localisation and mapping (slam) in therobotics community, and we argue that such real-time mappingresearch, despite rarely being camera-based, is morerelevant here than off-line structure from motion methodsdue to the more fundamental emphasis placed on propagationof uncertainty.we present a top-down bayesian framework for single-cameralocalisation via mapping of a sparse set of naturalfeatures using motion modelling and an information-guidedactive measurement strategy, in particular addressingthe difficult issue of real-time feature initialisation viaa factored sampling approach. real-time handling of uncertaintypermits robust localisation via the creating andactive measurement of a sparse map of landmarks such thatregions can be re-visited after periods of neglect and localisationcan continue through periods when few features arevisible. results are presented of real-time localisation fora hand-waved camera with very sparse prior scene knowledgeand all processing carried out on a desktop pc.
active search for real-time vision. in most cases when information is to be extracted from an image, there are priors available on the state of the world and therefore on the detailed measurements which will be obtained. while such priors are commonly combined with the actual measurements via bayes¿ rule to calculate posterior probability distributions on model parameters, their additional value in guiding efficient image processing has almost always been overlooked. priors tell us where to look for information in an image, how much computational effort we can expect to expend to extract it, and of how much utility to the task in hand it is likely to be. such considerations are of importance in all practical real-time vision systems, where the processing resources available at each frame in a sequence are strictly limited ¿ and it is exactly in high frame-rate real-time systems such as trackers where strong priors are most likely to be available. in this paper, we use shannon information theory to analyse the fundamental value of measurements using mutual information scores in absolute units of bits, specifically looking at the overwhelming case where uncertainty can be characterised by gaussian probability distributions. we then compare these measurement values with the computational cost of the image processing required to obtain them. this theory puts on a firm footing for the first time principles of ¿active search¿ for efficient guided image processing, in which candidate features of possibly different types can be compared and selected automatically for measurement.
adaptive shape evolution using blending. we propose a shape representation scheme which allows two shapes to be combined into a single model. the desired regions of the two shapes are selected, and then merged together forming a blended shape. for reconstruction, blending is incorporated into a deformable model framework. the model automatically adapts to the data, blending when necessary. hierarchical blending allows multiple blends of a shape to occur forming an evolution from the initial shape of a sphere to the final shape. blending also allows the insertion of a hole between arbitrary locations. the models used are globally defined, making the recovered shape a natural symbolic description. we present reconstruction experiments involving shapes of various topologies.
deformable model-based shape and motion analysis from images using motion residual error. we present a novel method for the shape and motion estimation of a deformable model using error residuals from model-based motion analysis. the motion of the model is first estimated using a model-based least squares method. using the residuals from the least squares solution, the non-rigid structure of the model can be better estimated by computing how changes in the shape of the model affect its motion parameterization. this method is implemented as a component in a deformable model-based framework that uses optical flow information and edges. this general model-based framework is applied to human face shape and motion estimation. we present experiments that demonstrate that this framework is a considerable improvement over a framework that uses only optical flow information and edges.
3d articulated models and multi-view tracking with silhouettes. we propose a method to estimate the motion of a person filmed by two or more fixed cameras. the novelty of our technique is its ability to cope with fast movements, self-occlusions and noisy images. our algorithms are based on the latest works on calibration and image segmentation developed in our lab. we compare the projections of a 3d model of a person on the images to the detected silhouettes of the person, and create forces that will move the 3d model towards the final estimation of the real pose. we developed a fast algorithm that computes the motion of the articulated 3d model. we show that our results are good, even if the cameras are not synchronized.
initialization of deformable models from 3d data. the robustness of shape recovery based on deformable models dep ends in general, on the relative difference of position and topology of the initial model with respect to the data: a close initialization with correct topology guaranties a proper recovery of the object. furthermore, the closeness of the initial model greatly influences the time of computation needed for the recovery. in this paper, we propose a method for initializing deformable models from range data or volumetric images. the prop osed method solves two distinct problems. first, we use the topological segmentation of volumetric images in order to recover the approximate topology of the object. second, we use an efficient mesh sampling algorithm to control the number of vertices of the initial model. the method takes into account missing data and outliers.
constraining human body tracking. our paper addresses the problem of enforcing constraintsin human body tracking. a projection technique is derivedto impose kinematic constraints on independent multi-bodymotion: we show that for small motions the multi-body articulatedmotion space can be approximated by a linearmanifold estimated directly from the previous body pose.we propose a learning approach to model non-linear constraints;we train a support vector classifier from motioncapture data to model the boundary of the space of validposes. linear and non-linear body pose constraints are enforcedby first projecting unconstrained motions onto thearticulated motion space and then optimizing to find pointson this linear manifold that lie within the non-linear constraintsurface modeled by the svm classifier.
avoiding the "streetlight effect": tracking by exploring likelihood modes. classic methods for bayesian inference effectively constrain search to lie within regions of significant probability of the temporal prior. this is efficient with an accurate dynamics model, but otherwise is prone to ignore significant peaks in the true posterior. a more accurate posterior estimate can be obtained by explicitly finding modes of the likelihood function and combining them with a weak temporal prior. in our approach modes are found using effi- cient example-based matching followed by local refinement to find peaks and estimate peak bandwidth. by reweighting these peaks according to the temporal prior we obtain an estimate of the full posterior model. we show comparative results on real and synthetic images in a high degree of freedom articulated tracking task.
a symmetric patch-based correspondence model for occlusion handling. occlusion is one of the challenging problems in stereo. in this paper, we solve the problem in a segment-based style. both images are segmented, and we propose a novel patch-based stereo algorithm that cuts the segments of one image using the segments of the other, and handles occlusion areas in a proper way. a symmetric graph-cuts optimization framework is used to find correspondence and occlusions simultaneously. the experimental results show superior performance of the proposed algorithm, especially on occlusions, untextured areas and discontinuities.
information theoretic focal length selection for real-time active 3-d object tracking. active object tracking, for example, in surveillance tasks, becomes more and more important these days. besides the tracking algorithms themselves methodologies have to be developed for reasonable active control of the degrees of freedom of all involved cameras. in this paper we present an information theoretic approach that allows the optimal selection of the focal lengths of two cameras during active 3-d object tracking. the selection is based on the uncertainty in the 3-d estimation. this allows us to resolve the trade-off between small and large focal length: in the former case, the chance is increased to keep the object in the field of view of the cameras. in the latter one, 3-d estimation becomes more reliable. also, more details are provided, for example for recognizing the objects.beyond a rigorous mathematical framework we present real-time experiments demonstrating that we gain an improvementin 3-d trajectory estimation by up to 42% in comparison with tracking using a fixed focal length.
tracking through singularities and discontinuities by random sampling. some issues in marker-less tracking of human body motion are addressed. extended kalman filters have commonly been applied to kinematic variables, to combine predictions consistent with plausible motion, with the incoming stream of visual measurements. kalman filtering is applicable only when the underlying distribution is approximately gaussian. often, this assumption proves remarkably robust.there are two pervasive circumstances under which the gaussianity assumption can break down. the first is kinematic singularity, and the second is at joint end-stops. failure of kalman filtering under these circumstance is illustrated.the non-gaussian nature of the distributions is demonstrated experimentally by means of monte-carlo simulation. random simulation - particle filtering or condensation -proves to provide a robust alternative algorithm for tracking that can also deal with these difficult conditions.
markov-based failure prediction for human motion analysis. this paper presents a new method of detecting andpredicting motion tracking failures with applications inhuman motion and gait analysis. we define a trackingfailure as an event and describe its temporal characteristicsusing a hidden markov model (hmm). thisstochastic model is trained using previous examples oftracking failures. we derive vector observations for thehmm using the noise covariance matrices characterizinga tracked, 3-d structural model of the human body. weshow a causal relationship between the conditional outputprobability of the hmm, as transformed using alogarithmic mapping function, and impending trackingfailures. results are illustrated on several multi-viewsequences of complex human motion.
active concept learning for image retrieval in dynamic databases. concept learning in content-based image retrieval (cbir) systems is a challenging task. this paper presents an active concept learning approach based on mixture model to deal with the two basic aspects of a database system: changing (image insertion or removal) nature of a database and user queries. to achieve concept learning, we develop a novel model selection method based on bayesian analysis that evaluates the consistency of hypothesized models with the available information. the analysis of exploitation vs. exploration in the search space helps to find optimal model efficiently. experimental results on corel database show the efficacy of our approach.
cosmos-a representation scheme for free-form surfaces. we address the problem of representing and recognizing arbitrarily curved 3d rigid objects when: the objects may vary in shape and complexity, and no restrictive assumptions are made about the types of surfaces on the object. we propose a new and general surface representation scheme for recognizing objects with free form (sculpted) surfaces from range data. in this scheme, an object is described concisely in terms of maximal surface patches of constant shape index. these maximal patches are mapped onto the unit sphere via their orientations, and aggregated via shape spectral functions. properties such as surface area, curvedness and connectivity that capture local and global information are also built into the representation. the scheme yields not only a meaningful and rich surface description useful for the recoverability of the object, but also a set of powerful indexing primitives for object matching. we demonstrate the generality and the effectiveness of our scheme using real range images of complex objects. we also present results on the categorization of object views based on a novel shape spectral matching technique.
dynamic texture segmentation. we address the problem of segmenting a sequence of imagesof natural scenes into disjoint regions that are characterizedby constant spatio-temporal statistics. we model thespatio-temporal dynamics in each region by gauss-markovmodels, and infer the model parameters as well as theboundary of the regions in a variational optimization framework.numerical results demonstrate that - in contrast topurely texture-based segmentation schemes - our method iseffective in segmenting regions that differ in their dynamicseven when spatial statistics are identical.
selection of scale-invariant parts for object class recognition. this paper introduces a novel method for constructingand selecting scale-invariant object parts. scale-invariantlocal descriptors are first grouped into basic parts. a classifieris then learned for each of these parts, and featureselection is used to determine the most discriminative ones.this approach allows robust part detection, and it is invariantunder scale change-that is, neither the training imagesnor the test images have to be normalized.the proposed method is evaluated in car detectiontasks with significant variations in viewing conditions, andpromising results are demonstrated. different local regions,classifiers and feature selection methods are quantitativelycompared. our evaluation shows that local invariantdescriptors are an appropriate representation for objectclasses such as cars, and it underlines the importance offeature selection.
self-calibration of a stereo rig using monocular epipolar geometry. this paper addresses the problem of self-calibration from one unknown motion of an uncalibrated stereo rig. unlike the existing methods for stereo rig self-calibration, which have been focused on applying the autocalibration paradigm using both motion and stereo correspondences, our method does not require the recovery of stereo correspondences. our method combines purely algebraic constraints with implicit geometric constraints. assuming that the rotational part of the stereo geometry has two unknown degrees of freedom (i.e., the third dof is roughly known), and that the principle point of each camera is known, we first show that the computation of the intrinsic and extrinsic parameters of the stereo rig can be recovered from the motion correspondences only, i.e., the monocular fundamental matrices. we then provide an initialization procedure for the proposed non-linear method. we provide an extensive performance study for the method in the presence of image noise. in addition, we study some of the aspects related to the 3d motion that govern the accuracy of the proposed self-calibration method. experiments conducted on synthetic and real data/images demonstrate the effectiveness and efficiency of the proposed method.
simultaneous facial action tracking and expression recognition using a particle filter. the recognition of facial gestures and expressions in image sequences is an important and challenging problem. most of the existing methods adopt the following paradigm. first, facial actions/features are retrieved from the images, and then facial expressions are recognized based on the retrieved temporal parameters. unlike this main stream, this paper introduces a new approach allowing the simultaneous recovery of facial actions and expression using a particle filter adopting multi-class dynamics that are conditioned on the expression. for each frame in the video sequence, our approach is split in two consecutive stages. in the first stage, the 3d head pose is recovered using a deterministic registration technique based on online appearance models. in the second stage, the facial actions as well as the facial expression are simultaneously recovered using the stochastic framework with mixed states. the proposed fast scheme is either as robust as existing ones or more robust with respect to many regards. experimental results show the feasibility and robustness of the proposed approach.
on optimal light configurations in photometric stereo. this paper develops new theory for the optimal placement of photometric stereo lighting in the presence of camera noise. we show that for three lights, any triplet of orthogonal light directions minimises the uncertainty in scaled normal computation. the assumptions are that the camera noise is additive and normally distributed, and uncertainty is defined as the expectation of squared distance of scaled normal to the ground truth. if the camera noise is of zero mean and variance ¿², the optimal (minimum) uncertainty in the scaled normal is 3¿². for case of n > 3 lights, we show that the minimum uncertainty is 9¿²/n, and identify sets of light configurations which reach this theoretical minimum.
can two specular pixels calibrate photometric stereo?. lambertian photometric stereo with unknown light source parameters is ambiguous. provided that the object imaged constitutes a surface, the ambiguity is represented by the group of generalised bas-relief (gbr) transformations. we show that this ambiguity is resolved when specularreflection is present in two images taken under two different light source directions. we identify all configurations of the two directional lights which are singular and show that they can easily be tested for. while previous work used optimisation algorithms to apply the constraints implied by the specular reflectance component, we have developed a linear algorithm to achieve this goal. our theory can be utilised to construct fast algorithms for automatic reconstruction of smooth glossy surfaces.
illumination-invariant color object recognition via compressed chromaticity histograms of color-channel-normalized images. several color object recognition methods that are based on image retrieval algorithms attempt to discount changes of illumination in order to increase performance when test image illumination conditions differ from those that obtained when the image database was created. here we extend the seminal method of swain and ballard to discount changing illumination.the new method is based on the first stage of the simplest color indexing method, which uses angular invariants between color image and edge image channels. that method first normalizes image channels, and then effectively discards much of the remaining information. here we adopt the color-normalization stage as an adequate color constancy step. further,we replace 3d color histograms by 2d chromaticity histograms. treating these as images, we implement the method in a compressed histogram-image domain using a combination of wavelet compression and discrete cosine transform (dct) to fully exploit the technique of low-pass filtering for efficiency. results are very encouraging, with substantially better performance than other methods tested. the method is also fast, in that the indexing process is entirely carried out in the compressed domain and uses a feature vector of only 36 or 72 values.
incremental discovery of object parts in video sequences. this paper addresses the problem of automatically discovering the rigid parts of an initially unknown moving deformable object in a monocular video sequence. the parts are first extracted through motion-based segmentation, using a time scale automatically chosen with the quantity of motion concept. tracking and reobservation reinforce these low-level segmentation results and further segmentation is performed only when and where no modeled parts can be tracked. central to the system is the modeler that minimizes the impacts of erroneous segmentations and departures in tracking. the sequential nature of the framework allows incremental modeling and segmentation of parts that need not simultaneously be visible or in motion, making it possible to circumvent the typical constraint of model initialization. the fundamental principles are strictly ensemblist and do not rely on any specific pdf. the interest of this framework is demonstrated on three types of video sequences including human and robot motion.
a multi-scale generative model for animate shapes and parts. this paper presents a multi-scale generative model for representing animate shapes and extracting meaningful parts of objects. the model assumes that animate shapes (2d simple closed curves) are formed by a linear superposition of a number of shape bases. these shapebases resemble the multi-scale gabor bases in image pyramid representation, are well localized in both spatial and frequency domains, and form an over-complete dictionary. this model is simpler than the popular b-spline representation since it does not engage a domainpartition. thus it eliminates the interference between adjacent b-spline bases, and becomes a true linear additive model. we pursue the bases by reconstructing the shape in a coarse-to-fine procedure through curve evolution. these shape bases are further organized ina tree-structure where the bases in each subtree sum up to an intuitive part of the object. to build probabilistic model for a class of objects, we propose a markov random field model at each level of the tree representation to account for the spatial relationship between bases. thus the final model integrates a markov tree (generative) model over scales and a markov random field over space. we adopt em-type algorithm for learning the meaningful parts for a shape class, and show some results on shape synthesis.