waashk/dblp huggingface dataset:
structure from motion using sequential monte carlo methods. in this paper, the structure from motion (sfm) problem is addressed using sequential monte carlo methods. a new sfm algorithm based on random sampling is derived to estimate the posterior distributions of camera motion and scene structure for the perspective projection camera model. experimental results show that challenging issues in solving the sfm problem, due to erroneous feature tracking, feature occlusion, motion/structure ambiguity, mixed-domain sequences, mismatched features, and independently moving objects, can be well modeled and effectively addressed using the proposed method.
basic gray level aura matrices: theory and its application to texture synthesis. in this paper, we present a new mathematical framework for modeling texture images using independent basic gray level aura matrices (bglams). we prove that independent bglams are the basis of gray level aura matrices (glams), and that an image can be uniquely represented by its independent bglams. we propose a new bglam distance measure for automatically evaluating synthesis results w.r.t. input textures to determine if the output is a successful synthesis of the input. for the application to texture synthesis, we present a new algorithm to synthesize textures by sampling only the independent bglams of an input texture. with respect to synthesis of textures and evaluation of the results, the performance of our approach is extensively evaluated and compared with symmetric glams that are used in existing techniques and with gray level cooccurrence matrices (glcms). experimental results have shown that (1) our approach significantly outperforms both symmetric glams and glcms; (2) the new bglam distance measure has the ability to evaluate synthesis results, which can be used to automate the conventional visual inspection process for determining whether or not the output texture is a successful synthesis of the input; and (3) a broad range of textures can be faithfully synthesized using independent bglams and the synthesis results are comparable to existing techniques.
three dimensional mr brain segmentation. in mr brain images, segmentation using intensity values is severely limited owing to field inhomogeneities, susceptibility artifacts and partial volume effects. edge based segmentation methods suffer from spurious edges and gaps in boundaries. a method is presented which combines the advantages of edge based and region based segmentation. first a multiscale image representation is constructed which favors intra-tissue diffusion over inter-tissue diffusion by exploiting local contrast. subsequently a multiscale linking model (the hyperstack) is used to group voxels into a number of segments. this facilitates segmentation of grey matter, white matter and cerebrospinal fluid with minimal user interaction. using a supervised segmentation technique and mr simulations of a brain phantom as validation it is shown that the errors are in the order of or smaller than reported in literature.
phenomenological eigenfunctions for image irradiance. we present a framework for calculating low-dimensional bases to represent image irradiance from surfaces with isotropic reflectance under arbitrary illumination. by representing the illumination and the bidirectional reflectance distribution function (brdf) in frequency space, a model for the image irradiance is derived. this model is then reduced in dimensionality by analytically constructing the principal component basis for all images given the variations in both the illumination and the surface material. the principal component basis are constructed in such a way that all the symmetries (helmholtz reciprocity and isotropy) of the brdf are preserved in the basis functions. using the framework we calculate a basis using a database of natural illumination and the curet database containing brdfs of real world surface materials.
face recognition by stepwise nonparametric margin maximum criterion. linear discriminant analysis (lda) is a popular feature extraction technique in face recognition. however, it often suffers from the small sample size problem when dealing with the high dimensional data. moreover, while lda is guaranteed to find the best directions when each class has a gaussian density with a common covariance matrix, it can fail if the class densities are more general. in this paper, a new nonparametric linear feature extraction method, step-wise nonparametric margin maximum criterion(snmmc), is proposed to find the most discriminant directions, which does not assume that the class densities belong to any particular parametric family and does not depend on the non-singularity of the within-class scatter matrix either. on three datasets from att and feret face databases, our experimental results demonstrate that snmmc outperforms other methods and is robust to variations of pose, illumination and expression.
using eye reflections for face recognition under varying illumination. face recognition under varying illumination remains a challenging problem. much progress has been made toward a solution through methods that require multiple gallery images of each subject under varying illumination. yet for many applications, this requirement is too severe. in this paper, we propose a novel method that requires only a single gallery image per subject taken under unknown lighting. the method builds upon two contributions. we first estimate the lighting from its reflection in the eyes. this allows us to explicitly recover the illumination in the single gallery images as well as the probe image. next, we exploit the local linearity of face appearance variation across different people. we represent the gallery images as locally linear montages of images of many different faces taken under the same lighting (bootstrap images). then, we transfer the estimated combination of bootstrap images to synthesize each subject¿s face under the probe lighting to accomplish recognition. finally, we show through tests on the cmu pie database that we can achieve better recognition results using our lighting estimation method and locally linear montages than the current state-of-the-art.
real-time interactively distributed multi-object tracking using a magnetic-inertia potential model. this paper breaks with the common practice of using a joint state space representation and performing the joint data association in multi-object tracking. instead, we present an interactively distributed framework with linear complexity for real-time applications. when objects do not interact on each other, our approach performs like multiple independent trackers. when the objects are in close proximity or present occlusions, we propose a magnetic-inertia potential model to handle the "error merge" and "labelling" problems in a particle filtering framework. specifically, we propose to model the interactive likelihood densities by a "gravitation" and "magnetic" repulsion scheme and relax the common first-order markov chain assumption by using an "inertia" markov chain. our model represents the cumulative effect of virtual physical forces that objects undergo while interacting with others. it implicitly handles the "error merge" and "labelling" problems and thus solves the difficult object occlusion and data association problems using an innovative scheme. our preliminary work has demonstrated that the proposed approach is far superior to existing methods not only in robustness but also in speed.
invariant of a pair of non-coplanar conies in space: definition, geometric interpretation and computation. the joint invariants of a pair of coplanar conics has been widely used in recent vision literature. in this paper, the algebraic invariant of a pair of non-coplanar conics in space is concerned. the algebraic invariant of a pair of non-coplanar conics is first derived from the invariant algebra of a pair of quaternary quadratic forms by using the dual representation of space conics. then, this algebraic invariant is geometrically interpreted in terms of cross-ratios. finally, an analytical procedure for projective reconstruction of a space conic from two uncalibrated images is developed and the correspondence conditions of the conics between two views are also explicited.
linear n>=4-point pose determination. the determination of the position and the orientation of the camera from the known correspondences of the reference points and the image points is known as the problem of pose estimation in computer vision or space resection in photogrammetry. it is well known that using 3 corresponding points has at most 4 solutions. less appears to be known about the cases of 4 and 5 points. in this paper, we describe linear solutions that always give the unique solution to 4-point and 5-point pose determination for the reference points not lying on the critical confogurations. the same linear method can also be extended to any n ¿ 5 points.the robustness and accuracy of the method are experimented both on simulated and real images.
preemptive ransac for live structure and motion estimation. a system capable of performing robust live ego-motion estimation for perspective cameras is presented. the system is powered by random sample consensus with preemptive scoring of the motion hypotheses. a general statement of the problem of efficient preemptive scoring is given. then a theoretical investigation of preemptive scoring under a simple inlier-outlier model is performed. a practical preemption scheme is proposed and it is shown that the preemption is powerful enough to enable robust live structure and motion estimation.
non-parametric self-calibration. in this paper we develop a theory of non-parametric self-calibration. recently, schemes have been devised for non-parametric laboratory calibration, but not for self-calibration. we allow an arbitrary warp to model the intrinsic mapping, with the only restriction that the camera is central and that the intrinsic mapping has a well-defined non-singular matrix derivative at a finite number of points under study. we give a number of theoretical results, both forinfinitesimal motion and finite motion, for a finite number of observations and when observing motion over a dense image, for rotation and translation. our main result is that through observing the flow induced by three instantaneous rotations at a finite number of points of the distorted image, we can perform projective reconstruction of those image points on the undistorted image. we present some results with synthetic and real data.
recovering photometric properties of multiple strongly-reflective, partially-transparent surfaces from a single image. this paper introduces a method to recover photometric parameters of a set of 3d surfaces from a single image with significant global-illumination effects such as inter-reflections and transparencies. since this problem is ambiguous for arbitrary unknown scenes, our formulation assumes that the scene consists of a small set of photometrically homogeneous surfaces with known 3d shapes, illuminated by known light sources. we show that under these conditions, the system of non-linear equations that defines how the image is formed may be factorized into a vector composed only of products of some photometric parameters, and a matrix, whose elements depend non-linearly on both the known illumination, the known 3d shapes and the remaining photometric parameters. this factorization leads to an efficient optimization-based algorithm to compute all unknown photometric parameters from a single input image. experiments with real data show that this algorithm is more stable and efficient than simpler alternatives.
detecting kinetic occlusion. visual motion boundaries provide a powerful cue for the perceptual organization of scenes. motion boundaries are present when surfaces in motion occlude one another. conventional approaches to motion analysis have relied on assumptions of data conservation and smoothness, which has made analysis of motion boundaries difficult. we show that a common source of motion boundary, kinetic occlusion, can be detected using spatiotemporal junction analysis. junction analysis is accomplished by utilizing distributed representations of motion used in models of human visual motion sensing. by detecting changes in the direction of motion in these representations, spatiotemporal junctions are detected in a manner which differentiates accretion from deletion. we demonstrate successful occlusion detection on spatiotemporal imagery containing occluding surfaces in motion.
the catchment feature model for multimodal language analysis. the catchment feature model (cfm) addresses two questions in multimodal interaction: how do we bridge video and audio processing with the realities of human multimodal communication, and how information from the different modes may be fused. we discuss the need for our model, motivate the cfm from psycholinguistic research, and present the model. in contrast to 'whole gesture' recognition, the cfm applies a feature decomposition approach that facilitates cross-modal fusion at the level of discourse planning and conceptualization. we present our experimental framework for cfm-based research, and cite three concrete examples of catchment features (cf), and propose new directions of multimodal research based on the model.
learning dynamical models using expectation-maximisation. tracking with deformable contours in a filtering frame-work requir esa dynamical model for prediction. for any given application, tracking is improved by having an accurate model, learned from training data. we develop a method for learning dynamical models from training sequences, explicitly taking account of the fact that training data are noisy measurements and not true states. by introducing an "augmented-state smoothing filter" , we show how the technique of expectation-maximisation can be applied to this problem, and show that the resulting algorithm produces more robust and accurate tracking.
modeling scenes with local descriptors and latent aspects. we present a new approach to model visual scenes in image collections, based on local invariant features and probabilistic latent space models. our formulation provides answers to three open questions:(1) whether the invariant local features are suitable for scene (rather than object) classification; (2) whether unsupervised latent space models can be used for feature extraction in the classification task; and (3) whether the latent space formulation can discover visual co-occurrence patterns, motivating novel approaches for image organization and segmentation. using a 9500-image dataset, our approach is validated on each of these issues. first, we show with extensive experiments on binary and multi-class scene classification tasks, that a bag-of-visterm representation, derived from local invariant descriptors, consistently outperforms state-of-the-art approaches. second, we show that probabilistic latent semantic analysis (plsa) generates a compact scene representation, discriminative for accurate classification, and significantly more robust when less training data are available. third, we have exploited the ability of plsa to automatically extract visually meaningful aspects, to propose new algorithms for aspect-based image ranking and context-sensitive image segmentation.
a snake for model-based segmentation. despite the promising results of numerous applications, the hitherto proposed snake techniques share some common problems: snake attraction by spurious edge points, snake degeneration (shrinking and flattening), convergence and stability of the deformation process, snake initialization and local determination of the parameters of elasticity. we argue here that these problems can be solved only when all the snake aspects are considered. the snakes proposed here implement a new potential field and external force in order to provide a deformation convergence, attraction by both near and far edges as well as snake behaviour selective according to the edge orientation. furthermore, we conclude that in the case of model-based segmentation, the internal force should include structural information about the expected snake shape. experiments using this kind of snakes for segmenting bones in complex hand radiographs show a significant improvement.
learning and inference in parametric switching linear dynamical systems. we introduce parametric switching linear dynamic systems (p-slds) for learning and interpretation of parametrized motion, i.e., motion that exhibits systematic temporal and spatial variations. our motivating example is the honeybee dance: bees communicate the orientation and distance to food sources through the dance angles and waggle lengths of their stylized dances. switching linear dynamic systems (slds) are a compelling way to model such complex motions. however, slds does not provide a means to quantify systematic variations in the motion. previously, wilson ¿ bobick presented parametric hmms [21], an extension to hmms with which they successfully interpreted human gestures. inspired by their work, we similarly extend the standard slds model to obtain parametric slds. we introduce additional global parameters that represent systematic variations in the motion, and present general expectation-maximization (em) methods for learning and inference. in the learning phase, p-slds learns canonical slds model from data. in the inference phase, p-slds simultaneously quantifies the global parameters and labels the data. we apply these methods to the automatic interpretation of honey-bee dances, and present both qualitative and quantitative experimental results on actual bee-tracks collected from noisy video data.
a graph cut algorithm for generalized image deconvolution. the goal of deconvolution is to recover an image x from its convolution with a known blurring function. this is equivalent to inverting the linear system y = hx. in this paper we consider the generalized problem where the system matrix h is an arbitrary non-negative matrix. linear inverse problems can be solved by adding a regularization term to impose spatial smoothness. to avoid oversmoothing, the regularization term must preserve discontinuities; this results in a particularly challenging energy minimization problem. where h is diagonal, as occurs in image denoising, the energy function can be solved by techniques such as graph cuts, which have proven to be very effective for problems in early vision. when h is non-diagonal, however, the data cost for a pixel to have a intensity depends on the hypothesized intensities of nearby pixels, so existing graph cut methods cannot be applied. this paper shows how to use graph cuts to obtain a discontinuity-preserving solution to a linear inverse system with an arbitrary non-negative system matrix. we use a dynamically chosen approximation to the energy which can be minimized by graph cuts; minimizing this approximation also decreases the original energy. experimental results are shown for mri reconstruction from fourier data.
higher order statistical learning for vehicle detection in images. the paper describes a scheme for detecting vehicles in images. the proposed method approximately models the unknown distribution of the images of vehicles by learning higher order statistics (hos) information of the `vehicle class' from sample images. given a test image, statistical information about the background is learnt `on the y'. an hos-based decision measure then classifies test patterns as vehicles or otherwise.when tested on real images of aerial views of vehicular activity, the method gives good results even on complicated scenes. it does not require any a priori information about the site. however, it is amenable to augmentation with contextual information. the method can serve as an important step towards building an automated roadway monitoring system.
obstacle detection using projective invariant and vanishing lines. this paper presents a novel method for detecting vehicles as obstacles in various road scenes using a single on-board camera. vehicles are detected by testing whether the motion of a set of three horizontal line segments, which are always on the vehicles, satisfies the motion constraint of the ground plane or that of the surface plane of the vehicles. the motion constraint of each plane is derived from the projective invariant combined with the vanishing line of the plane that is a prior knowledge of road scenes. the proposed method is implemented into a newly developed on-board lsi. experimental results for real road scenes under various conditions show the effectiveness of the proposed method.
optimal recovery of depth from defocused images using an mrf model. a map-mrf based scheme is proposed for recovering the depth and the focused image of a scene from two defocused images. the space-variant blur parameter and the focused image of the scene are both modeled as mrfs and their map estimates are obtained using simulated annealing. the performance of the proposed scheme is tested on both synthetic as well as real data and the estimates of the depth are found to be better than that of the existing window-based technique.
autocalibration of a projector-screen-camera system: theory and algorithm for screen-to-camera homography estimation. this paper deals with the autocalibration of a systemthat consists of a planar screen, multiple projectors, and acamera. in the system, either multiple projectors or a singlemoving projector projects patterns on a screen while a stationarycamera placed in front of the screen takes images ofthe patterns. we treat the case in which the patterns that theprojectors project toward space are assumed to be known(i.e., the projectors are calibrated), whereas poses of theprojectors are unknown. under these conditions, we considerthe problem of estimating screen-to-camera homographyfrom the images alone. this is intended for cases wherethere is no clue on the screen surface that enables directestimation of the screen-to-camera homography. one applicationis a 6dof input device; poses of a multi-beamprojector freely moving in space are computed from the imagesof beam spots on the screen. the primary contributionof the paper is theoretical results on the uniqueness of solutionsand a noniterative algorithm for the problem. theeffectiveness of the method is shown by experimental resultson synthetic as well as on real images.
finding faces in photographs. two new schemes are presented for finding human faces in a photograph. the first scheme approximates the unknown distributions of the face and the face-like manifolds using higher order statistics (hos). an hos-based data clustering algorithm is also proposed. in the second scheme, the face to non-face and non-face to face transitions are learnt using a hidden markov model (hmm). the hmm parameters are estimated corresponding to a given photograph and the faces are located by examining the optimal state sequence of the hmm. experimental results are presented on the performance of both the schemes.
fast and accurate self-calibration. this paper describes new techniques for self calibration and for recovering the motion from a projective reconstruction when the calibration is known. we show that our approach deals with the ambiguities in self calibration produced by special motions. we extend our techniques to deal with varying calibration parameters. in passing, we prove convergence for the iterative projective reconstruction algorithm of sturm/triggs and berthilsson/heyden/sparr.