waashk/dblp huggingface dataset:
bilinear voting. a geometric-vision approach to solve bilinear problems in general, and the color constancy and illuminant estimation problem in particular, is presented in this paper. we show a general framework, based on ideas from the generalized (probabilistic) hough transform, to estimate the unknown variables in the bilinear form. in the case of illuminant and reflectance estimation in natural images, each image pixel "votes" for possible illuminants (or reflectance), and the estimation is based on cumulative votes. in the general case, the voting is for the parameters of the bilinear model. the framework is natural for the introduction of physical constraints. for the case of illuminant estimation, we briefly show the relation of this work with previous algorithms for color constancy, and present examples.
steerable wedge filters. steerable filters, as developed by freeman and adelson (1991), are a class of rotation-invariant linear operators that may be used to analyze local orientation patterns in imagery. the most common examples of such operators are directional derivatives of gaussians and their 2d hilbert transforms. the inherent symmetry of these filters produces an orientation response that is periodic with period /spl pi/, even when the underlying image structure does not have such symmetry. this problem may be alleviated by reconsidering the full class of steerable filters. we develop a family of even- and odd-symmetric steerable filters that have a spatially asymmetric "wedge-like" shape and are optimally localized in their orientation response. unlike the original steerable filters, these filters are not based on directional derivatives and the hilbert transform relationship is imposed on their angular components. we demonstrate the ability of these filters to properly represent oriented structures.
regression based bandwidth selection for segmentation using parzen windows. we consider the problem of segmentation of images that can be modelled as piecewise continuous signals having unknown, non-stationary statistics. we propose a solution to this problem which first uses a regression framework to estimate the image pdf, and then mean-shift to find the modes of this pdf. the segmentation follows from mode identification wherein pixel clusters or image segments are identified with unique modes of the multi-modal pdf. each pixel is mapped to a mode using a convergent, iterative process. the effectiveness of the approach depends upon the accuracy of the (implicit) estimate of the underlying multi-modal density function and thus on the bandwidth parameters used for its estimate using parzen windows. automatic selection of bandwidth parameters is a desired feature of the algorithm. we show that the proposed regression-based model admits a realistic framework to automatically choose bandwidth parameters which minimizes a global error criterion. we validate the theory presented with results on real images.
separating transparent layers of repetitive dynamic behaviors. in this paper we present an approach for separating two transparent layers of complex non-rigid scene dynamics. the dynamics in one of the layers is assumed to be repetitive, while the other can have any arbitrary dynamics. such repetitive dynamics includes, among other, human actions in video (e.g., a walking person), or a repetitive musical tune in audio signals. we use a global-to-local space-time alignment approach to detect and align the repetitive behavior. once aligned, a median operator applied to space-time derivatives is used to recover the intrinsic repeating behavior, and separate it from the other transparent layer. we show results on synthetic and real video sequences. in addition, we show the applicability of our approach to separating mixed audio signals (from a single source).
multi-view reconstruction using photo-consistency and exact silhouette constraints: a maximum-flow formulation. this paper describes a novel approach for reconstructing a closed continuous surface of an object from multiple calibrated color images and silhouettes. any accurate reconstruction must satisfy (1) photo-consistency and (2) silhouette consistency constraints. most existing techniques treat these cues identically in optimization frameworks where silhouette constraints are traded off against photo-consistency and smoothness priors. our approach strictly enforces silhouette constraints, while optimizing photo-consistency and smoothness in a global graph-cut framework. we transform the reconstruction problem into computing max-flow/mincut in a geometric graph, where any cut corresponds to a surface satisfying exact silhouette constraints (its silhouettes should exactly coincide with those of the visual hull); a minimum cut is the most photo-consistent surface amongst them. our graph-cut formulation is based on the rim mesh, (the combinatorial arrangement of rims or contour generators from many views) which can be computed directly from the silhouettes. unlike other methods, our approach enforces silhouette constraints without introducing a bias near the visual hull boundary and also recovers the rim curves. results are presented for synthetic and real datasets.
affine reconstruction of curved surfaces from uncalibrated views of apparent contours. in this paper, we show that even if the camera is uncalibrated, and its translational motion is unknown, curved surfaces can be reconstructed from their apparent contours up to a 3d affine ambiguity. furthermore, we show that even if the reconstruction is non-metric (non-euclidean), we can still extract useful information for many computer vision applications just from the apparent contours.we first show that if the camera undergoes pure translation (unknown direction and magnitude), the epipolar geometry can be recovered from the apparent contours without using any search or optimisation process. the extracted epipolar geometry is next used for reconstructing curved surfaces from the deformations of the apparent contours viewed from uncalibrated cameras. the result is applied to distinguishing curved surfaces from fixed features in images. it is also shown that the time-to-contact to the curved surfaces can be computed from simple measurements of the apparent contours. the proposed method is implemented and tested on real images of curved surfaces.
discovering objects and their localization in images. we seek to discover the object categories depicted in a set of unlabelled images. we achieve this using a model developed in the statistical text literature: probabilistic latent semantic analysis (plsa). in text analysis this is used to discover topics in a corpus using the bag-of-words document representation. here we treat object categories as topics, so that an image containing instances of several categories is modeled as a mixture of topics. the model is applied to images by using a visual analogue of a word, formed by vector quantizing sift-like region descriptors. the topic discovery approach successfully translates to the visual domain: for a small set of objects, we show that both the object categories and their approximate spatial layout are found without supervision. performance of this unsupervised method is compared to the supervised approach of fergus et al. [8] on a set of unseen images containing only one object per image. we also extend the bag-of-words vocabulary to include ¿doublets¿ which encode spatially local co-occurring regions. it is demonstrated that this extended vocabulary gives a cleaner image segmentation. finally, theclassification and segmentation methods are applied to a set of images containing multiple objects per image. these results demonstrate that we can successfully build object class models from an unsupervised analysis of images.
appearance sampling for obtaining a set of basis images for variable illumination. previous studies have demonstrated that the appearance ofan object under varying illumination conditions can be representedby a low-dimensional linear subspace. a set ofbasis images spanning such a linear subspace can be obtainedby applying the principal component analysis (pca)for a large number of images taken under different lightingconditions. while the approaches based on pca havebeen used successfully for object recognition under varyingillumination conditions, little is known about how many imageswould be required in order to obtain the basis imagescorrectly. in this study, we present a novel method for analyticallyobtaining a set of basis images of an object forarbitrary illumination from input images of the object takenunder a point light source. the main contribution of ourwork is that we show that a set of lighting directions canbe determined for sampling images of an object dependingon the spectrum of the object's brdf in the angularfrequency domain such that a set of harmonic images canbe obtained analytically based on the sampling theorem onspherical harmonics. in addition, unlike the previously proposedtechniques based on spherical harmonics, our methoddoes not require the 3d shape and reflectance properties ofan object used for rendering harmonics images of the objectsynthetically.
video google: a text retrieval approach to object matching in videos. we describe an approach to object and scene retrievalwhich searches for and localizes all the occurrences of auser outlined object in a video. the object is represented bya set of viewpoint invariant region descriptors so that recognitioncan proceed successfully despite changes in viewpoint,illumination and partial occlusion. the temporalcontinuity of the video within a shot is used to track theregions in order to reject unstable regions and reduce theeffects of noise in the descriptors.the analogy with text retrieval is in the implementationwhere matches on descriptors are pre-computed (using vectorquantization), and inverted file systems and documentrankings are used. the result is that retrieval is immediate,returning a ranked list of key frames/shots in the manner ofgoogle.the method is illustrated for matching on two full lengthfeature films.
using extended light sources for modeling object appearance under varying illumination. in this study, we demonstrate the effectiveness of using extended light sources for modeling the appearance of an object for varying illumination. extended light sources have a radiance distribution that is similar to that of the gaussian function and have the potential of functioning as a low-pass filter when the appearance of an object is sampled under them. this enables us to obtain a set of basis images of an object for variable illumination from input images of the object taken under those light sources without suffering aliasing caused by insufficient sampling of its appearance. furthermore, extended light sources are useful in terms of reducing high contrast in image intensities due to specular and diffuse reflection components. this helps us observe both specular and diffuse reflection components of an object in the same image taken with a single shutter speed. we have tested our proposed approach based on extended light sources with objects of complex appearance that are generally difficult to model using image-based modeling techniques.
weighted and robust incremental method for subspace learning. visual learning is expected to be a continuous and robustprocess, which treats input images and pixels selectively.in this paper we present a method for subspace learning,which takes these considerations into account. wepresent an incremental method, which sequentially updatesthe principal subspace considering weighted influence ofindividual images as well as individual pixels within an image.this approach is further extended to enable determinationof consistencies in the input data and imputation of thevalues in inconsistent pixels using the previously acquiredknowledge, resulting in a novel incremental, weighted androbust method for subspace learning.
perceptual organization in an interactive sketch editing application. the paper shows how techniques from computational vision can be deployed to support interactive sketch editing. while conventional computer supported drawing tools give users access to visible marks or image objects at a single level of abstraction, a human user's visual system rapidly constructs complex groupings and associations among image elements according to his or her immediate purposes. we have been exploring perceptually supported sketch editors in which computer vision algorithms run continuously, behind the scenes, to afford users efficient access to emergent visual objects in a drawing. we employ a flexible image interpretation architecture based on token grouping in a multiscale blackboard data structure. this organization supports multiple perceptual interpretations of line drawing data, domain specific knowledge bases for interpreting visual structures, and natural gesture based selection of visual objects.
conditional random fields for contextual human motion recognition. we present algorithms for recognizing human motion in monocular video sequences, based on discriminative conditional random field (crf) and maximum entropy markov models (memm). existing approaches to this problem typically use generative (joint) structures like the hidden markov model (hmm). therefore they have to make simplifying, often unrealistic assumptions on the conditional independence of observations given the motion class labels and cannot accommodate overlapping features or long term contextual dependencies in the observation sequence. in contrast, conditional models like the crfs seamlessly represent contextual dependencies, support efficient, exact inference using dynamic programming, and their parameters can be trained using convex optimization. we introduce conditional graphical models as complementary tools for human motion recognition and present an extensive set of experiments that show how these typically outperform hmms in classifying not only diverse human activities like walking, jumping, running, picking or dancing, but also for discriminating among subtle motion styles like normal walk and wander walk.
asset-2: real-time motion segmentation and shape tracking. the paper describes how image sequences taken by a moving video camera may be processed to detect and track moving objects against a moving background in real-time. the motion segmentation and shape tracking system as known as asset-2-a scene segmenter establishing tracking, version 2. motion is found by tracking image features, and segmentation is based on first-order (i.e., six parameter) flow fields. shape tracking is performed using two dimensional radial map representation. the system runs in real-time, and is accurate and reliable. it requires no camera calibration and no knowledge of the camera's motion.
fast pose estimation with parameter-sensitive hashing. example-based methods are effective for parameter estimationproblems when the underlying system is simple orthe dimensionality of the input is low. for complex andhigh-dimensional problems such as pose estimation, thenumber of required examples and the computational complexityrapidly become prohibitively high. we introduce anew algorithm that learns a set of hashing functions that efficientlyindex examples in a way relevant to a particularestimation task. our algorithm extends locality-sensitivehashing, a recently developed method to find approximateneighbors in time sublinear in the number of examples. thismethod depends critically on the choice of hash functions;we show how to find the set of hash functions that are optimallyrelevant to a particular estimation problem. experimentsdemonstrate that the resulting algorithm, which wecall parameter-sensitive hashing, can rapidly and accuratelyestimate the articulated pose of human figures from alarge database of example images.
computing map trajectories by representing, propagating and combining pdfs over groups. this paper addresses the problem of computing the trajectoryof a camera from sparse positional measurementsthat have been obtained from visual localisation, and densedifferential measurements from odometry or inertial sensors.a fast method is presented for fusing these two sourcesof information to obtain the maximum a posteriori estimateof the trajectory. a formalism is introduced for representingprobability density functions over euclidean transformations,and it is shown how these density functions can bepropagated along the data sequence and how multiple estimatesof a transformation can be combined. a three-passalgorithm is described which makes use of these results toyield the trajectory of the camera.simulation results are presented which are validatedagainst a physical analogue of the vision problem, and resultsare then shown from sequences of approximately 1,800frames captured from a video camera mounted on a go-kart.several of these frames are processed using computer visionto obtain estimates of the position of the go-kart. the algorithmfuses these estimates with odometry from the entiresequence in 150 ms to obtain the trajectory of the kart.
image statistics and anisotropic diffusion. many sensing techniques and image processing applicationsare characterized by noisy, or corrupted, image data.anisotropic diffusion is a popular, and theoretically wellunderstood, technique for denoising such images. diffusionapproaches however require the selection of an "edgestopping" function, the definition of which is typically adhoc. we exploit and extend recent work on the statisticsof natural images to define principled edge stopping functionsfor different types of imagery. we consider a varietyof anisotropic diffusion schemes and note that they computespatial derivatives at fixed scales from which we estimatethe appropriate algorithm-specific image statistics. goingbeyond traditional work on image statistics, we also modelthe statistics of the eigenvalues of the local structure tensor.novel edge-stopping functions are derived from these imagestatistics giving a principled way of formulating anisotropicdiffusion problems in which all edge-stopping parametersare learned from training data.
recovering facial shape and albedo using a statistical model of surface normal direction. this paper describes how facial shape can be modelled using a statistical model that captures variations in surface normal direction. to construct this model we make use of the azimuthal equidistant projection to map surface normals from the unit sphere to points on a local tangent plane. the variations in surface normal direction are captured using the covariance matrix for the projected point positions. this allows us to model variations in face shape using a standard point distribution model. we train the model on fields of surface normals extracted from range data and show how to fit the model to intensity data using constraints on the surface normal direction provided by lambert¿s law. we demonstrate that this process yields accurate facial shape recovery and allows an estimate of the albedo map to be made from single, real world face images.
the optimal axial interval in estimating depth from defocus. we analyze the effect of perturbations on the estimation of depth from defocus (dfd) implemented by changing the focus setting (e.g., axially moving the sensor). the analysis yields the optimal change of focus setting, and the spatial frequencies for which estimation is most robust. for stable estimation at all spatial frequencies, the change in focus setting should be less than twice the depth of field. for the most robust estimation in the highest spatial frequencies the axial interval should be equal to the depth of field.
resolving hand over face occlusion. the ability to segment or track the hand is an important problem in computer vision. while various solutions have been proposed, many methods do not work against complex or cluttered backgrounds. solving these cases is essential to solving many problems in the domain of computer vision such as, human-computer interaction (hci), surveillance, and virtual reality (i.e., augmented desks). this paper presents a method to segment the hand over complex backgrounds, such as the face. the similar colors and texture of the hand and face make the problem particularly challenging. the method is not restricted to only segmenting hands across faces and uses no knowledge of hands. our method is based on the underlying concept of an image force field. in this representation change is measured through how particles move through the field. each individual image location consists of a vector value which is a nonlinear combination of the remaining pixels in the image. we introduce and develop a novel physics-based feature that is able to measure regional structure in the image thus avoiding the problem of local pixel-based analysis, which breaks down under our conditions. the regional image structure changes in the occluded region during occlusion, while elsewhere the regional structure remains relatively constant. we model the regional image structure at all image locations over time using a mixture of gaussians (mog) to detect the occluded region in the image. we have tested the method on a number of sequences demonstrating the versatility of the proposed approach.
separation of transparent layers using focus. consider situations where the depth at each point in the scene is multi-valued, due to the presence of a virtual image semi-reflected by a transparent surface. the semi-reflected image is linearly superimposed on the image of the object that is behind the transparent surface. a novel approach is proposed for the recovery of the superimposed layers. by searching for the images in which either of the objects (layers) is focused, the transparent areas are detected and an estimate of the depth map of each layer is obtained. as a result of the focusing, an initial separation of the layers is achieved. the separation is enhanced via mutual blurring of the perturbing components in the images, based on the depths estimate and the parameters of the imaging system.
temporalboost for event recognition. this paper contributes a new boosting paradigm to achieve detection of events in video. previous boosting paradigms in vision focus on single frame detection and do not scale to video events. thus new concepts need to be introduced to address questions such as determining if an event has occurred, localizing the event, handling same action performed at different speeds, incorporating previous classifier responses into current decision, using temporal consistency of data to aid detection and recognition. the proposed method has the capability to improve weak classifiers by allowing them to use previous history in evaluating the current frame. a learning mechanism built into the boosting paradigm is also given which allows event level decisions to be made. this is contrasted with previous work in boosting which uses limited higher level temporal reasoning and essentially makes object detection decisions at the frame level. our approach makes extensive use of temporal continuity of video at the classifier and detector levels. we also introduce a relevant set of activity features. features are evaluated at multiple zoom levels to improve detection. we show results for a system that is able to recognize 11 actions.