waashk/dblp huggingface dataset:
a theory of multiplexed illumination. imaging of objects under variable lighting directions is animportant and frequent practice in computer vision andimage-based rendering. we introduce an approach that significantlyimproves the quality of such images. traditionalmethods for acquiring images under variable illuminationdirections use only a single light source per acquired image.in contrast, our approach is based on a multiplexing principle,in which multiple light sources illuminate the objectsimultaneously from different directions. thus, the objectirradiance is much higher. the acquired images are thencomputationally demultiplexed. the number of image acquisitionsis the same as in the single-source method. theapproach is useful for imaging dim object areas. we givethe optimal code by which the illumination should be multiplexedto obtain the highest quality output. for n imagescorresponding to n light sources, the noise is reduced by\sqrt n/2 relative to the signal. this noise reduction translatesto a faster acquisition time or an increase in density of illuminationdirection samples. it also enables one to use lightingwith high directional resolution using practical setups,as we demonstrate in our experiments.
dynamic rigid motion estimation from weak perspective. "weak perspective" represents a simplified projection model that approximates the imaging process when the scene is viewed under a small viewing angle and its depth relief is small relative to its distance from the viewer. we study how to generate dynamic models for estimating rigid 3d motion from weak perspective. a crucial feature in dynamic visual motion estimation is to decouple structure from motion in the estimation model. the reasons are both geometric-to achieve global observability of the model-and practical, for a structure independent motion estimator allows us to deal with occlusions and appearance of new features in a principled way. it is also possible to push the decoupling even further, and isolate the motion parameters that are affected by the so called "bas relief ambiguity" from the ones that are not. we present a novel method for reducing the order of the estimator by decoupling portions of the state space from the time evolution of the measurement constraint. we use this method to construct an estimator of full rigid motion (modulo a scaling factor) on a six dimensional state space, an approximate estimator for a four dimensional subset of the motion space, and a reduced filter with only two states. the latter two are immune to the bas relief ambiguity. we compare strengths and weaknesses of each of the schemes on real and synthetic image sequences.
polarization-based decorrelation of transparent layers: the inclination angle of an invisible surface. when a transparent surface is present between an observer and an object, an image reflected by the surface may be superimposed on the image of the observed object. we present a new approach to recover the scenes (layers) and to classify which is the reflected/transmitted one, based on imaging through a polarizing filter at two orientations. estimates of the separate layers are obtained by weighted pixel-wise differences of these images, inverting the image formation process. however, the weights depend on the angle of incidence, hence on the inclination of the transparent (invisible) surface. this angle is estimated by seeking the angle-value which (through the weights) leads to decorrelation of the estimated layers. experimental results, obtained using real photos of actual objects, demonstrate the success of angle estimation and consequent layer separation and labeling. the method is shown to be superior to earlier methods where only raw optical data was used.
tales of shape and radiance in multi-view stereo. to what extent can three-dimensional shape and radiancebe inferred from a collection of images? can the two be estimatedseparately while retaining optimality? how shouldthe optimality criterion be computed? when is it necessaryto employ an explicit model of the reflectance properties ofa scene? in this paper we introduce a separation principlefor shape and radiance estimation that applies to lambertianscenes and holds for any choice of norm. when thescene is not lambertian, however, shape cannot be decoupledfrom radiance, and therefore matching image-to-imageis not possible directly. we employ a rank constraint onthe radiance tensor, which is commonly used in computergraphics, and construct a novel cost functional whose minimizationleads to an estimate of both shape and radiancefor non-lambertian objects, which we validate experimentally.
transinformation for active object recognition. this article develops an analogy between object recognition and the transmission of information through a channel based on the statistical representation of the appearances of 3d objects. this analogy provides a means to quantitatively evaluate the contribution of individual receptive field vectors, and to predict the performance of the object recognition process. transinformation also provides a quantitative measure of the discrimination provided by each viewpoint, thus permitting the determination of the most discriminant viewpoints. as an application, the article develops an active object recognition algorithm which is able to resolve ambiguities inherent in a single-view recognition algorithm.
the beltrami flow over implicit manifolds. in many medical computer vision tasks the relevant data isattached to a specific tissue such as the colon or the cortex.this situation calls for regularization techniques whichare defined over surfaces.we introduce in this paper thebeltrami flow over implicit manifolds.this new regularizationtechnique overcomes the over-smoothing of the l2flow and the staircasing effects of the l1 flow, that wererecently suggested via the harmonic map methods.the keyof our approach is first to clarify the link between the intrinsic polyakov action and the implicit harmonic energyfunctional and then use the geometrical understanding ofthe beltrami flow to generalize it to images on implicitlydefined non flat surfaces.it is shown that once again thebeltrami flow interpolates between the l2 and l1 flows onnon-flat surfaces.the implementation scheme of this flowis presented and various experimental results obtained on aset of various real images illustrate the performances of theapproach as well as the differences with the harmonic mapflows.this extension of the beltrami flow to the case of nonflat surfaces opens new perspectives in the regularization ofnoisy data defined on manifolds.
comparing and evaluating interest points. many computer vision tasks rely on feature extraction. inter est points are such features. this paper shows that interest points are geometrically stable under different transformations and have high information content (distinctiveness). these two properties make interest points very successful in the context of image matching. to measure these two properties quantitatively, we introduce two evaluation criteria: repeatability rate and information content.the quality of the interest points depends on the detector used. in this paper several detectors are compared according to the criteria specified above. we determine which detector gives the best results and show that it satisfies the criteria well.
monocular perception of biological motion - detection and labeling. computer perception of biological motion is key to developing convenient and powerful human-computer inter-faces. successful body tracking algorithms have been developed; however, initialization is done by hand.we propose a method for detecting a moving human body and for labeling its parts automatically. it is based on maximizing the joint probability density function (pdf) of the position and velocity of the body parts. the pdf is estimated from training data. dynamic programming is used for calculating efficiently the best global labeling on an approximation of the pdf. the computational cost is on the order of n 4 where n is the number of features detected.we explore the performance of our method with experiments carried on a variety of periodic and non-periodic body motions viewed monocularly for a total of approximately 30,000 frames. point-markers were strapped to the joints of the subject for facilitating image analysis. we find an average of 2.3% labeling error; the experiments also suggest a high degree of viewpoint-invariance.
a model-based vehicle segmentation method for tracking. our goal is to detect and track moving vehicles on a road observed from cameras placed on poles or buildings. inter-vehicle occlusion is significant under these conditions and traditional blob tracking methods will be unable to separate the vehicles in the merged blobs. we use vehicle shape models, in addition to camera calibration and ground plane knowledge, to detect, track and classify moving vehicles in presence of occlusion. we use a 2-stage approach. in the first stage, hypothesis for vehicle types, positions and orientations are formed by a coarse search, which is then refined by a data driven markov chain monte carlo (ddmcmc) process. we show results and evaluations on some real urban traffic video sequence using three types of vehicle models.
computing ritz approximations of primary images. ritz vectors approximate eigenvectors that are a common choice for primary images in content based indexing. they can be computed efficiently even when the images are accessed through slow communication such as the internet. we develop an algorithm that computes ritz vectors in one pass through the images. when iterated, the algorithm can recover the exact eigenvectors. in applications to image indexing and learning it may be necessary to compute primary images for indexing many sub-categories of the image set the proposed algorithm can compute these additional primary images "offline", without the image data much more costly even when access to the images is inexpensive.
manifold clustering. manifold learning has become a vital tool in data driven methods for interpretation of video, motion capture, and handwritten character data when they lie on a low dimensional, non-linear manifold. this work extends manifold learning to classify and parameterize unlabeled data which lie on multiple, intersecting manifolds. this approach significantly increases the domain to which manifold learning methods can be applied, allowing parameterization of example manifolds such as figure eights and intersecting paths which are quite common in natural data sets. this approach introduces several technical contributions which may be of broader interest, including node-weighted multi-dimensional scaling and a fast algorithm for weighted low-rank approximation for rank-one weight matrices. we show examples for intersecting manifolds of mixed topology and dimension and demonstrations on human motion capture data.
indexing images by trees of visual content. an unsupervised algorithm for arranging an image database as a binary tree is described. tree nodes are associated with image subsets, maintaining the property that the similarity among the images associated with the children of a node is higher than the similarity among the images associated with the parent node. experiments with datasets of hundreds and thousands of images show that shallow trees can produce clustering into "meaningful" classes. visual-content search trees can be used to automate image retrieval by content, orhelp a human to interactively search for images.
limitations of markov random fields as models of textured images of real surfaces. we investigate to what extent textures can be distinguished using conditional markov fields and small samples. we establish that the least square (ls) estimator is the only reasonable choice for this task and we prove its asymptotic consistency and normality for a general class of random fields that include gaussian markov fields as a special case. the performance of this estimator when applied to textured images of real surfaces is poor if small boxes are used (20/spl times/20 or less). we investigate the nature of this problem by comparing the behavior predicted by the rigorous theory to the one that has been experimentally observed. our analysis reveal that 20/spl times/20 samples contain enough information to distinguish between the textures in our experiments and that the poor performance mentioned above should be attributed to the fact that conditional markov fields do not provide accurate models for textured images of many real surfaces. a more general model that exploits more efficiently the information contained in small samples is also suggested.
utilizing scatter for pixel subspace selection. measures of scatter are used in statistical pattern recognition to identify and select important features, computed as linear combinations of the given features. examples include principal components and linear discriminants. the classic computational procedures require eigenvector decomposition of large matrices, and in the case of images they are only practical for identifying a low dimensional feature subspace.we investigate the case in which the selected features are required to be a subset of the given features. it is shown that the same scatter measures used in the general case can also be used in this discrete selection case, but the computational procedure no longer involves matrix eigenvector decomposition.instead, the selection of pixels that optimize scatter measures can be accomplished by a very simple and efficient discrete optimization technique that runs in linear time regardless of the subspace size. applications to clustering and content based indexing are discussed.
active blobs. a new region-based approach to nonrigid motion tracking is described. shape is defined in terms of a deformable triangular mesh that captures object shape plus a color texture map that captures object appearance. photometric variations are also modeled. nonrigid shape registration and motion tracking are achieved by posing the problem as an energy-based, robust minimization procedure. the approach provides robustness to occlusions, wrinkles, shadows, and specular highlights. the formulation is tailored to take advantage of texture mapping hardware available in many workstations, pc's, and game consoles. this enablesnonrigid tracking at speeds approaching video rate.
convex grouping combining boundary and region information. convexity is an important geometric property of many natural and man-made structures. prior research has shown that it is imperative to many perceptual-organization and image-understanding tasks. this paper presents a new grouping method for detecting convex structures from noisy images in a globally optimal fashion. particularly, this method combines both region and boundary information: the detected structural boundary is closed and well aligned with detected edges while the enclosed region has good intensity homogeneity. we introduce a ratio-form cost function for measuring the structural desirability, which avoids a possible bias to detect small structures. a new fragment-pruning algorithm is developed to achieve the structural convexity. the proposed method can also be extended to detect open boundaries, which correspond to the structures that are partially cropped by the image perimeter, and incorporate a human-computer interaction for detecting a convex boundary around a specified point. we test the proposed method on a set of real images and compare it with the jacobs¿ convex-grouping method.
complete scene structure from four point correspondences. a technique is presented for computing 3d scene structure from point and line features in monocular image sequences. unlike previous methods, the technique guarantees the completeness of the recovered scene, ensuring that every scene feature that is detected in each image is reconstructed. the approach relies on the presence of four or more reference features whose correspondences are known in all the images. under an orthographic or affine camera model, the parallax of the reference features provides constraints that simplify the recovery of the rest of the visible scene. an efficient recursive algorithm is described that uses a unified framework for point and line features. the algorithm integrates the tasks of feature correspondence and structure recovery, ensuring that all reconstructible features are tracked. in addition, the algorithm is immune to outliers and feature drift, two weaknesses of existing structure from motion techniques. experimental results are presented for real images.
model-based multiple view reconstruction of people. this paper presents a framework to reconstruct a scenecaptured in multiple camera views based on a prior modelof the scene geometry. the framework is applied to thecapture of animated models of people. a multiple camerastudio is used to simultaneously capture a moving personfrom multiple viewpoints. a humanoid computer graphicsmodel is animated to match the pose at each time frame.constrained optimisation is then used to recover the multipleview correspondence from silhouette, stereo and featurecues, updating the geometry and appearance of the model.the key contribution of this paper is a model-based computervision framework for the reconstruction of shape andappearance from multiple views. this is compared to currentmodel-free approaches for multiple view scene capture.the technique demonstrates improved scene reconstructionin the presence of visual ambiguities and providesthe means to capture a dynamic scene with a consistentmodel that is instrumented with an animation structure toedit the scene dynamics or to synthesise new content.
plenoptic image editing. this paper presents a new class of interactive image editing operations designed to maintain consistency between multiple images of a physical 3d scene. the distinguishing feature of these operations is that edits to any one image propagate automatically to all other images asif the (unknown) 3d scene had itself been modified. the modified scene can then be viewed interactively from any other camera viewpoint and under different scene illuminations. the approach is useful first as a power-assist that enables a user to quickly modify many images by editing just a few, and second as a means for constructing and editing image-based scene representations by manipulating a set of photographs. the approach works by extending operations like image painting, scissoring, and morphing so that they alter a scene's generalized plenoptic function in a physically-consistent way, thereby affecting scene appearance from all viewpoints simultaneously. a key element in realizing these operations is a new volumetric decomposition technique for reconstructing an scene's plenoptic function from an incomplete set of camera viewpoints.
spherical matching for temporal correspondence of non-rigid surfaces. this paper introduces spherical matching to estimate dense temporal correspondence of non-rigid surfaces with genus-zero topology. the spherical domain gives a consistent 2d parameterisation of non-rigid surfaces for matching. non-rigid 3d surface correspondence is formulated as the recovery of a bijective mapping between two surfaces in the 2d domain. formulating matching as a 2d bijection guarantees a continuous one-to-one surface correspondence without overfolding. this overcomes limitations of direct estimation of non-rigid surface correspondence in the 3d domain. a multiple resolution coarse-to-fine algorithm is introduced to robustly estimate the dense correspondence which minimises the disparity in shape and appearance between two surfaces. spherical matching is applied to derive the temporal correspondence between non-rigid surfaces reconstructed at successive frames from multiple view video sequences of people. dense surface correspondence is recovered across complete motion sequences for both textured and uniform regions, without the requirement for a prior model of human shape or kinematic structure for tracking.
a theory of inverse light transport. in this paper we consider the problem of computing and removing interreflections in photographs of real scenes. towards this end, we introduce the problem of inverse light transport ¿ given a photograph of an unknown scene, decompose it into a sum of n-bounce images, where each image records the contribution of light that bounces exactly n times before reaching the camera. we prove the existence of a set of interreflection cancelation operators that enable computing each n-bounce image by multiplying the photograph by a matrix. this matrix is derived from a set of "impulse images" obtained by probing the scene with a narrow beam of light. the operators work under unknown and arbitrary illumination, and exist for scenes that have arbitrary spatially-varying brdfs. we derive a closed-form expression for these operators in the lambertian case and present experiments with textured and untextured lambertian scenes that confirm our theory¿s predictions.
minimally-supervised classification using multiple observation sets. this paper discusses building complex classifiers from a single labeled example and vast number of unlabeled observation sets, each derived from observation of a single processor object. when data can be measured by observation, it is often plentiful and it is often possible to make more than one observation of the state of a process or object. this paper discusses how to exploit the variability across such sets of observations of the same object to estimate class labels for unlabeled examples given a minimal number of labeled examples. in contrast to similar semi-supervised classification procedures that define the likelihood that two observations share a label as a function of the embedded distance between the two observations, this method uses the naive bayes estimate of how often the two observations did result from the same observed process. exploiting this additional source of information in an iterative estimation procedure can generalize complex classification models from single labeled observations. some examples involving classification of tracked objects in a low-dimensional feature space given thousands of unlabeled observation sets are used to illustrate the effectiveness of this method.
optimal subpixel matching of contour chains and segments. this paper introduces a new general purpose algorithm that allows the optimal geometric match between contours to be determined, that is the transformation yielding a minimal deformation is obtained. the algorithm relies only on the geometric properties of the contours and does not call for any other constraint, so that it is particularly suitable when no parameterization of title deformation is available or desirable. contour deformation is explicitly incorporated in the computation, allowing for a thorough use of all geometric information available. moreover, no discretization is involved in the computation, resulting in two main advantages: first, the algorithm is robust to differences in the segmentation of contours and allows the matching of polygonal approximations of contours with very little loss of precision, second, subpixel precision matching can be achieved.
spectral partitioning for structure from motion. we propose a spectral partitioning approach for large-scaleoptimization problems, specifically structure from motion.in structure from motion, partitioning methods reduce theproblem into smaller and better conditioned subproblemswhich can be efficiently optimized.our partitioning methoduses only the hessian of the reprojection error and its eigenvector.we show that partitioned systems that preserve theeigenvectors corresponding to small eigenvalues result inlower residual error when optimized.we create partitionsby clustering the entries of the eigenvectors of the hessiancorresponding to small eigenvalues.this is a more generaltechnique than relying on domain knowledge and heuristicssuch as bottom-up structure from motion approaches.simultaneously,it takes advantage of more information thangeneric matrix partitioning algorithms.