waashk/dblp huggingface dataset:
catadioptric sensors that approximate wide-angle perspective projections. we present two families of reflective surfaces that are capable of providing a wide field of view, and yet still approximate a perspective projection to a high degree. these surfaces are derived by considering a plane perpendicular to the axis of a surface of revolution and finding the equations governing the distortion of the image of the plane in this surface.we then view this relation as a differential equation and prescribe the distortion term to be linear. by choosing appropriate initial conditions for the differential equation and solving it numerically, we derive the surface shape and obtain a precise estimate as to what degree the resulting sensor can approximate a perspective projection. thus, these surfaces act as computational sensors, allowing for a wide-angle perspective view of a scene without processing the image in software. the applications of such a sensor should be numerous, including surveillance, robotics and traditional photography.recently, many researchers in the robotics and vision community have begun to consider visual sensors that are able to obtain wide fields of view. such devices are the natural solution to various difficulties encountered with conventional imaging systems.the two most common means of obtaining wide fields of view are fish-eye lenses and reflective surfaces, also known as catoptrics. when catoptrics are combined with conventional lens systems, known as dioptrics, the resulting sensors are known as catadioptrics. the possible uses of these systems include applications such as robot control and surveillance. in this paper, we will consider only catadioptri-based sensors. often such systems consist of a camera pointing at a convex mirror.how to interpret and make use of the visual information obtained by such systems, e.g. how they should be used to control robots is not at all obvious. there are infinitely many different shapes that a mirror can have, and at least two different camera models (perspective and orthographic projection) with which to combine each mirror.the properties of the resulting sensors are very sensitive to these choices. the classic need for wide-angle lenses have, of course been in photography. in particular, underwater and architectural photography are two examples in which having a wide-angle lens is often crucial. the commercially available lens with the widest field of view (without radial distortion) that the authors are aware of is the nikon 13mm f/5.6 nikkor ais, which provides a field of view of 118 degrees at a cost of $(us) 12000. note that our prototype orthographic sensor provides a field of view of 142 degrees.
real-time estimation of human body posture from monocular thermal images. this paper introduces a new real-time method to estimate the posture of a human from thermal images acquired by an infrared camera regardless of the background and lighting conditions. distance transformation is performed for the human body area extracted from the thresholded thermal image for the calculation of the center of gravity. after the orientation of the upper half of the body is obtained by calculating the moment of inertia, significant points such as the top of the head, the tips of the hands and foot are heuristically located. in addition, the elbow and knee positions are estimated from the detected (significant) points using a genetic algorithm based learning procedure. the experimental results demonstrate the robustness of the proposed algorithm and real-time (faster than 20 frames per second) performance.
calibration of a structured light system: a projective approach. we present in this paper a novel calibration method that uses cross ratio to compute world points falling onto any given light stripe plane of a structured light system. we show that, by using 4 known non-coplanar sets of 3 collinear world points, the direct 4 x 3 image to-world transformation matrix for each light stripe plane can also be recovered from plane-to-plane homography. preliminary experiments conducted with a calibration target and a mannequin suggest that this novel calibration method is robust and is applicable to many shape measurement tasks.
linear fitting with missing data: applications to structure-from-motion and to characterizing intensity images. several vision problems can be reduced to the problem of fitting a linear surface of low dimension to data, including the problems of structure-from-affine-motion, and of characterizing the intensity images of a lambertian scene by constructing the {\em intensity manifold}. for these problems, one must deal with a data matrix with some missing elements. in structure-from-motion, missing elements will occur if some point features are not visible in some frames. to construct the intensity manifold missing matrix elements will arise when the surface normals of some scene points do not face the light source in some images. we propose a novel method for fitting a low rank matrix to a matrix with missing elements. we show experimentally that our method produces good results in the presence of noise. these results can be either used directly, or can serve as an excellent starting point for an iterative method.
3-d to 2-d recognition with regions. this paper presents a novel approach to parts-based object recognition in the presence of occlusion. we focus on the problem of determining the pose of a 3-d object from a single 2-d image when convex parts of the object have been matched to corresponding regions in the image. we consider three types of occlusions: self-occlusion, occlusions whose locus is identified in the image, and completely arbitrary occlusions. we derive efficient algorithms for the first two cases, and characterize their performance. for the last case, we prove that the problem of finding valid poses is computationally hard, but provide an efficient, approximate algorithm. this work generalizes our previous work on region-based object recognition, which focused on the case of planar models.
image based view synthesis of articulated agents. using a combination of techniques from visual representations, view synthesis, and visual-motor model estimation, we present a method for animating movements of an articulated agent (e.g. human or robot arm), without the use of any prior models or explicit 3d information. the information needed to generate simulated images can be acquired either on or off line, by watching the agent doing an arbitrary, possibly unrelated task.we present experimental results synthesizing image sequences of the simulated movement of a human arm and a puma 760 robot arm. control is in either image (camera), motor (joint), or cartesian world coordinates. we have created a user interface, where a user can input a movement program, and then upon execution, view movies of the (simulated) agent executing the program, along with the instantaneous values of the dynamics variables
building reconstruction from optical and range images. a technique is introduced for extracting and reconstructing a wide class of building types from a registered range image and optical image. an attentional focus stage, followed by model indexing, allows top-down robust surface fittting to reconstruct the 3d nature of the buildings in the data. because of the effectiveness of model selection, top-down processing of noisy range data still succeeds and the algorithm is capable of detecting and reconstructing several different building roof classes, including fiat single level, fiat multi-leveled, peaked, and curued rooftops. the algorithm is applicable to range data that may have been collected from several different range sensor types. we demonstrate reconstructions of different buildings classes in the presence of large amounts of noise. our results underline the usefuless of range data when processed in the context of a focus-of-attention area derived from the monocular optical image.
parametrized structure from motion for 3d adaptive feedback tracking of faces. a real-time system is described for automatically detecting, modeling and tracking faces in 3d. a closed loop approach is proposed which utilizes structure from motion to generate a 3d model of a face and then feed back the estimated structure to constrain feature tracking in the next frame. the system initializes by using skin classification, symmetry operations, 3d warping and eigenfaces to find a face. feature trajectories are then computed by ssd or correlation-based tracking. the trajectories are simultaneously processed by an extended kalman filter to stably recover 3d structure, camera geometry and facial pose. adaptively weighted estimation is used in this filter by modeling the noise characteristics of the 2d image patch tracking technique. in addition, the structural estimate is constrained by using parametrized models of facial structure (eigen-heads). the kalman filter's estimate of the 3d state and motion of the face predicts the trajectory of the features which constrains the search space for the next frame in the video sequence. the feature tracking and kalman filtering closed loop system operates at 25 hz.
reconstruction of linearly parameterized models from single images with a camera of unknown focal length. this paper deals with the problem of recovering the dimensions of an object and its pose from a single image acquired with a camera of unknown focal length. it is assumed that the object in question can be modeled as a polyhedron where the coordinates of the vertices can be expressed as a linear function of a dimension vector, $\lambda$. the reconstruction program takes as input, a set of correspondences between features in the model and features in the image. from this information, the program determines an appropriate projection model for the camera (scaled orthographic or perspective), the dimensions of the object, its pose relative to the camera and, in the case of perspective projection, the focal length of the camera. this paper describes how the reconstruction problem can be framed as an optimization over a compact set with low dimension¿no more than four. this optimization problem can be solved efficiently by coupling standard nonlinear optimization techniques with a multistart method which generates multiple starting points for the optimizer by sampling the parameter space uniformly. the result is an efficient, reliable solution system that does not require initial estimates for any of the parameters being estimated.
sitecity: a semi-automated site modelling system. this paper presents sitecity, a semi-automated building extraction system integrating photogrammetry, geometric constraints and image understanding algorithms. existing automated building extraction systems produce mixed results and it is clear that human intervention is required to correct mistakes from fully automated systems. sitecity gives human operators the ability to construct and manipulate three dimensional building objects using multiple images. image understanding algorithms are integrated into sitecity to assist users. the automated processes in sitecity use user-delineated roof boundaries as cues, and attempt to locate the floor of a building and match the building object in other images. in addition, photogrammetric cues are used to assist automated processes. these automated processes are described and their performance is evaluated, illustrating that automated processes in sitecity produce comparable performance to that of human subjects.
object recognition using appearance-based parts and relations. the recognition of general three-dimensional objects in cluttered scenes is a challenging problem. in particular, the design of a good representation suitable to model large numbers of generic objects that is also robust to occlusion has been an stumbling block in achieving success. in this paper, we propose a representation using appearance-based parts and relations to overcome these problems. appearance-based parts and relations are defined in terms of closed regions and the union of these regions, respectively. the regions are segmented using the mdl principle, and their appearance is obtained from collection of images and compactly represented by parametric manifolds in the two eigenspaces spanned by the parts and the relations.
image indexing using color correlograms. we define a new image feature called the color correlogram and use it for image indexing and comparison. this feature distills the spatial correlation of colors, and is both effective and inexpensive for content-based image retrieval. the correlogram robustly tolerates large changes in appearance and shape caused by changes in viewing positions, camera zooms, etc. experimental evidence suggests that this new feature outperforms not only the traditional color histogram method but also the recently proposed histogram refinement methods for image indexing/retrieval.
recognizing objects by matching oriented points. we present an approach to recognition of complex objects in cluttered 3-d scenes that does not require feature extraction or segmentation. our object representation comprises descriptive images associated with each oriented point on the surface of an object. using a single point basis constructed from an oriented point, the position of other points on the surface of the object can be described by two parameters. the accumulation of these parameters for many points on the surface of the object results in an image at each oriented point. these images, localized descriptions of the global shape of the object, are invariant to rigid transformations. through correlation of images, point correspondences between a model and scene data are established and then grouped using geometric consistency. the effectiveness of our algorithm is demonstrated with results showing recognition of complex objects in cluttered scenes with occlusion.
statistical color models with application to skin detection. the existence of large image datasets such as the set of photos on the world wide web make it possible to build powerful generic models for low-level image attributes like color using simple histogram learning techniques. we describe the construction of color models for skin and non-skin classes from a dataset of nearly 1 billion labelled pixels. these classes exhibit a surprising degree of separability which we exploit by building a skin pixel detector achieving a detection rate of 80% with 8.5% false positives. we compare the performance of histogram and mixture models in skin detection and find histogram models to be superior in accuracy and computational cost. using aggregate features computed from the skin pixel detector we build a surprisingly effective detector for naked people. our results suggest that color can be a more powerful cue for detecting people in unconstrained imagery than was previously suspected. we believe this work is the most comprehensive and detailed exploration of skin color models to date.
recognizing three-dimensional objects by comparing two-dimensional images. in this paper we address the problem of recognizing an object from a novel viewpoint, given a single ``model'' view of that object. as is common in model-based recognition, objects and images are represented as sets of feature points. we present an efficient algorithm for determining whether two sets of image points (in the plane) could be projections of a common object (a three-dimensional point set). the method relies on the fact that two sets of points in the plane are orthographic projections of the same three-dimensional point set exactly when they have a common projection onto a line. this is a form of the well-known epipolar constraint used in stereopsis. our algorithm can be used to recognize an object by comparing a stored two-dimensional view of the object against an unknown view, without requiring the correspondence between points in the views to be known a priori. we provide some examples illustrating the approach.
skin and bones: multi-layer, locally affine, optical flow and regularization with transparency. this paper describes a new method for estimating optical flow that strikes a balance between the flexibility of local dense computations and the robustness and accuracy of global parameterized flow models. an affine model of image motion is used within local image patches while a spatial smoothness constraint on the affine flow parameters of neighboring patches enforces continuity of the motion. we refer to this as a ``skin and bones'' model in which the affine patches can be thought of as rigid ``bones'' connected by a flexible ``skin''. since local image patches may contain multiple motions we use a layered representation for the affine bones. to regularize this layered motion representation we develop a new framework for "regularization with transparency".
analysis of gesture and action in technical talks for video indexing. in this paper, we present an automatic system for analyzing and annotating video sequences of technical talks. our method uses a robust motion estimation technique to detect key frames and segment the video sequence into subsequences containing a single overhead slide. the subsequences are stabilized to remove motion that occurs when the speaker adjusts their slides. any changes remaining between frames in the stabilized sequences may be due to speaker gestures such as pointing or writing and we use active contours to automatically track these potential gestures. given the constrained domain we define a simple ``vocabulary'' of actions which can easily be recognized based on the active contour shape and motion. the recognized actions provide a rich annotation of the sequence that can be used to access a condensed version of the talk from a web page.
gesture recognition using the perseus architecture. interpersonal communication involves more than simply spoken information. gestures are commonly used to more efficiently and precisely communicate. an important gesture because of its descriptive power and frequency of use is pointing. to produce a more natural and powerful human-robot interface, we have developed a purposive visual architecture called perseus and have used it to locate objects a person is pointing to. with perseus, in real-time, we are able to determine when a person enters the scene, track the relevant parts of the person including the hands and head, and recognize when she is pointing. once the person points, the object pointed to is located. the perseus architecture allows knowledge about the task and context to be used at all levels of visual analysis for improved performance. this knowledge is explicitly represented in the perseus system to facilitate the extension of perseus to other tasks and environments. in this paper we describe perseus and how it is used to solve this task. previous work on perseus is extended by describing a more sophisticated representation of the person and superior segmenting and tracking methods. experiments showing the success of the perseus system with numerous naive users in varied environments is presented.
model-based estimation of 3d human motion with occlusion based on active multi-viewpoint selection. dept. of comput. & inf. sci., pennsylvania univ., philadelphia, pa, usa abstract: we present a new method for the 3d model-based tracking of human body parts. to mitigate the difficulties arising due to occlusion among body parts, we employ multiple calibrated cameras in a mutually orthogonal configuration. in addition, we develop criteria for a time varying active selection of a set of cameras to track the motion of a particular human part. in particular, at every frame, each camera tracks a number of parts depending on the visibility of these parts and the observability of their predicted motion from the specific camera. to relate points on the occluding contours of the parts to points on their models we apply concepts from projective geometry. then, within the physics-based framework we compute the generalized forces applied from the parts' occluding contours to model points of the body parts. these forces update the translational and rotational degrees of freedom of the model, such as to minimize the discrepancy between the sensory data and the estimated model state. we present initial tracking results from a series of experiments involving the recovery of complex 3d motions in the presence of significant occlusion.
a stereo machine for video-rate dense depth mapping and its new applications. we have developed a video-rate stereo machine that has the capability of generating a dense depth map at the video rate. the performance bench marks of the cmu video-rate stereo machine are: 1) multi image input of up to 6 cameras; 2) throughput of 30 million point-disparity measurements per second; 3) frame rate of 30 frame/sec; 4) a dense depth map of up to 256 x 240 pixels; 5) disparity search range of up to 60 pixels; 6) high precision of depth output up to 8 bi ts (with interpolation). the capability of passively producing such a dense depth map (3d representation) of a scene at the video rate can open up a new class of applications of 3d vision:merging real and virtual worlds in real time.
3-d scene data recovery using omnidirectional multibaseline stereo. a traditional approach to extracting geometric information from a large scene is to compute multiple 3-d depth maps from stereo pairs or direct range finders, and then to merge the 3-d data. however, the resulting merged depth maps may be subject to merging errors if the relative poses between depth maps are not known exactly. in addition, the 3-d data may also have to be resampled before merging, which adds additional complexity and potential sources of errors.this paper provides a means of directly extracting 3-d data covering a very wide field of view, thus by-passing the need for numerous depth map merging. in our work, cylindrical images are first composited from sequences of images taken while the camera is rotated 360&deg; about a vertical axis. by taking such image panoramas at different camera locations, we can recover 3-d data of the scene using a set of simple techniques: feature tracking, an 8-point structure from motion algorithm, and multibaseline stereo. we also investigate the effect of median filtering on the recovered 3-d point distributions, and show the results of our approach applied to both synthetic and real scenes.
characterization of errors in compositing panoramic images. in this paper, we describe the effect of errors in the intrinsic camera parameters on reconstructed panoramic images. a panoramic image is created by first capturing a sequence of images while rotating the camera about a vertical axis a full 360 degrees. the subsequent steps are projecting the original rectilinear images onto cylindrical surfaces and compositing them to form the panoramic image.our analysis has led to a technique that allows simultaneous recovery of the camera focal length and properly composited panoramic images. the correct focal length can be determined by iterating the processes of projecting the original rectilinear images onto cylindrical surfaces given an estimate of the focal length and compositing the resulting images to yield an increasingly better estimate of the focal length. this paper shows that the convergence towards the correct focal length is exponential.