waashk/dblp huggingface dataset:
using temporal coherence to build models of animals. this paper describes a system that can build appearance models of animals automatically from a video sequence of the relevant animal with no explicit supervisory information. the video sequence need not have any form of special background. animals are modeled as a 2d kinematic chain of rectangular segments, where the number of segments and the topology of the chain are unknown. the system detects possible segments, clusters segments whose appearance is coherent over time, and then builds a spatial model of such segment clusters. the resulting representation of the spatial configuration of the animal in each frame can be seen either as a track - in which case the system described should be viewed as a generalized tracker, that is capable of modeling objects while tracking them - or as the source of an appearance model which can be used to build detectors for the particular animal. this is because knowing a video sequence is temporally coherent - i.e. that a particular animal is present through the sequence - is a strong supervisory signal. the method is shown to be successful as a tracker on video sequences of real scenes showing three different animals. for the same reason it is successful as a tracker, the method results in detectors that can be used to find each animal fairly reliably within the corel collection of images.
new algorithms for two-frame structure from motion. we describe two new algorithms for two frame structure from motion from tracked point features. one is the first fast algorithm for computing an exact least squares estimate. it exploits our observation that the rotationally invariant least squares error can be written in a simple form that depends just on the motion. the other is essentially as accurate as the least squares estimate and is more efficient, probably faster, and potentially more robust than previous algorithms of comparable accuracy. we also analyze theoretically the accuracy of the optical flow approximation to the least squares error.
a bayesian network framework for relational shape matching. a bayesian network formulation for relational shapematching is presented. the main advantage of the relationalshape matching approach is the obviation ofthe non-rigid spatial mappings used by recent non-rigidmatching approaches. the basic variables that need tobe estimated in the relational shape matching objectivefunction are the global rotation and scale and the localdisplacements and correspondences. the new bethefree energy approach is used to estimate the pairwisecorrespondences between links of the template graphsand the data. the resulting framework is useful inboth registration and recognition contexts. results areshown on hand-drawn templates and on 2d transverset1-weighted mr images.
a theory of specular surface geometry. a theoretical framework is introduced for the perception of specular surface geometry. when an observer moves in three-dimensional space, real scene features, such as surface markings, remain stationary with respect to the surfaces they belong to. in contrast, a virtual feature, which is the specular reflection of a real feature, travels on the surface. based on the notion of caustics, a novel feature classification algorithm is developed that distinguishes real and virtual features from their image trajectories that result from observer motion. next, using support functions of curves, a closed-form relation is derived between the image trajectory of a virtual feature and the geometry of the specular surface it travels on. it is shown that in the 2d case where camera motion and the surface profile are coplanar, the profile is uniquely recovered by tracking just two unknown virtual features. finally, these results are generalized to the case of arbitrary 3d surface profiles that are travelled by virtual features when camera motion is not confined to a plane. an algorithm is developed that uniquely recovers 3d surface profiles using a single virtual feature tracked from the occluding boundary of the object. all theoretical derivations and proposed algorithms are substantiated by experiments.
object indexing using an iconic sparse distributed memory. a general-purpose object indexing technique is described that combines the virtues of principal component analysis with the favorable matching properties of high-dimensional spaces to achieve high-precision recognition. an object is represented by a set of high-dimensional iconic feature vectors comprised of the responses of derivatives of gaussian filters at a range of orientations and scales. since these filters can be shown to form the eigenvectors of arbitrary images containing both natural and man-made structures, they are well-suited for indexing in disparate domains. the indexing algorithm uses an active vision system in conjunction with a modified form of kanerva's (1988, 1993) sparse distributed memory which facilitates interpolation between views and provides a convenient platform for learning the association between an object's appearance and its identity. the robustness of the indexing method was experimentally confirmed by subjecting the method to a range of viewing conditions and the accuracy was verified using a well-known model database containing a number of complex 3d objects under varying pose.
view-invariant alignment and matching of video sequences. in this paper, we propose a novel method to establish temporalcorrespondence between the frames of two videos. 3d epipolargeometry is used to eliminate the distortion generated bythe projection from 3d to 2d. although the fundamental matrixcontains the extrinsic property of the projective geometrybetween views, it is sensitive to noise. therefore, wepropose the use of a rank constraint of corresponding pointsin two views to measure the similarity between trajectories.this rank constraint shows more robustness and avoids computationof the fundamental matrix. a dynamic programmingapproach using the similarity measurement is proposed to findthe non-linear time-warping function for videos containinghuman activities. in this way, videos of different individualstaken at different times and from distinct viewpoints canbe synchronized. a temporal pyramid of trajectories is appliedto improve the accuracy of the view-invariant dynamictime-warping approach. we show various applications of thisapproach such as video synthesis, human action recognition,and computer aider training. compared to state-of-the-arttechniques, our method shows a great improvement.
on the equivalence of common approaches to lighting insensitive recognition. lighting variation is commonly handled by methods invariant to additive and multiplicative changes in image intensity. it has been demonstrated that comparing images using the direction of the gradient can produce broader insensitivity to changes in lighting conditions, even for 3d scenes. we analyze two common approaches to image comparison that are invariant, normalized correlation using small correlation windows, and comparison based on a large set of oriented difference of gaussian filters. we show analytically that these methods calculate a monotonic (cosine) function of the gradient direction difference and hence are equivalent to the direction of gradient method. our analysis is supported with experiments on both synthetic and real scenes.
prior-based segmentation by projective registration and level sets. object detection and segmentation can be facilitated by the availability of a reference object. however, accounting for possible transformations between the different object views, as part of the segmentation process, remains a challenge. recent works address this problem by using comprehensive training data. other approaches are applicable only to limited object classes or can only accommodate similarity transformations. we suggest a novel variational approach to prior-based segmentation, which accounts for planar projective transformation, using a single reference object. the prior shape is registered concurrently with the segmentation process, without point correspondence. the algorithm detects the object of interest and correctly extracts its boundaries. the homography between the two object views is accurately recovered as well. extending the chan-vese level set framework, we propose a region-based segmentation functional that includes explicit representation of the projective homography between the prior shape and the shape to segment. the formulation is derived from two-view geometry. segmentation of a variety of objects is demonstrated and the recovered transformation is verified.
segmentation of hybrid motions via hybrid quadratic surface analysis. in this paper, we investigate the mathematical problem underlying segmentation of hybrid motions: given a series of tracked feature correspondences between two (perspective) images, we seek to segment and estimate multiple motions, possibly of different types (e.g., affine, epipolar, and homography). in order to accomplish this task, we cast the problem into a more general mathematical framework of segmenting data samples drawn from a mixture of linear subspaces and quadratic surfaces. the result is a novel algorithm called hybrid quadratic surface analysis (hqsa). hqsa uses both the derivatives and hessians of fitting poly-nomials for the data to separate linear data samples from quadratic data samples. these derivatives and hessians also lead to important necessary conditions, based on the so-called mutual contraction subspace, to separate data samples on different quadratic surfaces. the algebraic solution we derive is non-iterative and numerically stable. it tolerates moderate noise and can be used in conjunction with outlier removal techniques. we show how to solve the hybrid motion segmentation problem using hqsa, and demonstrate its performance on simulated data with noise and on real perspective images.
retrieving images by appearance. a system to retrieve images using a description of the image intensity surface is presented. gaussian derivative filters at several scales are applied to the image and low order 2d differential invariants are computed. the resulting multi-scale representation is indexed for rapid retrieval. queries are designed by the users from an example image by selecting appropriate regions. the invariant vectors corresponding to these regions are matched with the database counter-parts both in feature and coordinate space. this yields a match score per image. images are sorted by the match score and displayed. experiments conducted with over 1500 images of objects embedded in arbitrary backgrounds are described. it is observed that images similar in appearance and whose viewpoint is within small view variations of the query can be retrieved with an average precision1 of 56%.
an ensemble prior of image structure for cross-modal inference. in cross-modal inference, we estimate complete fields from noisy and missing observations of one sensory modality using structure found in another sensory modality. this inference problem occurs in several areas including texture reconstruction and reconstruction of geophysical fields. we propose a method for cross-modal inference that simultaneously learns shape recipes between two modalities and estimates missing information by using a prior on image structure gleaned from the alternate modality. in the absence of a physical basis for representing image priors, we use a statistical one that represents correlations in differential features. this is done efficiently using a perturbation sampling scheme. using just one example of the alternate modality, we produce a factorized ensemble representation of feature correlations that yields efficient solutions to large-sized spatial inference problems. we demonstrate the utility of this approach on cross-modal inference with depth and spectral data.
model-based tracking of self-occluding articulated objects. computer sensing of hand and limb motion is an important problem for applications in human computer interaction and computer graphics. we describe a framework for local trading of self occluding motion, in which one part of an object obstructs the visibility of another. our approach uses a kinematic model to predict occlusions and windowed templates to track partially occluded objects. we present offline 3d tracking results for hand motion with significant self occlusion.
agent orientated annotation in model based visual surveillance. the paper presents an agent based surveillance system for use in monitoring scenes involving both pedestrians and vehicles. the surveillance system supplies textual descriptions for the dynamic activity occurring in the 3d world. these are derived by means of dynamic andprobabilistic inference based on geometric information provided by a vision system that tracks vehicles and pedestrians. the symbolic scene annotation is given at two major levels of description: the object level and the interobject level. at object level, each tracked object(pedestrian or vehicle) is assigned a behaviour agent which uses a bayesian network to infer the fundamental features of the objects' trajectory, and continuously updates its textual description. the inter-object interaction level is interpreted by a situation agent which is created dynamically when two objects are in close proximity.
recovering human body configurations using pairwise constraints between parts. the goal of this work is to recover human bodyconfigurations from static images. without assuming a priori knowledge of scale, pose or appearance, this problem is extremely challenging and demands the use of all possible sources of information. we develop a framework which can incorporate arbitrary pairwise constraints between body parts, such as scale compatibility, relative position, symmetry of clothing and smooth contour connections between parts. we detect candidate body parts from bottom-up using parallelism, and use various pairwise configuration constraints to assemble them together into body configurations. to find the most probable configuration, we solve an integer quadratic programming problem with a standard technique using linear approximations. approximate iqp allows us to incorporate much more information than the traditional dynamic programming and remains computationally efficient. 15 hand-labeled images are used to train the low-level part detector and learn the pairwise constraints. we show test results on a variety of images.
scale-invariant contour completion using conditional random fields. we present a model of curvilinear grouping using piece-wise linear representations of contours and a conditional random field to capture continuity and the frequency of different junction types. potential completions are generated by building a constrained delaunay triangulation (cdt) over the set of contours found by a local edge detector. maximum likelihood parameters for the model are learned from human labeled groundtruth. using held out test data, we measure how the model, by incorporating continuity structure, improves boundary detection over the local edge detector. we also compare performance with a baseline local classifier that operates on pairs of edgels. both algorithms consistently dominate the low-level boundary detector at all thresholds. to our knowledge, this is the first time that curvilinear continuity has been shown quantitatively useful for a large variety of natural images. better boundary detection has immediate application in the problem of object detection and recognition.
learning a classification model for segmentation. we propose a two-class classification model for grouping. human segmented natural images are used as positive examples. negative examples of grouping are constructed by randomly matching human segmentations and images. in a preprocessing stage an image is over segmented into superpixels. we define a variety of features derived from the classical gestalt cues, including contour, texture, brightness and good continuation. information-theoretic analysis is applied to evaluate the power of these grouping cues. we train a linear classifier to combine these features. to demonstrate the power of the classification model, a simple algorithm is used to randomly search for good segmentations. results are shown on a wide range of images.
a cluster-based statistical model for object detection. this paper presents an approach to object detection which is based on recent work in statistical models for texture synthesis and recognition (heeger and bergen 1995, de bonet and viola 1998, zhu et al. 1998, simoncelli and portilla 1998). our method follows the texture recognition work of de bonet and viola. we use feature vectors which capture the joint occurrence of local features at multiple resolutions. the distribution of feature vectors for a set of training images of an object class is estimated by clustering the data and then forming a mixture of gaussian model. the mixture model is further refined by determining which clusters are the most discriminative for the class and retaining only those clusters. after the model is learned, test images are classified by computing the likelihood of their feature vectors with respect to the model. we present promising results in applying our technique to face detection and car detection.
weakly-calibrated stereo perception for rover navigation. presents a vision system for autonomous navigation based on stereo perception without 3d reconstruction. this approach uses weakly calibrated stereo images, i.e. images for which only the epipolar geometry is known. the vision system first rectifies the images, matches selected points between the two images, and then computes the relative elevation of the points relative to a reference plane as well as the images of their projections on this plane. we have integrated this vision module into a complete navigation system. in this system, the relative elevation is used as a shape indicator in order to compute appropriate steering directions everytime a new stereo pair is processed. we have conducted initial experiments in unstructured, outdoor environments with an wheeled rover.
vehicle identification between non-overlapping cameras without direct feature matching. we propose a novel method for identifying road vehicles between two non-overlapping cameras. the problem is formulated as a same-different classification problem: probability of two vehicle images from two distinct cameras being from the same vehicle or from different vehicles. the key idea is to compute the probability without matching the two vehicle images directly, which is a process vulnerable to drastic appearance and aspect changes. we represent each vehicle image as an embedding amongst representative exemplars of vehicles within the same camera. the embedding is computed as a vector each of whose components is a non-metric distance for a vehicle to an exemplar. the non-metric distances are computed using robust matching of oriented edge images. a set of truthed training examples of same-different vehicle pairings across the two cameras is used to learn a classifier that encodes the probability distributions. a pair of the embeddings representing two vehicles across two cameras are then used to compute the same-different probability. in order for the vehicle exemplars to be representative for both cameras, we also propose a method for jointly selection of corresponding exemplars using the training data. experiments on observations of over 400 vehicles under drastically illumination and camera conditions demonstrate promising results.
behaviour understanding in video: a combined method. in this paper we develop a system for human behaviour recognition in video sequences. human behaviour is modelled as a stochastic sequence of actions. actions are described by a feature vector comprising both trajectory information (position and velocity), and a set of local motion descriptors. action recognition is achieved via probabilistic search of image feature databases representing previously seen actions. a hmm which encodes the rules of the scene is used to smooth sequences of actions. high-level behaviour recognition is achieved by computing the likelihood that a set of predefined hidden markov models explains the current action sequence. thus, human actions and behaviour are represented using a hierarchy of abstraction: from simple actions, to actions with spatio-temporal context, to action sequences and finally general behaviours. while the upper levels all use (parametric) bayes networks and belief propagation, the lowest level uses non-parametric sampling from a previously learned database of actions. the combined method represents a general framework for human behaviour modelling. in this paper we demonstrate the results chiefly on broadcast tennis sequences for automated video annotation.
face recognition with mrc-boosting. in this paper, a novel classification algorithm called mrc-boosting is proposed. through aggregating maximal-rejection-classifier features under boosting framework, this algorithm can deal with complicated two-class classification problem, especially for the category called target detection problem where a target class should be discriminated from the surrounding clutter class. mrc-boosting is efficient since unlike many other boosting based algorithms, at each iteration the optimal feature is computed in closed-form, with neither exhaustive search nor time-consuming numerical optimization. furthermore, a variant of mrc-boosting is derived and applied to face recognition. this variant mrc-boosting algorithm is able to utilize large amount of training samples efficiently, overcoming the difficulty faced by other algorithms like adaboost. the effectiveness of the proposed algorithm is validated by face recognition experiments on cmu-pie database.
a non-iterative greedy algorithm for multi-frame point correspondence. this paper presents a framework for finding point correspondences in monocular image sequences over multiple frames.the general problem of multi-frame point correspondence is np hard for three or more frames. a polynomial time algorithm for a restriction of this problem is presented, and is used as the basis of proposed greedy algorithm for the general problem. the greedy nature of the proposed algorithm allows it to be used in real time systems for tracking and surveillance etc. in addition, the proposed algorithm deals with the problems of occlusion, missed detections, and false positives, by using a single non-iterative greedy optimization scheme, and hence, reduces the complexity of the overall algorithm as compared to most existing approaches, where multiple heuristics are used for the same purpose.while most greedy algorithms for point tracking do not allow for entry and exit of points from the scene, this is not a limitation for the proposed algorithm. experiments with real and synthetic data show that the proposed algorithm outperforms the existing techniques and is applicable in more general settings.