waashk/dblp huggingface dataset:
vision-based multiple interacting targets tracking via on-line supervised learning. successful multi-target tracking requires locating the targets and labeling their identities. this mission becomes significantly more challenging when many targets frequently interact with each other (present partial or complete occlusions). this paper presents an on-line supervised learning based method for tracking multiple interacting targets. when the targets do not interact with each other, multiple independent trackers are employed for training a classifier for each target. when the targets are in close proximity or present occlusions, the learned classifiers are used to assist in tracking. the tracking and learning supplement each other in the proposed method, which not only deals with tough problems encountered in multi-target tracking, but also ensures the entire process to be completely on-line. various evaluations have demonstrated that this method performs better than previous methods when the interactions occur, and can maintain the correct tracking under various complex tracking situations, including crossovers, collisions and occlusions.
3d non-rigid surface matching and registration based on holomorphic differentials. 3d surface matching is fundamental for shape registration, deformable 3d non-rigid tracking, recognition and classification. in this paper we describe a novel approach for generating an efficient and optimal combined matching from multiple boundary-constrained conformal parameterizations for multiply connected domains (i.e., genus zero open surface with multiple boundaries), which always come from imperfect 3d data acquisition (holes, partial occlusions, change of pose and non-rigid deformation between scans). this optimality criterion is also used to assess how consistent each boundary is, and thus decide to enforce or relax boundary constraints across the two surfaces to be matched. the linear boundary-constrained conformal parameterization is based on the holomorphic differential forms, which map a surface with n boundaries conformally to a planar rectangle with (n - 2) horizontal slits, other two boundaries as constraints. the mapping is a diffeomorphism and intrinsic to the geometry, handles an open surface with arbitrary number of boundaries, and can be implemented as a linear system. experimental results are given for real facial surface matching, deformable cloth non-rigid tracking, which demonstrate the efficiency of our method, especially for 3d non-rigid surfaces with significantly inconsistent boundaries.
efficient camera smoothing in sequential structure-from-motion using approximate cross-validation. in the sequential approach to three-dimensional reconstruction, adding prior knowledge about camera pose improves reconstruction accuracy. we add a smoothing penalty on the camera trajectory. the smoothing parameter, usually fixed by trial and error, is automatically estimated using cross-validation. this technique is extremely expensive in its basic form. we derive gauss-newton cross-validation, which closely approximates cross-validation, while being much cheaper to compute. the method is substantiated by experimental results on synthetic and real data. they show that it improves accuracy and stability in the reconstruction process, preventing several failure cases.
the bi-directional framework for unifying parametric image alignment approaches. in this paper, a generic bi-directional framework is proposed for parametric image alignment, that extends the classification of [1]. four main categories (forward, inverse, dependent and bi-directional) form the basis of a consistent set of subclasses, onto which state-of-the-art methods have been mapped. new formulations for the esm [2] and the inverse additive [3] algorithms are proposed, that show the ability of this framework to unify existing approaches. new explicit equivalence relationships are given for the case of first-order optimization that provide some insights into the choice of an update rule in iterative algorithms.
multi-layered decomposition of recurrent scenes. there is considerable interest in techniques capable of identifying anomalies and unusual events in busy outdoor scenes, e.g. road junctions. many approaches achieve this by exploiting deviations in spatial appearance from some expected norm accumulated by a model over time. in this work we show that much can be gained from explicitly modelling temporal aspects in detail. specifically, many traffic junctions are regulated by lights controlled by a timing device of considerable precision, and it is in these situations that we advocate a model which learns periodic spatio-temporal patterns with a view to highlighting anomalous events such as broken-down vehicles, traffic accidents, or pedestrians jay-walking. more specifically, by estimating autocovariance of self-similarity, used previously in the context gait recognition, we characterize a scene by identifying a global fundamental period. as our model, we introduce a spatio-temporal grid of histograms built in accordance with some chosen feature. this model is then used to classify objects found in subsequent test data. in particular we demonstrate the effect of such characterization experimentally by monitoring the bounding box aspect ratio and optical flow field of objects detected on a road traffic junction, enabling our model to discriminate between people and cars sufficiently well to provide useful warnings of adverse behaviour in real time.
discriminative sparse image models for class-specific edge detection and image interpretation. sparse signal models learned from data are widely used in audio, image, and video restoration. they have recently been generalized to discriminative image understanding tasks such as texture segmentation and feature selection. this paper extends this line of research by proposing a multiscale method to minimize least-squares reconstruction errors and discriminative cost functions under ¿0 or ¿1 regularization constraints. it is applied to edge detection, category-based edge selection and image classification tasks. experiments on the berkeley edge detection benchmark and the pascal voc'05 and voc'07 datasets demonstrate the computational efficiency of our algorithm and its ability to learn local image descriptions that effectively support demanding computer vision tasks.
estimating 3d trajectories of periodic motions from stationary monocular views. we present a method for estimating the 3d trajectory of an object undergoing periodic motion in world coordinates by observing its apparent trajectory in a video taken from a single stationary camera. periodicity in 3d is used here as a physical constraint, from which accurate solutions can be obtained. a detailed analysis is performed, from which we gain significant insight regarding the nature of the problem and the information that is required to arrive at a unique solution. subsequently, a robust, numerical approach is proposed, and it is demonstrated that the cost function exhibits strong local convexity which is amenable to local optimization methods. experimental results indicate the effectiveness of the proposed method for reconstructing periodic trajectories in 3d.
multiple tree models for occlusion and spatial constraints in human pose estimation. tree-structured models have been widely used for human pose estimation, in either 2d or 3d. while such models allow efficient learning and inference, they fail to capture additional dependencies between body parts, other than kinematic constraints between connected parts. in this paper, we consider the use of multiple tree models, rather than a single tree model for human pose estimation. our model can alleviate the limitations of a single tree-structured model by combining information provided across different tree models. the parameters of each individual tree model are trained via standard learning algorithms in a single tree-structured model. different tree models can be combined in a discriminative fashion by a boosting procedure. we present experimental results showing the improvement of our approaches on two different datasets. on the first dataset, we use our multiple tree framework for occlusion reasoning. on the second dataset, we combine multiple deformable trees for capturing spatial constraints between non-connected body parts.
csdd features: center-surround distribution distance for feature extraction and matching. we present an interest region operator and feature descriptor called center-surround distribution distance (csdd) that is based on comparing feature distributions between a central foreground region and a surrounding ring of background pixels. in addition to finding the usual light(dark) blobs surrounded by a dark(light) background, csdd also detects blobs with arbitrary color distribution that "stand out" perceptually because they look different from the background. a proof-of-concept implementation using an isotropic scale-space extracts feature descriptors that are invariant to image rotation and covariant with change of scale. detection repeatability is evaluated and compared with other state-of-the-art approaches using a standard dataset, while use of csdd features for image registration is demonstrated within a ransac procedure for affine image matching.
beyond loose lp-relaxations: optimizing mrfs by repairing cycles. this paper presents a new mrf optimization algorithm, which is derived from linear programming and manages to go beyond current state-of-the-art techniques (such as those based on graph-cuts or belief propagation). it does so by relying on a much tighter class of lp-relaxations, called cycle-relaxations. with the help of this class of relaxations, our algorithm tries to deal with a difficulty lying at the heart of mrf optimization: the existence of inconsistent cycles. to this end, it uses an operation called cycle-repairing. the goal of that operation is to fix any inconsistent cycles that may appear during optimization, instead of simply ignoring them as usually done up to now. the more the repaired cycles, the tighter the underlying lp relaxation becomes. as a result of this procedure, our algorithm is capable of providing almost optimal solutions even for very general mrfs with arbitrary potentials. experimental results verify its effectiveness on difficult mrf problems, as well as its better performance compared to the state of the art.
a statistical confidence measure for optical flows. confidence measures are crucial to the interpretation of any optical flow measurement. even though numerous methods for estimating optical flow have been proposed over the last three decades, a sound, universal, and statistically motivated confidence measure for optical flow measurements is still missing. we aim at filling this gap with this contribution, where such a confidence measure is derived, using statistical test theory and measurable statistics of flow fields from the regarded domain. the new confidence measure is computed from merely the results of the optical flow estimator and hence can be applied to any optical flow estimation method, covering the range from local parametric to global variational approaches. experimental results using state-of-the-art optical flow estimators and various test sequences demonstrate the superiority of the proposed technique compared to existing 'confidence' measures.
star shape prior for graph-cut image segmentation. in recent years, segmentation with graph cuts is increasingly used for a variety of applications, such as photo/video editing, medical image processing, etc. one of the most common applications of graph cut segmentation is extracting an object of interest from its background. if there is any knowledge about the object shape (i.e. a shape prior), incorporating this knowledge helps to achieve a more robust segmentation. in this paper, we show how to implement a star shape prior into graph cut segmentation. this is a generic shape prior, i.e. it is not specific to any particular object, but rather applies to a wide class of objects, in particular to convex objects. our major assumption is that the center of the star shape is known, for example, it can be provided by the user. the star shape prior has an additional important benefit - it allows an inclusion of a term in the objective function which encourages a longer object boundary. this helps to alleviate the bias of a graph cut towards shorter segmentation boundaries. in fact, we show that in many cases, with this new term we can achieve an accurate object segmentation with only a single pixel, the center of the object, provided by the user, which is rarely possible with standard graph cut interactive segmentation.
optimizing binary mrfs with higher order cliques. widespread use of efficient and successful solutions of computer vision problems based on pairwise markov random field (mrf) models raises a question: does any link exist between the pairwise and higher order mrfs such that the like solutions can be applied to the latter models? this work explores such a link for binary mrfs that allow us to represent gibbs energy of signal interaction with a polynomial function. we show how a higher order polynomial can be efficiently transformed into a quadratic function. then energy minimization tools for the pairwise mrf models can be easily applied to the higher order counterparts. also, we propose a method to analytically estimate the potential parameter of the asymmetric potts prior. the proposed framework demonstrates very promising experimental results of image segmentation and can be used to solve other computer vision problems.
automatic image colorization via multimodal predictions. we aim to color greyscale images automatically, without any manual intervention. the color proposition could then be interactively corrected by user-provided color landmarks if necessary. automatic colorization is nontrivial since there is usually no one-to-one correspondence between color and local texture. the contribution of our framework is that we deal directly with multimodality and estimate, for each pixel of the image to be colored, the probability distribution of all possible colors, instead of choosing the most probable color at the local level. we also predict the expected variation of color at each pixel, thus defining a non-uniform spatial coherency criterion. we then use graph cuts to maximize the probability of the whole colored image at the global level. we work in the l-a-b color space in order to approximate the human perception of distances between colors, and we use machine learning tools to extract as much information as possible from a dataset of colored examples. the resulting algorithm is fast, designed to be more robust to texture noise, and is above all able to deal with ambiguity, in contrary to previous approaches.
unsupervised learning of skeletons from motion. humans demonstrate a remarkable ability to parse complicated motion sequences into their constituent structures and motions. we investigate this problem, attempting to learn the structure of one or more articulated objects, given a time-series of two-dimensional feature positions. we model the observed sequence in terms of "stick figure" objects, under the assumption that the relative joint angles between sticks can change over time, but their lengths and connectivities are fixed. we formulate the problem in a single probabilistic model that includes multiple sub-components: associating the features with particular sticks, determining the proper number of sticks, and finding which sticks are physically joined. we test the algorithm on challenging datasets of 2d projections of optical human motion capture and feature trajectories from real videos.
an extended phase field higher-order active contour model for networks and its application to road network extraction from vhr satellite images. this paper addresses the segmentation from an image of entities that have the form of a `network', i.e. the region in the image corresponding to the entity is composed of branches joining together at junctions, e.g. road or vascular networks. we present a new phase field higher-order active contour (hoac) prior model for network regions, and apply it to the segmentation of road networks from very high resolution satellite images. this is a hard problem for two reasons. first, the images are complex, with much `noise' in the road region due to cars, road markings, etc., while the background is very varied, containing many features that are locally similar to roads. second, network regions are complex to model, because they may have arbitrary topology. in particular, we address a severe limitation of a previous model in which network branch width was constrained to be similar to maximum network branch radius of curvature, thereby providing a poor model of networks with straight narrow branches or highly curved, wide branches. to solve this problem, we propose a new hoac prior energy term, and reformulate it as a nonlocal phase field energy. we analyse the stability of the new model, and find that in addition to solving the above problem by separating the interactions between points on the same and opposite sides of a network branch, the new model permits the modelling of two widths simultaneously. the analysis also fixes some of the model parameters in terms of network width(s). after adding a likelihood energy, we use the model to extract the road network quasi-automatically from pieces of a quickbird image, and compare the results to other models in the literature. the results demonstrate the superiority of the new model, the importance of strong prior knowledge in general, and of the new term in particular.
training hierarchical feed-forward visual recognition models using transfer learning from pseudo-tasks. building visual recognition models that adapt across different domains is a challenging task for computer vision. while feature-learning machines in the form of hierarchial feed-forward models (e.g., convolutional neural networks) showed promise in this direction, they are still difficult to train especially when few training examples are available. in this paper, we present a framework for training hierarchical feed-forward models for visual recognition, using transfer learning from pseudo tasks. these pseudo tasks are automatically constructed from data without supervision and comprise a set of simple pattern-matching operations. we show that these pseudo tasks induce an informative inverse-wishart prior on the functional behavior of the network, offering an effective way to incorporate useful prior knowledge into the network training. in addition to being extremely simple to implement, and adaptable across different domains with little or no extra tuning, our approach achieves promising results on challenging visual recognition tasks, including object recognition, gender recognition, and ethnicity recognition.
an effective approach to 3d deformable surface tracking. the key challenge with 3d deformable surface tracking arises from the difficulty in estimating a large number of 3d shape parameters from noisy observations. a recent state-of-the-art approach attacks this problem by formulating it as a second order cone programming (socp) feasibility problem. the main drawback of this solution is the high computational cost. in this paper, we first reformulate the problem into an unconstrained quadratic optimization problem. instead of handling a large set of complicated socp constraints, our new formulation can be solved very efficiently by resolving a set of sparse linear equations. based on the new framework, a robust iterative method is employed to handle large outliers. we have conducted an extensive set of experiments to evaluate the performance on both synthetic and real-world testbeds, from which the promising results show that the proposed algorithm not only achieves better tracking accuracy, but also executes significantly faster than the previous solution.
detecting carried objects in short video sequences. we propose a new method for detecting objects such as bags carried by pedestrians depicted in short video sequences. in common with earlier work [1,2] on the same problem, the method starts by averaging aligned foreground regions of a walking pedestrian to produce a representation of motion and shape (known as a temporal template) that has some immunity to noise in foreground segmentations and phase of the walking cycle. our key novelty is for carried objects to be revealed by comparing the temporal templates against view-specific exemplars generated offline for unencumbered pedestrians. a likelihood map obtained from this match is combined in a markov random field with a map of prior probabilities for carried objects and a spatial continuity assumption, from which we obtain a segmentation of carried objects using the map solution. we have re-implemented the earlier state of the art method [1] and demonstrate a substantial improvement in performance for the new method on the challenging pets2006 dataset [3]. although developed for a specific problem, the method could be applied to the detection of irregularities in appearance for other categories of object that move in a periodic fashion.
semi-automatic motion segmentation with motion layer mosaics. a new method for motion segmentation based on reference motion layer mosaics is presented. we assume that the scene is composed of a set of layers whose motion is well described by parametric models. this usual assumption is compatible with the notion of motion layer mosaic, which allows a compact representation of the sequence with a small number of mosaics only. we segment the sequence using a reduced number of distant image-to-mosaic comparisons instead of a larger number of close image-to-image comparisons. apart from computational advantage, another interest lies in the fact that motions estimated between distant images are more likely to be different from one region to another than when estimated between consecutive images. this helps the segmentation process. the segmentation is obtained by graph cut minimization of a cost function which includes an original image-to-mosaic data term. at the end of the segmentation process, it may happen that the obtained boundaries are not precisely the expected ones. often the user has no other possibility than modifying manually every segmentation one after another or than starting over all again the process with different parameters. we propose an original easy way for the user to manually correct the possible errors on the mosaics themselves. these corrections are then propagated to all the images of the corresponding video interval thanks to a second segmentation pass. experimental results demonstrate the potential of our approach.
object recognition by integrating multiple image segmentations. the joint tasks of object recognition and object segmentation from a single image are complex in their requirement of not only correct classification, but also deciding exactly which pixels belong to the object. exploring all possible pixel subsets is prohibitively expensive, leading to recent approaches which use unsupervised image segmentation to reduce the size of the configuration space. image segmentation, however, is known to be unstable, strongly affected by small image perturbations, feature choices, or different segmentation algorithms. this instability has led to advocacy for using multiple segmentations of an image. in this paper, we explore the question of how to best integrate the information from multiple bottom-up segmentations of an image to improve object recognition robustness. by integrating the image partition hypotheses in an intuitive combined top-down and bottom-up recognition approach, we improve object and feature support. we further explore possible extensions of our method and whether they provide improved performance. results are presented on the msrc 21-class data set and the pascal voc2007 object segmentation challenge.