waashk/dblp huggingface dataset:
active image labeling and its application to facial action labeling. for many tasks in computer vision, it is very important to produce the groundtruth data. at present, this is mostly done manually. manual data labeling is labor-intensive and prone to the human errors. the training data it produces often lacks in both quantity and quality. fully automatic data labeling, on the other hand, is not feasible and reliable. in this paper, we propose an interactive image labeling technique for efficient and accurate data labeling.the proposed technique includes two parts: an automatic labeling part and a human intervention part. constructed on a bayesian network, the automatic image labeler produces an initial labeling of the image. a person then examines the initial labeling and makes some minor corrections. the selected human corrections and the image measurements are then integrated by the bayesian network framework to produce a refined labeling. to minimize the human involvement, an active user feedback strategy is developed, through which the optimal user feedback is determined, so that the labeling errors in the subsequent re-labeling process can be maximally reduced. the proposed framework combines the advantages of the human input with those of the machine so that the reliable, accurate, and efficient data labeling can be achieved. we demonstrate the validity of the proposed framework for interactive labeling of facial action units. the proposed methodology, however, is not limited to labeling of facial action units. it can be easily extended to other areas such as interactive image segmentation.
a lattice-preserving multigrid method for solving the inhomogeneous poisson equations used in image analysis. the inhomogeneous poisson (laplace) equation with internal dirichlet boundary conditions has recently appeared in several applications ranging from image segmentation [1, 2, 3] to image colorization [4], digital photo matting [5, 6] and image filtering [7, 8]. in addition, the problem we address may also be considered as the generalized eigenvector problem associated with normalized cuts [9], the linearized anisotropic diffusion problem [10, 11, 8] solved with a backward euler method, visual surface reconstruction with discontinuities [12, 13] or optical flow [14]. although these approaches have demonstrated quality results, the computational burden of finding a solution requires an efficient solver. design of an efficient multigrid solver is difficult for these problems due to unpredictable inhomogeneity in the equation coefficients and internal dirichlet boundary conditions with unpredictable location and value. previous approaches to multigrid solvers have typically employed either a data-driven operator (with fast convergence) or the maintenance of a lattice structure at coarse levels (with low memory overhead). in addition to memory efficiency, a lattice structure at coarse levels is also essential to taking advantage of the power of a gpu implementation [15,16,5,3]. in this work, we present a multigrid method that maintains the low memory overhead (and gpu suitability) associated with a regular lattice while benefiting from the fast convergence of a data-driven coarse operator.
multiple component learning for object detection. object detection is one of the key problems in computer vision. in the last decade, discriminative learning approaches have proven effective in detecting rigid objects, achieving very low false positives rates. the field has also seen a resurgence of part-based recognition methods, with impressive results on highly articulated, diverse object categories. in this paper we propose a discriminative learning approach for detection that is inspired by part-based recognition approaches. our method, multiple component learning (mcl), automatically learns individual component classifiers and combines these into an overall classifier. unlike previous methods, which rely on either fairly restricted part models or labeled part data, mcl learns powerful component classifiers in a weakly supervised manner, where object labels are provided but part labels are not. the basis of mcl lies in learning a set classifier; we achieve this by combining boosting with weakly supervised learning, specifically the multiple instance learning framework (mil). mcl is general, and we demonstrate results on a range of data from computer audition and computer vision. in particular, mcl outperforms all existing methods on the challenging inria pedestrian detection dataset, and unlike methods that are not part-based, mcl is quite robust to occlusions.
euclidean group invariant computation of stochastic completion fields using shiftable-twistable functions. we describe a method for computing the likelihood that a completion joining two contour fragments passes through any given position and orientation in the image plane. like computations in primary visual cortex (and unlike all previous models of contour completion), the output of our computation is invariant under rotations and translations of the input pattern. this is achieved by representing the input, output, and intermediate states of the computation in a basis of shiftable-twistable functions.
scale invariant action recognition using compound features mined from dense spatio-temporal corners. the use of sparse invariant features to recognise classes of actions or objects has become common in the literature. however, features are often "engineered" to be both sparse and invariant to transformation and it is assumed that they provide the greatest discriminative information. to tackle activity recognition, we propose learning compound features that are assembled from simple 2d corners in both space and time. each corner is encoded in relation to its neighbours and from an over complete set (in excess of 1 million possible features), compound features are extracted using data mining. the final classifier, consisting of sets of compound features, can then be applied to recognise and localise an activity in real-time while providing superior performance to other state-of-the-art approaches (including those based upon sparse feature detectors). furthermore, the approach requires only weak supervision in the form of class labels for each training sequence. no ground truth position or temporal alignment is required during training.
semi-supervised on-line boosting for robust tracking. recently, on-line adaptation of binary classifiers for tracking have been investigated. on-line learning allows for simple classifiers since only the current view of the object from its surrounding background needs to be discriminiated. however, on-line adaption faces one key problem: each update of the tracker may introduce an error which, finally, can lead to tracking failure (drifting). the contribution of this paper is a novel on-line semi-supervised boosting method which significantly alleviates the drifting problem in tracking applications. this allows to limit the drifting problem while still staying adaptive to appearance changes. the main idea is to formulate the update process in a semi-supervised fashion as combined decision of a given prior and an on-line classifier. this comes without any parameter tuning. in the experiments, we demonstrate real-time tracking of our semiboost tracker on several challenging test sequences where our tracker outperforms other on-line tracking methods.
what is a good nearest neighbors algorithm for finding similar patches in images? many computer vision algorithms require searching a set of images for similar patches, which is a very expensive operation. in this work, we compare and evaluate a number of nearest neighbors algorithms for speeding up this task. since image patches follow very different distributions from the uniform and gaussian distributions that are typically used to evaluate nearest neighbors methods, we determine the method with the best performance via extensive experimentation on real images. furthermore, we take advantage of the inherent structure and properties of images to achieve highly efficient implementations of these algorithms. our results indicate that vantage point trees, which are not well known in the vision community, generally offer the best performance.
localizing objects with smart dictionaries. we present an approach to determine the category and location of objects in images. it performs very fast categorization of each pixel in an image, a brute-force approach made feasible by three key developments: first, our method reduces the size of a large generic dictionary (on the order of ten thousand words) to the low hundreds while increasing classification performance compared to k-means. this is achieved by creating a discriminative dictionary tailored to the task by following the information bottleneck principle. second, we perform feature-based categorization efficiently on a dense grid by extending the concept of integral images to the computation of local histograms. third, we compute sift descriptors densely in linear time. we compare our method to the state of the art and find that it excels in accuracy and simplicity, performing better while assuming less.
grassmann registration manifolds for face recognition. motivated by image perturbation and the geometry of manifolds, we present a novel method combining these two elements. first, we form a tangent space from a set of perturbed images and observe that the tangent space admits a vector space structure. second, we embed the approximated tangent spaces on a grassmann manifold and employ a chordal distance as the means for comparing subspaces. the matching process is accelerated using a coarse to fine strategy. experiments on the feret database suggest that the proposed method yields excellent results using both holistic and local features. specifically, on the feret dup2 data set, our proposed method achieves 83.8% rank 1 recognition: to our knowledge the currently the best result among all non-trained methods. evidence is also presented that peak recognition performance is achieved using roughly 100 distinct perturbed images.
active matching. in the matching tasks which form an integral part of all types of tracking and geometrical vision, there are invariably priors available on the absolute and/or relative image locations of features of interest. usually, these priors are used post-hoc in the process of resolving feature matches and obtaining final scene estimates, via `first get candidate matches, then resolve' consensus algorithms such as ransac. in this paper we show that the dramatically different approach of using priors dynamically to guide a feature by feature matching search can achieve global matching with much fewer image processing operations and lower overall computational cost. essentially, we put image processing into the loop of the search for global consensus. in particular, our approach is able to cope with significant image ambiguity thanks to a dynamic mixture of gaussians treatment. in our fully bayesian algorithm, the choice of the most efficient search action at each step is guided intuitively and rigorously by expected shannon information gain. we demonstrate the algorithm in feature matching as part of a sequential slam system for 3d camera tracking. robust, real-time matching can be achieved even in the previously unmanageable case of jerky, rapid motion necessitating weak motion modelling and large search regions.
range flow for varying illumination. in this paper range flow estimation is extended to handle brightness changes in image data caused by inhomogeneous illumination. standard range flow computes 3d velocity fields from range and intensity image sequences. to this end it combines a depth change model and a brightness constancy model. in this contribution, the brightness constancy model is exchanged by (1) a gradient constancy model, (2) a combination of gradient and brightness constancy constraint that has been used successfully for optical flow estimation in literature, and (3) a physics-based brightness change model. insensitivity to brightness changes can also be achieved by prefiltering of the input intensity data. high pass or homomorphic filtering are the most well known approaches from literature. in performance tests therefore the well known version and the novel versions of range flow estimation are investigated on prefiltered or non-prefiltered data using synthetic ground-truth and real data from a botanical experiment.
beyond nouns: exploiting prepositions and comparative adjectives for learning visual classifiers. learning visual classifiers for object recognition from weakly labeled data requires determining correspondence between image regions and semantic object classes. most approaches use co-occurrence of "nouns" and image features over large datasets to determine the correspondence, but many correspondence ambiguities remain. we further constrain the correspondence problem by exploiting additional language constructs to improve the learning process from weakly labeled data. we consider both "prepositions" and "comparative adjectives" which are used to express relationships between objects. if the models of such relationships can be determined, they help resolve correspondence ambiguities. however, learning models of these relationships requires solving the correspondence problem. we simultaneously learn the visual features defining "nouns" and the differential visual features defining such "binary-relationships" using an em-based approach.
segmentation and recognition using structure from motion point clouds. we propose an algorithm for semantic segmentation based on 3d point clouds derived from ego-motion. we motivate five simple cues designed to model specific patterns of motion and 3d world structure that vary with object category. we introduce features that project the 3d cues back to the 2d image plane while modeling spatial layout and context. a randomized decision forest combines many such features to achieve a coherent 2d segmentation and recognize the object categories present. our main contribution is to show how semantic segmentation is possible based solely on motion-derived 3d world structure. our method works well on sparse, noisy point clouds, and unlike existing approaches, does not need appearance-based descriptors.experiments were performed on a challenging new video database containing sequences filmed from a moving car in daylight and at dusk. the results confirm that indeed, accurate segmentation and recognition are possible using only motion and 3d world structure. further, we show that the motion-derived information complements an existing state-of-the-art appearance-based method, improving both qualitative and quantitative performance.
illumination and person-insensitive head pose estimation using distance metric learning. head pose estimation is an important task for many face analysis applications, such as face recognition systems and human computer interactions. in this paper we aim to address the pose estimation problem under some challenging conditions, e.g., from a single image, large pose variation, and un-even illumination conditions. the approach we developed combines non-linear dimension reduction techniques with a learned distance metric transformation. the learned distance metric provides better intra-class clustering, therefore preserving a smooth low-dimensional manifold in the presence of large variation in the input images due to illumination changes. experiments show that our method improves the performance, achieving accuracy within 2-3 degrees for face images with varying poses and within 3-4 degrees error for face images with varying pose and illumination changes.
joint parametric and non-parametric curve evolution for medical image segmentation. this paper proposes a new joint parametric and nonparametric curve evolution algorithm of the level set functions for medical image segmentation. traditional level set algorithms employ non-parametric curve evolution for object matching. although matching image boundaries accurately, they often suffer from local minima and generate incorrect segmentation of object shapes, especially for images with noise, occlusion and low contrast. on the other hand, statistical model-based segmentation methods allow parametric object shape variations subject to some shape prior constraints, and they are more robust in dealing with noise and low contrast. in this paper, we combine the advantages of both of these methods and jointly use parametric and non-parametric curve evolution in object matching. our new joint curve evolution algorithm is as robust as and at the same time, yields more accurate segmentation results than the parametric methods using shape prior information. comparative results on segmenting ventricle frontal horn and putamen shapes in mr brain images confirm both robustness and accuracy of the proposed joint curve evolution algorithm.
efficiently learning random fields for stereo vision with sparse message passing. as richer models for stereo vision are constructed, there is a growing interest in learning model parameters. to estimate parameters in markov random field (mrf) based stereo formulations, one usually needs to perform approximate probabilistic inference. message passing algorithms based on variational methods and belief propagation are widely used for approximate inference in mrfs. conditional random fields (crfs) are discriminative versions of traditional mrfs and have recently been applied to the problem of stereo vision. however, crf parameter training typically requires expensive inference steps for each iteration of optimization. inference is particularly slow when there are many discrete disparity levels, due to high state space cardinality. we present a novel crf for stereo matching with an explicit occlusion model and propose sparse message passing to dramatically accelerate the approximate inference needed for parameter optimization. we show that sparse variational message passing iteratively minimizes the kl divergence between the approximation and model distributions by optimizing a lower bound on the partition function. our experimental results show reductions in inference time of one order of magnitude with no loss in approximation quality. learning using sparse variational message passing improves results over prior work using graph cuts.
building a compact relevant sample coverage for relevance feedback in content-based image retrieval. conventional approaches to relevance feedback in content-based image retrieval are based on the assumption that relevant images are physically close to the query image, or the query regions can be identified by a set of clustering centers. however, semantically related images are often scattered across the visual space. it is not always reliable that the refined query point or the clustering centers are capable of representing a complex query region.in this work, we propose a novel relevance feedback approach which directly aims at extracting a set of samples to represent the query region, regardless of its underlying shape. the sample set extracted by our method is competent as well as compact for subsequent retrieval. moreover, we integrate feature re-weighting in the process to estimate the importance of each image descriptor. unlike most existing relevance feedback approaches in which all query points share a same feature weight distribution, our method re-weights the feature importance for each relevant image respectively, so that the representative and discriminative ability for all the images can be maximized. experimental results on two databases show the effectiveness of our approach.
cross-view action recognition from temporal self-similarities. this paper concerns recognition of human actions under view changes. we explore self-similarities of action sequences over time and observe the striking stability of such measures across views. building upon this key observation we develop an action descriptor that captures the structure of temporal similarities and dissimilarities within an action sequence. despite this descriptor not being strictly view-invariant, we provide intuition and experimental validation demonstrating the high stability of self-similarities under view changes. self-similarity descriptors are also shown stable under action variations within a class as well as discriminative for action recognition. interestingly, self-similarities computed from different image features possess similar properties and can be used in a complementary fashion. our method is simple and requires neither structure recovery nor multi-view correspondence estimation. instead, it relies on weak geometric properties and combines them with machine learning for efficient cross-view action recognition. the method is validated on three public datasets, it has similar or superior performance compared to related methods and it performs well even in extreme conditions such as when recognizing actions from top views while using side views for training only.
tracking with dynamic hidden-state shape models. hidden state shape models (hssms) were previously proposed to represent and detect objects in images that exhibit not just deformation of their shape but also variation in their structure. in this paper, we introduce dynamic hidden-state shape models (dhssms) to track and recognize the non-rigid motion of such objects, for example, human hands. our recursive bayesian filtering method, called dp-tracking, combines an exhaustive local search for a match between image features and model states with a dynamic programming approach to find a global registration between the model and the object in the image. our contribution is a technique to exploit the hierarchical structure of the dynamic programming approach that on average considerably speeds up the search for matches. we also propose to embed an online learning approach into the tracking mechanism that updates the dhssm dynamically. the learning approach ensures that the dhssm accurately represents the tracked object and distinguishes any clutter potentially present in the image. our experiments show that our method can recognize the digits of a hand while the fingers are being moved and curled to various degrees. the method is robust to various illumination conditions, the presence of clutter, occlusions, and some types of self-occlusions. the experiments demonstrate a significant improvement in both efficiency and accuracy of recognition compared to the non-recursive way of frame-by-frame detection.
hamming embedding and weak geometric consistency for large scale image search. this paper improves recent methods for large scale image search. state-of-the-art methods build on the bag-of-features image representation. we, first, analyze bag-of-features in the framework of approximate nearest neighbor search. this shows the sub-optimality of such a representation for matching descriptors and leads us to derive a more precise representation based on 1) hamming embedding (he) and 2) weak geometric consistency constraints (wgc). he provides binary signatures that refine the matching based on visual words. wgc filters matching descriptors that are not consistent in terms of angle and scale. he and wgc are integrated within the inverted file and are efficiently exploited for all images, even in the case of very large datasets. experiments performed on a dataset of one million of images show a significant improvement due to the binary signature and the weak geometric consistency constraints, as well as their efficiency. estimation of the full geometric transformation, i.e., a re-ranking step on a short list of images, is complementary to our weak geometric consistency constraints and allows to further improve the accuracy.