waashk/dblp huggingface dataset:
generative image segmentation using random walks with restart. we consider the problem of multi-label, supervised image segmentation when an initial labeling of some pixels is given. in this paper, we propose a new generative image segmentation algorithm for reliable multi-label segmentations in natural images. in contrast to most existing algorithms which focus on the inter-label discrimination, we address the problem of finding the generative model for each label. the primary advantage of our algorithm is that it produces very good segmentation results under two difficult problems: the weak boundary problem and the texture problem. moreover, single-label image segmentation is possible. these are achieved by designing the generative model with the random walks with restart (rwr). experimental results with synthetic and natural images demonstrate the relevance and accuracy of our algorithm.
signature-based document image retrieval. as the most pervasive method of individual identification and document authentication, signatures present convincing evidence and provide an important form of indexing for effective document image processing and retrieval in a broad range of applications. in this work, we developed a fully automatic signature-based document image retrieval system that handles: 1) automatic detection and segmentation of signatures from document images and 2) translation, scale, and rotation invariant signature matching for document image retrieval. we treat signature retrieval in the unconstrained setting of non-rigid shape matching and retrieval, and quantitatively study shape representations, shape matching algorithms, measures of dissimilarity, and the use of multiple query instances in document image retrieval. extensive experiments using large real world collections of english and arabic machine printed and handwritten documents demonstrate the excellent performance of our system. to the best of our knowledge, this is the first automatic retrieval system for general document images by using signatures as queries, without manual annotation of the image collection.
multi-thread parsing for recognizing complex events in videos. this paper presents a probabilistic grammar approach to the recognition of complex events in videos. firstly, based on the original motion features, a rule induction algorithm is adopted to learn the event rules. then, a multi-thread parsing (mtp) algorithm is adopted to recognize the complex events involving parallel temporal relation in sub-events, whereas the commonly used parser can only handle the sequential relation. additionally, a viterbi-like error recovery strategy is embedded in the parsing process to correct the large time scale errors, such as insertion and deletion errors. extensive experiments including indoor gymnastic exercises and outdoor traffic events are performed. as supported by experimental results, the mtp algorithm can effectively recognize the complex events due to the strong discriminative representation and the error recovery strategy.
projected texture for object classification. algorithms for classification of 3d objects either recover the depth information lost during imaging using multiple images, structured lighting, image cues, etc. or work directly the images for classification. while the latter class of algorithms are more efficient and robust in comparison, they are less accurate due to the lack of depth information. we propose the use of structured lighting patterns projected on the object, which gets deformed according to the shape of the object. since our goal is object classification and not shape recovery, we characterize the deformations using simple texture measures, thus avoiding the error prone and computationally expensive step of depth recovery. moreover, since the deformations encode depth variations of the object, the 3d shape information is implicitly used for classification. we show that the information thus derived can significantly improve the accuracy of object classification algorithms, and derive the theoretical limits on height variations that can be captured by a particular projector-camera setup. a 3d texture classification algorithm derived from the proposed approach achieves a ten-fold reduction in error rate on a dataset of 30 classes, when compared to state-of-the-art image based approaches. we also demonstrate the effectiveness of the approach for a hand geometry based authentication system, which achieves a four-fold reduction in the equal error rate on a dataset containing 149 users.
structuring visual words in 3d for arbitrary-view object localization. we propose a novel and efficient method for generic arbitrary-view object class detection and localization. in contrast to existing single-view and multi-view methods using complicated mechanisms for relating the structural information in different parts of the objects or different viewpoints, we aim at representing the structural information in their true 3d locations. uncalibrated multi-view images from a hand-held camera are used to reconstruct the 3d visual word models in the training stage. in the testing stage, beyond bounding boxes, our method can automatically determine the locations and outlines of multiple objects in the test image with occlusion handling, and can accurately estimate both the intrinsic and extrinsic camera parameters in an optimized way. with exemplar models, our method can also handle shape deformation for intra-class variance. to handle large data sets from models, we propose several speedup techniques to make the prediction efficient. experimental results obtained based on some standard data sets demonstrate the effectiveness of the proposed approach.
background subtraction on distributions. environmental monitoring applications present a challenge to current background subtraction algorithms that analyze the temporal variability of pixel intensities, due to the complex texture and motion of the scene. they also present a challenge to segmentation algorithms that compare intensity or color distributions between the foreground and the background in each image independently, because objects of interest such as animals have adapted to blend in. therefore, we have developed a background modeling and subtraction scheme that analyzes the temporal variation of intensity or color distributions, instead of either looking at temporal variation of point statistics, or the spatial variation of region statistics in isolation. distributional signatures are less sensitive to movements of the textured background, and at the same time they are more robust than individual pixel statistics in detecting foreground objects. they also enable slow background update, which is crucial in monitoring applications where processing power comes at a premium, and where foreground objects, when present, may move less than the background and therefore disappear into it when a fast update scheme is used. our approach compares favorably with the state of the art both in generic low-level detection metrics, as well as in application-dependent criteria.
an incremental learning method for unconstrained gaze estimation. this paper presents an online learning algorithm for appea- rance-based gaze estimation that allows free head movement in a casual desktop environment. our method avoids the lengthy calibration stage using an incremental learning approach. our system keeps running as a background process on the desktop pc and continuously updates the estimation parameters by taking user's operations on the pc monitor as input. to handle free head movement of a user, we propose a pose-based clustering approach that efficiently extends an appearance manifold model to handle the large variations of the head pose. the effectiveness of the proposed method is validated by quantitative performance evaluation with three users.
a linear time histogram metric for improved sift matching. we present a new metric between histograms such as sift descriptors and a linear time algorithm for its computation. it is common practice to use the l 2 metric for comparing sift descriptors. this practice assumes that sift bins are aligned, an assumption which is often not correct due to quantization, distortion, occlusion etc.in this paper we present a new earth mover's distance (emd) variant. we show that it is a metric (unlike the original emd [1] which is a metric only for normalized histograms). moreover, it is a natural extension of the l 1 metric. second, we propose a linear time algorithm for the computation of the emd variant, with a robust ground distance for oriented gradients. finally, extensive experimental results on the mikolajczyk and schmid dataset [2] show that our method outperforms state of the art distances.
multi-camera tracking and atypical motion detection with behavioral maps. we introduce a novel behavioral model to describe pedestrians motions, which is able to capture sophisticated motion patterns resulting from the mixture of different categories of random trajectories. due to its simplicity, this model can be learned from video sequences in a totally unsupervised manner through an expectation-maximization procedure.when integrated into a complete multi-camera tracking system, it improves the tracking performance in ambiguous situations, compared to a standard ad-hoc isotropic markovian motion model. moreover, it can be used to compute a score which characterizes atypical individual motions.experiments on outdoor video sequences demonstrate both the improvement of tracking performance when compared to a state-of-the-art tracking system and the reliability of the atypical motion detection.
non-local regularization of inverse problems. this article proposes a new framework to regularize linear inverse problems using the total variation on non-local graphs. this non-local graph allows to adapt the penalization to the geometry of the underlying function to recover. a fast algorithm computes iteratively both the solution of the regularization process and the non-local graph adapted to this solution. we show numerical applications of this method to the resolution of image processing inverse problems such as inpainting, super-resolution and compressive sampling.
learning two-view stereo matching. we propose a graph-based semi-supervised symmetric matching framework that performs dense matching between two uncalibrated wide-baseline images by exploiting the results of sparse matching as labeled data. our method utilizes multiple sources of information including the underlying manifold structure, matching preference, shapes of the surfaces in the scene, and global epipolar geometric constraints for occlusion handling. it can give inherent sub-pixel accuracy and can be implemented in a parallel fashion on a graphics processing unit (gpu). since the graphs are directly learned from the input images without relying on extra training data, its performance is very stable and hence the method is applicable under general settings. our algorithm is robust against outliers in the initial sparse matching due to our consideration of all matching costs simultaneously, and the provision of iterative restarts to reject outliers from the previous estimate. some challenging experiments have been conducted to evaluate the robustness of our method.
unified frequency domain analysis of lightfield cameras. this paper presents a theory that encompasses both "plenoptic" (microlens based) and "heterodyning" (mask based) cameras in a single frequency-domain mathematical formalism. light-field capture has traditionally been analyzed using spatio-angular representation, with the exception of the frequency-domain "heterodyning" work. in this paper we interpret "heterodyning" as a general theory of multiplexing the radiance in the frequency domain. using this interpretation, we derive a mathematical theory of recovering the 4d spatial and angular information from the multiplexed 2d frequency representation. the resulting method is applicable to all lightfield cameras, lens-based and mask-based. the generality of our approach suggests new designs for lightfield cameras. we present one such novel lightfield camera, based on a mask outside a conventional camera. experimental results are presented for all cameras described.
real-time shape analysis of a human body in clothing using time-series part-labeled volumes. we propose a real-time method for simultaneously refining the reconstructed volume of a human body with loose-fitting clothing and identifying body-parts in it. time-series volumes, which are acquired by a slow but sophisticated 3d reconstruction algorithm, with body-part labels are obtained offline. the time-series sample volumes are represented by trajectories in the eigenspaces using pca. an input visual hull reconstructed online is projected into the eigenspace and compared with the trajectories in order to find similar high-precision samples with body-part labels. the hierarchical search taking into account 3d reconstruction errors can achieve robust and fast matching. experimental results demonstrate that our method can refine the input visual hull including loose-fitting clothing and identify its body-parts in real time.
calibration from statistical properties of the visual world. what does a blind entity need in order to determine the geometry of the set of photocells that it carries through a changing lightfield? in this paper, we show that very crude knowledge of some statistical properties of the environment is sufficient for this task.we show that some dissimilarity measures between pairs of signals produced by photocells are strongly related to the angular separation between the photocells. based on real-world data, we model this relation quantitatively, using dissimilarity measures based on the correlation and conditional entropy. we show that this model allows to estimate the angular separation from the dissimilarity. although the resulting estimators are not very accurate, they maintain their performance throughout different visual environments, suggesting that the model encodes a very general property of our visual world.finally, leveraging this method to estimate angles from signal pairs, we show how distance geometry techniques allow to recover the complete sensor geometry.
using 3d line segments for robust and efficient change detection from multiple noisy images. in this paper, we propose a new approach to change detection that is based on the appearance or disappearance of 3d lines, which may be short, as seen in a new image. these 3d lines are estimated automatically and quickly from a set of previously-taken learning-images from arbitrary view points and under arbitrary lighting conditions. 3d change detection traditionally involves unsupervised estimation of scene geometry and the associated brdf at each observable voxel in the scene, and the comparison of a new image with its prediction. if a significant number of pixels differ in the two aligned images, a change in the 3d scene is assumed to have occurred. the importance of our approach is that by comparing images of lines rather than of gray levels, we avoid the computationally intensive, and some-times impossible, tasks of estimating 3d surfaces and their associated brdfs in the model-building stage. we estimate 3d lines instead where the lines are due to 3d ridges or brdf ridges which are computationally much less costly and are more reliably detected. our method is widely applicable as man-made structures consisting of 3d line segments are the main focus of most applications. the contributions of this paper are: change detection based on appropriate interpretation of line appearance and disappearance in a new image; unsupervised estimation of "short" 3d lines from multiple images such that the required computation is manageable and the estimation accuracy is high.
facetracer: a search engine for large collections of images with faces. we have created the first image search engine based entirely on faces. using simple text queries such as "smiling men with blond hair and mustaches," users can search through over 3.1 million faces which have been automatically labeled on the basis of several facial attributes. faces in our database have been extracted and aligned from images downloaded from the internet using a commercial face detector, and the number of images and attributes continues to grow daily. our classification approach uses a novel combination of support vector machines and adaboost which exploits the strong structure of faces to select and train on the optimal set of features for each attribute. we show state-of-the-art classification results compared to previous works, and demonstrate the power of our architecture through a functional, large-scale face search engine. our framework is fully automatic, easy to scale, and computes all labels off-line, leading to fast on-line search performance. in addition, we describe how our system can be used for a number of applications, including law enforcement, social networks, and personal photo management. our search engine will soon be made publicly available.
fusion of feature- and area-based information for urban buildings modeling from aerial imagery. accurate and realistic building models of urban environments are increasingly important for applications, like virtual tourism or city planning. initiatives like virtual earth or google earth are aiming at offering virtual models of all major cities world wide. the prohibitively high costs of manual generation of such models explain the need for an automatic workflow.this paper proposes an algorithm for fully automatic building reconstruction from aerial images. sparse line features delineating height discontinuities and dense depth data providing the roof surface are combined in an innovative manner with a global optimization algorithm based on graph cuts. the fusion process exploits the advantages of both information sources and thus yields superior reconstruction results compared to the indiviual sources. the nature of the algorithm also allows to elegantly generate image driven levels of detail of the geometry.the algorithm is applied to a number of real world data sets encompassing thousands of buildings. the results are analyzed in detail and extensively evaluated using ground truth data.
compressive structured light for recovering inhomogeneous participating media. we propose a new method named compressive structured light for recovering inhomogeneous participating media. whereas conventional structured light methods emit coded light patterns onto the surface of an opaque object to establish correspondence for triangulation, compressive structured light projects patterns into a volume of participating medium to produce images which are integral measurements of the volume density along the line of sight. for a typical participating medium encountered in the real world, the integral nature of the acquired images enables the use of compressive sensing techniques that can recover the entire volume density from only a few measurements. this makes the acquisition process more efficient and enables reconstruction of dynamic volumetric phenomena. moreover, our method requires the projection of multiplexed coded illumination, which has the added advantage of increasing the signal-to-noise ratio of the acquisition. finally, we propose an iterative algorithm to correct for the attenuation of the participating medium during the reconstruction process. we show the effectiveness of our method with simulations as well as experiments on the volumetric recovery of multiple translucent layers, 3d point clouds etched in glass, and the dynamic process of milk drops dissolving in water.
texture-consistent shadow removal. this paper presents an approach to shadow removal that preserves texture consistency between the original shadow and lit area. illumination reduction in the shadow area not only darkens that area, but also changes the texture characteristics there. we achieve texture-consistent shadow removal by constructing a shadow-free and texture-consistent gradient field. first, we estimate an illumination change surface which causes the shadow and remove the gradients it induces. we approximate the illumination change surface with illumination change splines across the shadow boundary. we formulate estimating these splines as an optimization problem which balances the smoothness between the neighboring splines and their fitness to the image data. second, we sample the shadow effect on the texture characteristics in the umbra and lit area near the shadow boundary, and remove it by transforming the gradients inside the shadow area to be compatible with the lit area. experiments on photos from flickr demonstrate the effectiveness of our method.
sample sufficiency and pca dimension for statistical shape models. statistical shape modelling(ssm) is a popular technique in computer vision applications, where the variation of shape of a given structure is modelled by principal component analysis (pca) on a set of training samples. the issue of sample size sufficiency is not generally considered. in this paper, we propose a framework to investigate the sources of ssm inaccuracy. based on this framework, we propose a procedure to determine sample size sufficiency by testing whether the training data stabilises the ssm. also, the number of principal modes to retain (pca dimension) is usually chosen using rules that aim to cover a percentage of the total variance or to limit the residual to a threshold. however, an ideal rule should retain modes that correspond to real structural variation and discard those that are dominated by noise. we show that these commonly used rules are not reliable, and we propose a new rule that uses bootstrap stability analysis on mode directions to determine the pca dimension.for validation we use synthetic 3d face datasets generated using a known number of structural modes with added noise. a 4-way anova is applied for the model reconstruction accuracy on sample size, shape vector dimension, pca dimension, and the noise level. it shows that there is no universal sample size guideline for ssm, nor is there a simple relationship to the shape vector dimension (with p-value=0.2932). validation of our rule for retaining structural modes showed it detected the correct number of modes to retain where the conventional methods failed. the methods were also tested on real 2d (22 points) and 3d (500 points) face data, retaining 24 and 70 modes with sample sufficiency being reached at approximately 50 and 150 samples respectively. we provide a foundation for appropriate selection of pca dimension and determination of sample size sufficiency in statistical shape modelling.
key object driven multi-category object recognition, localization and tracking using spatio-temporal context. in this paper we address the problem of recognizing, localizing and tracking multiple objects of different categories in meeting room videos. difficulties such as lack of detail and multi-object co-occurrence make it hard to directly apply traditional object recognition methods. under such circumstances, we show that incorporating object-level spatio-temporal relationships can lead to significant improvements in inference of object category and state. contextual relationships are modeled by a dynamic markov random field, in which recognition, localization and tracking are done simultaneously. further, we define human as the key object of the scene, which can be detected relatively robustly and therefore is used to guide the inference of other objects. experiments are done on the chil meeting video corpus. performance is evaluated in terms of object detection and false alarm rates, object recognition confusion matrix and pixel-level accuracy of object segmentation.