waashk/dblp huggingface dataset:
deformed lattice discovery via efficient mean-shift belief propagation. we introduce a novel framework for automatic detection of repeated patterns in real images. the novelty of our work is to formulate the extraction of an underlying deformed lattice as a spatial, multi-target tracking problem using a new and efficient mean-shift belief propagation (msbp) method. compared to existing work, our approach has multiple advantages, including: 1) incorporating higher order constraints early-on to propose highly plausible lattice points; 2) growing a lattice in multiple directions simultaneously instead of one at a time sequentially; and 3) achieving more efficient and more accurate performance than state-of-the-art algorithms. these advantages are demonstrated by quantitative experimental results on a diverse set of real world photos.
smd: a locally stable monotonic change invariant feature descriptor. extraction and matching of discriminative feature points in images is an important problem in computer vision with applications in image classification, object recognition, mosaicing, automatic 3d reconstruction and stereo. features are represented and matched via descriptors that must be invariant to small errors in the localization and scale of the extracted feature point, viewpoint changes, and other kinds of changes such as illumination, image compression and blur. while currently used feature descriptors are able to deal with many of such changes, they are not invariant to a generic monotonic change in the intensities, which occurs in many cases. furthermore, their performance degrades rapidly with many image degradations such as blur and compression where the intensity transformation is non-linear. in this paper, we present a new feature descriptor that obtains invariance to a monotonic change in the intensity of the patch by looking at orders between certain pixels in the patch. an order change between pixels indicates a difference between the patches which is penalized. summation of such penalties over carefully chosen pixel pairs that are stable to small errors in their localization and are independent of each other leads to a robust measure of change between two features. promising results were obtained using this approach that show significant improvement over existing methods, especially in the case of illumination change, blur and jpeg compression where the intensity of the points changes from one image to the next.
robust object tracking by hierarchical association of detection responses. we present a detection-based three-level hierarchical association approach to robustly track multiple objects in crowded environments from a single camera. at the low level, reliable tracklets (i.e. short tracks for further analysis) are generated by linking detection responses based on conservative affinity constraints. at the middle level, these tracklets are further associated to form longer tracklets based on more complex affinity measures. the association is formulated as a map problem and solved by the hungarian algorithm. at the high level, entries, exits and scene occluders are estimated using the already computed tracklets, which are used to refine the final trajectories. this approach is applied to the pedestrian class and evaluated on two challenging datasets. the experimental results show a great improvement in performance compared to previous methods.
optimization of symmetric transfer error for sub-frame video synchronization. in this work we present a method to synchronize video sequences of events that are acquired via uncalibrated cameras at unknown and dynamically varying temporal offsets. unlike existing methods that synchronize videos of similar events (i.e., videos related to each other through the motion in the scene) up to an integer alignment, we establish sub-frame video synchronization. while contemporary synchronization algorithms implement a unidirectional alignment which biases the results towards a single reference sequence, we adopt a bi-directional or symmetrical alignment approach that results in a more optimal synchronization. to this end, we propose a novel symmetric transfer error which is dynamically minimized, and reduces the propagation of error from feature extraction and spatial mapping into temporal synchronization. the advantages of our approach are validated by tests conducted on (publicly available) real and synthetic sequences. we present qualitative and quantitative comparisons with another state-of-the-art algorithm. a unique application of this work in generating high-resolution 4d mri data from multiple low-resolution mri scans is described.
linear time maximally stable extremal regions. in this paper we present a new algorithm for computing maximally stable extremal regions (mser), as invented by matas et al. the standard algorithm makes use of a union-find data structure and takes quasi-linear time in the number of pixels. the new algorithm provides exactly identical results in true worst-case linear time. moreover, the new algorithm uses significantly less memory and has better cache-locality, resulting in faster execution. our cpu implementation performs twice as fast as a state-of-the-art fpga implementation based on the standard algorithm.the new algorithm is based on a different computational ordering of the pixels, which is suggested by another immersion analogy than the one corresponding to the standard connected-component algorithm. with the new computational ordering, the pixels considered or visited at any point during computation consist of a single connected component of pixels in the image, resembling a flood-fill that adapts to the grey-level landscape. the computation only needs a priority queue of candidate pixels (the boundary of the single connected component), a single bit image masking visited pixels, and information for as many components as there are grey-levels in the image. this is substantially more compact in practice than the standard algorithm, where a large number of connected components must be considered in parallel. the new algorithm can also generate the component tree of the image in true linear time. the result shows that mser detection is not tied to the union-find data structure, which may open more possibilities for parallelization.
video registration using dynamic textures. we propose a dynamic texture feature-based algorithm for registering two video sequences of a rigid or nonrigid scene taken from two synchronous or asynchronous cameras. we model each video sequence as the output of a linear dynamical system, and transform the task of registering frames of the two sequences to that of registering the parameters of the corresponding models. this allows us to perform registration using the more classical image-based features as opposed to space-time features, such as space-time volumes or feature trajectories. as the model parameters are not uniquely defined, we propose a generic method to resolve these ambiguities by jointly identifying the parameters from multiple video sequences. we finally test our algorithm on a wide variety of challenging video sequences and show that it matches the performance of significantly more computationally expensive existing methods.
a comparative analysis of ransac techniques leading to adaptive real-time random sample consensus. the random sample consensus (ransac) algorithm is a popular tool for robust estimation problems in computer vision, primarily due to its ability to tolerate a tremendous fraction of outliers. there have been a number of recent efforts that aim to increase the efficiency of the standard ransac algorithm. relatively fewer efforts, however, have been directed towards formulating ransac in a manner that is suitable for real-time implementation. the contributions of this work are two-fold: first, we provide a comparative analysis of the state-of-the-art ransac algorithms and categorize the various approaches. second, we develop a powerful new framework for real-time robust estimation. the technique we develop is capable of efficiently adapting to the constraints presented by a fixed time budget, while at the same time providing accurate estimation over a wide range of inlier ratios. the method shows significant improvements in accuracy and speed over existing techniques.
hierarchical support vector random fields: joint training to combine local and global features. recently, impressive results have been reported for the detection of objects in challenging real-world scenes. interestingly however, the underlying models vary greatly even between the most successful approaches. methods using a global feature descriptor (e.g. ) paired with discriminative classifiers such as svms enable high levels of performance, but require large amounts of training data and typically degrade in the presence of partial occlusions. local feature-based approaches (e.g. ) are more robust in the presence of partial occlusions but often produce a significant number of false positives. this paper proposes a novel approach called hierarchical support vector random field that allows 1) to combine the power of global feature-based approaches with the flexibility of local feature-based methods in one consistent multi-layer framework and 2) to automatically learn the tradeoff and the optimal interplay between local, semi-local and global feature contributions. experiments show that both the combination of local and global features as well as the joint training result in improved detection performance on challenging datasets.
fourier analysis of the 2d screened poisson equation for gradient domain problems. we analyze the problem of reconstructing a 2d function that approximates a set of desired gradients and a data term. the combined data and gradient terms enable operations like modifying the gradients of an image while staying close to the original image. starting with a variational formulation, we arrive at the "screened poisson equation" known in physics. analysis of this equation in the fourier domain leads to a direct, exact, and efficient solution to the problem. further analysis reveals the structure of the spatial filters that solve the 2d screened poisson equation and shows gradient scaling to be a well-defined sharpen filter that generalizes laplacian sharpening, which itself can be mapped to gradient domain filtering. results using a dct-based screened poisson solver are demonstrated on several applications including image blending for panoramas, image sharpening, and de-blocking of compressed images.
an efficient dense and scale-invariant spatio-temporal interest point detector. over the years, several spatio-temporal interest point detectors have been proposed. while some detectors can only extract a sparse set of scale-invariant features, others allow for the detection of a larger amount of features at user-defined scales. this paper presents for the first time spatio-temporal interest points that are at the same time scale-invariant (both spatially and temporally) and densely cover the video content. moreover, as opposed to earlier work, the features can be computed efficiently. applying scale-space theory, we show that this can be achieved by using the determinant of the hessian as the saliency measure. computations are speeded-up further through the use of approximative box-filter operations on an integral video structure. a quantitative evaluation and experimental results on action recognition show the strengths of the proposed detector in terms of repeatability, accuracy and speed, in comparison with previously proposed detectors.
efficient edge-based methods for estimating manhattan frames in urban imagery. we address the problem of efficiently estimating the rotation of a camera relative to the canonical 3d cartesian frame of an urban scene, under the so-called "manhattan world" assumption [1,2]. while the problem has received considerable attention in recent years, it is unclear how current methods stack up in terms of accuracy and efficiency, and how they might best be improved. it is often argued that it is best to base estimation on all pixels in the image [2]. however, in this paper, we argue that in a sense, less can be more: that basing estimation on sparse, accurately localized edges, rather than dense gradient maps, permits the derivation of more accurate statistical models and leads to more efficient estimation. we also introduce and compare several different search techniques that have advantages over prior approaches. a cornerstone of the paper is the establishment of a new public groundtruth database which we use to derive required statistics and to evaluate and compare algorithms.
regularized partial matching of rigid shapes. matching of rigid shapes is an important problem in numerous applications across the boundary of computer vision, pattern recognition and computer graphics communities. a particularly challenging setting of this problem is partial matching, where the two shapes are dissimilar in general, but have significant similar parts. in this paper, we show a rigorous approach allowing to find matching parts of rigid shapes with controllable size and regularity. the regularity term we use is similar to the spirit of the mumford-shah functional, extended to non-euclidean spaces. numerical experiments show that the regularized partial matching produces better results compared to the non-regularized one.
unsupervised classification and part localization by consistency amplification. we present a novel method for unsupervised classification, including the discovery of a new category and precise object and part localization. given a set of unlabelled images, some of which contain an object of an unknown category, with unknown location and unknown size relative to the background, the method automatically identifies the images that contain the objects, localizes them and their parts, and reliably learns their appearance and geometry for subsequent classification. current unsupervised methods construct classifiers based on a fixed set of initial features. instead, we propose a new approach which iteratively extracts new features and re-learns the induced classifier, improving class vs. non-class separation at each iteration. we develop two main tools that allow this iterative combined search. the first is a novel star-like model capable of learning a geometric class representation in the unsupervised setting. the second is learning of "part specific features" that are optimized for parts detection, and which optimally combine different part appearances discovered in the training examples. these novel aspects lead to precise part localization and to improvement in overall classification performance compared with previous methods. we applied our method to multiple object classes from caltech-101, uiuc and a sub-classification problem from pascal. the obtained results are comparable to state-of-the-art supervised classification techniques and superior to state-of-the-art unsupervised approaches previously applied to the same image sets.
simultaneous visual recognition of manipulation actions and manipulated objects. the visual analysis of human manipulation actions is of interest for e.g. human-robot interaction applications where a robot learns how to perform a task by watching a human. in this paper, a method for classifying manipulation actions in the context of the objects manipulated, and classifying objects in the context of the actions used to manipulate them is presented. hand and object features are extracted from the video sequence using a segmentation based approach. a shape based representation is used for both the hand and the object. experiments show this representation suitable for representing generic shape classes. the action-object correlation over time is then modeled using conditional random fields. experimental comparison show great improvement in classification rate when the action-object correlation is taken into account, compared to separate classification of manipulation actions and manipulated objects.
finding actions using shape flows. we propose a novel method for action detection based on a new action descriptor called a shape flow that represents both the shape and movement of an object in a holistic and parsimonious manner. we find actions by finding shape flows in a target video that are similar to a template shape flow. shape flows are largely independent of appearance, and the match cost function that we propose is invariant to scale changes and smooth nonlinear deformation in space and time. the problem of matching shape flows is difficult, however, yielding a large, non-convex, integer program. we propose a novel relaxation method based on successive convexification that converts this hard program into a vastly smaller linear program: by using only those variables that appear on the 4d lower convex hull of the matching cost volume, most of the variables in the linear program may be eliminated. experiments confirm that the proposed shape flow method can successfully detect complex actions in cluttered video, even with self-occlusion, camera motion, and intra-class variation.
scene segmentation using the wisdom of crowds. given a collection of images of a static scene taken by many different people, we identify and segment interesting objects. to solve this problem, we use the distribution of images in the collection along with a new field-of-view cue, which leverages the observation that people tend to take photos that frame an object of interest within the field of view. hence, image features that appear together in many images are likely to be part of the same object. we evaluate the effectiveness of this cue by comparing the segmentations computed by our method against hand-labeled ones for several different models. we also show how the results of our segmentations can be used to highlight important objects in the scene and label them using noisy user-specified textual tag data. these methods are demonstrated on photos of several popular tourist sites downloaded from the internet.
feature correspondence via graph matching: models and global optimization. in this paper we present a new approach for establishing correspondences between sparse image features related by an unknown non-rigid mapping and corrupted by clutter and occlusion, such as points extracted from a pair of images containing a human figure in distinct poses. we formulate this matching task as an energy minimization problem by defining a complex objective function of the appearance and the spatial arrangement of the features. optimization of this energy is an instance of graph matching, which is in general a np-hard problem. we describe a novel graph matching optimization technique, which we refer to as dual decomposition (dd), and demonstrate on a variety of examples that this method outperforms existing graph matching algorithms. in the majority of our examples dd is able to find the global minimum within a minute. the ability to globally optimize the objective allows us to accurately learn the parameters of our matching model from training examples. we show on several matching tasks that our learned model yields results superior to those of state-of-the-art methods.
fast automatic single-view 3-d reconstruction of urban scenes. we consider the problem of estimating 3-d structure from a single still image of an outdoor urban scene. our goal is to efficiently create 3-d models which are visually pleasant. we chose an appropriate 3-d model structure and formulate the task of 3-d reconstruction as model fitting problem. our 3-d models are composed of a number of vertical walls and a ground plane, where ground-vertical boundary is a continuous polyline. we achieve computational efficiency by special preprocessing together with stepwise search of 3-d model parameters dividing the problem into two smaller sub-problems on chain graphs. the use of conditional random field models for both problems allows to various cues. we infer orientation of vertical walls of 3-d model vanishing points.
statistical analysis of global motion chains. multiple elements such as lighting, colors, dialogue, and camera motion contribute to the style of a movie. among them, camera motion is commonly overlooked yet a crucial point. for instance, documentaries tend to use long smooth pans whereas action movies usually have short and dynamic movements. this information, also referred to as global motion, could be leveraged by various applications in video clustering, stabilization, and editing. we perform analyses to study the in-class characteristics of these motions as well as their relationship with motions of other movie types. in particular, we model global motion as a multi-scale distribution of transformation matrices from frame to frame. secondly, we quantify the difference between pairs of videos using the kl-divergence of these distributions. finally, we demonstrate an application modeling and clustering commercial and amateur videos. experiments performed show advantage compared to the usage of some local motion-based approaches.
region-based 2d deformable generalized cylinder for narrow structures segmentation. in this paper, we present a region-based deformable cylinder model, extending the work on classical region-based active contours and gradient-based ribbon snakes. defined by a central curve playing the role of the medial axis and a variable thickness, the model is endowed with a region-dependent term.this energy follows the narrow band principle, in order to handle local region properties while overcoming limitations of classical edge-based models. the energy is subsequently transformed and derived in order to allow implementation on a polygonal line deformed with gradient descent. the model is used to extract path-like objects in medical and aerial images.
robust scale estimation from ensemble inlier sets for random sample consensus methods. this paper proposes a ransac modification that performs automatic estimation of the scale of inlier noise. the scale estimation takes advantage of accumulated inlier sets from all proposed models. it is shown that the proposed method gives robust results in case of high outlier ratio data, in spite that no user specified threshold is needed. the method also improves sampling efficiency, without requiring any auxiliary information other than the data to be modeled.
kernel codebooks for scene categorization. this paper introduces a method for scene categorization by modeling ambiguity in the popular codebook approach. the codebook approach describes an image as a bag of discrete visual codewords, where the frequency distributions of these words are used for image categorization. there are two drawbacks to the traditional codebook model: codeword uncertainty and codeword plausibility. both of these drawbacks stem from the hard assignment of visual features to a single codeword. we show that allowing a degree of ambiguity in assigning codewords improves categorization performance for three state-of-the-art datasets.