waashk/dblp huggingface dataset:
toward global minimum through combined local minima. there are many local and greedy algorithms for energy minimization over markov random field (mrf) such as iterated condition mode (icm) and various gradient descent methods. local minima solutions can be obtained with simple implementations and usually require smaller computational time than global algorithms. also, methods such as icm can be readily implemented in a various difficult problems that may involve larger than pairwise clique mrfs. however, their short comings are evident in comparison to newer methods such as graph cut and belief propagation. the local minimum depends largely on the initial state, which is the fundamental problem of its kind. in this paper, disadvantages of local minima techniques are addressed by proposing ways to combine multiple local solutions. first, multiple icm solutions are obtained using different initial states. the solutions are combined with random partitioning based greedy algorithm called combined local minima (clm). there are numerous mrf problems that cannot be efficiently implemented with graph cut and belief propagation, and so by introducing ways to effectively combine local solutions, we present a method to dramatically improve many of the pre-existing local minima algorithms. the proposed approach is shown to be effective on pairwise stereo mrf compared with graph cut and sequential tree re-weighted belief propagation (trw-s). additionally, we tested our algorithm against belief propagation (bp) over randomly generated 30 ×30 mrf with 2 ×2 clique potentials, and we experimentally illustrate clm's advantage over message passing algorithms in computation complexity and performance.
temporal dithering of illumination for fast active vision. active vision techniques use programmable light sources, such as projectors, whose intensities can be controlled over space and time. we present a broad framework for fast active vision using digital light processing (dlp) projectors. the digital micromirror array (dmd) in a dlp projector is capable of switching mirrors "on" and "off" at high speeds (106/s). an off-the-shelf dlp projector, however, effectively operates at much lower rates (30-60hz) by emitting smaller intensities that are integrated over time by a sensor (eye or camera) to produce the desired brightness value. our key idea is to exploit this "temporal dithering" of illumination, as observed by a high-speed camera. the dithering encodes each brightness value uniquely and may be used in conjunction with virtually any active vision technique. we apply our approach to five well-known problems: (a) structured light-based range finding, (b) photometric stereo, (c) illumination de-multiplexing, (d) high frequency preserving motion-blur and (e) separation of direct and global scene components, achieving significant speedups in performance. in all our methods, the projector receives a single image as input whereas the camera acquires a sequence of frames.
what does the sky tell us about the camera? as the main observed illuminant outdoors, the sky is a rich source of information about the scene. however, it is yet to be fully explored in computer vision because its appearance depends on the sun position, weather conditions, photometric and geometric parameters of the camera, and the location of capture. in this paper, we propose the use of a physically-based sky model to analyze the information available within the visible portion of the sky, observed over time. by fitting this model to an image sequence, we show how to extract camera parameters such as the focal length, and the zenith and azimuth angles. in short, the sky serves as a geometric calibration target. once the camera parameters are recovered, we show how to use the same model in two applications: 1) segmentation of the sky and cloud layers, and 2) data-driven sky matching across different image sequences based on a novel similarity measure defined on sky parameters. this measure, combined with a rich appearance database, allows us to model a wide range of sky conditions.
flexible depth of field photography. the range of scene depths that appear focused in an image is known as the depth of field (dof). conventional cameras are limited by a fundamental trade-off between depth of field and signal-to-noise ratio (snr). for a dark scene, the aperture of the lens must be opened up to maintain snr, which causes the dof to reduce. also, today's cameras have dofs that correspond to a single slab that is perpendicular to the optical axis. in this paper, we present an imaging system that enables one to control the dof in new and powerful ways. our approach is to vary the position and/or orientation of the image detector, during the integration time of a single photograph. even when the detector motion is very small (tens of microns), a large range of scene depths (several meters) is captured both in and out of focus.our prototype camera uses a micro-actuator to translate the detector along the optical axis during image integration. using this device, we demonstrate three applications of flexible dof. first, we describe extended dof, where a large depth range is captured with a very wide aperture (low noise) but with nearly depth-independent defocus blur. applying deconvolution to a captured image gives an image with extended dof and yet high snr. next, we show the capture of images with discontinuous dofs. for instance, near and far objects can be imaged with sharpness while objects in between are severely blurred. finally, we show that our camera can capture images with tilted dofs (scheimpflug imaging) without tilting the image detector. we believe flexible dof imaging can open a new creative dimension in photography and lead to new capabilities in scientific imaging, vision, and graphics.
movie/script: alignment and parsing of video and text transcription. movies and tv are a rich source of diverse and complex video of people, objects, actions and locales "in the wild". harvesting automatically labeled sequences of actions from video would enable creation of large-scale and highly-varied datasets. to enable such collection, we focus on the task of recovering scene structure in movies and tv series for object tracking and action retrieval. we present a weakly supervised algorithm that uses the screenplay and closed captions to parse a movie into a hierarchy of shots and scenes. scene boundaries in the movie are aligned with screenplay scene labels and shots are reordered into a sequence of long continuous tracks or threads which allow for more accurate tracking of people, actions and objects. scene segmentation, alignment, and shot threading are formulated as inference in a unified generative model and a novel hierarchical dynamic programming algorithm that can handle alignment and jump-limited reorderings in linear time is presented. we present quantitative and qualitative results on movie alignment and parsing, and use the recovered structure to improve character naming and retrieval of common actions in several episodes of popular tv series.
what is a good image segment? a unified approach to segment extraction. there is a huge diversity of definitions of "visually meaningful" image segments, ranging from simple uniformly colored segments, textured segments, through symmetric patterns, and up to complex semantically meaningful objects. this diversity has led to a wide range of different approaches for image segmentation. in this paper we present a single unified framework for addressing this problem --- "segmentation by composition". we define a good image segment as one which can be easily composed using its own pieces, but is difficult to compose using pieces from other parts of the image. this non-parametric approach captures a large diversity of segment types, yet requires no pre-definition or modelling of segment types, nor prior training. based on this definition, we develop a segment extraction algorithm --- i.e., given a single point-of-interest, provide the "best" image segment containing that point. this induces a figure-ground image segmentation, which applies to a range of different segmentation tasks: single image segmentation, simultaneous co-segmentation of several images, and class-based segmentations.
semantic concept classification by joint semi-supervised learning of feature subspaces and support vector machines. the scarcity of labeled training data relative to the high-dimensionality multi-modal features is one of the major obstacles for semantic concept classification of images and videos. semi-supervised learning leverages the large amount of unlabeled data in developing effective classifiers. feature subspace learning finds optimal feature subspaces for representing data and helping classification. in this paper, we present a novel algorithm, locality preserving semi-supervised support vector machines (lpssvm), to jointly learn an optimal feature subspace as well as a large margin svm classifier. over both labeled and unlabeled data, an optimal feature subspace is learned that can maintain the smoothness of local neighborhoods as well as being discriminative for classification. simultaneously, an svm classifier is optimized in the learned feature subspace to have large margin. the resulting classifier can be readily used to handle unseen test data. additionally, we show that the lpssvm algorithm can be used in a reproducing kernel hilbert space for nonlinear classification. we extensively evaluate the proposed algorithm over four types of data sets: a toy problem, two uci data sets, the caltech 101 data set for image classification, and the challenging kodak's consumer video data set for semantic concept detection. promising results are obtained which clearly confirm the effectiveness of the proposed method.
linking pose and motion. algorithms designed to estimate 3d pose in video sequences enforce temporal consistency but typically overlook an important source of information: the 3d pose of an object, be it rigid or articulated, has a direct influence on its direction of travel.in this paper, we use the cases of an airplane performing aerobatic maneuvers and of pedestrians walking and turning to demonstrate that this information can and should be used to increase the accuracy and reliability of pose estimation algorithms.
automated delineation of dendritic networks in noisy image stacks. we present a novel approach to 3d delineation of dendritic networks in noisy image stacks. we achieve a level of automation beyond that of state-of-the-art systems, which model dendrites as continuous tubular structures and postulate simple appearance models. instead, we learn models from the data itself, which make them better suited to handle noise and deviations from expected appearance.from very little expert-labeled ground truth, we train both a classifier to recognize individual dendrite voxels and a density model to classify segments connecting pairs of points as dendrite-like or not. given these models, we can then trace the dendritic trees of neurons automatically by enforcing the tree structure of the resulting graph. we will show that our approach performs better than traditional techniques on brighfield image stacks.
scene discovery by matrix factorization. what constitutes a scene? defining a meaningful vocabulary for scene discovery is a challenging problem that has important consequences for object recognition. we consider scenes to depict correlated objects and present visual similarity. we introduce a max-margin factorization model that finds a low dimensional subspace with high discriminative power for correlated annotations. we postulate this space should allow us to discover a large number of scenes in unsupervised data; we show scene discrimination results on par with supervised approaches. this model also produces state of the art word prediction results including good annotation completion.
riemannian anisotropic diffusion for tensor valued images. tensor valued images, for instance originating from diffusion tensor magnetic resonance imaging (dt-mri), have become more and more important over the last couple of years. due to the nonlinear structure of such data it is nontrivial to adapt well-established image processing techniques to them. in this contribution we derive anisotropic diffusion equations for tensor-valued images based on the intrinsic riemannian geometric structure of the space of symmetric positive tensors. in contrast to anisotropic diffusion approaches proposed so far, which are based on the euclidian metric, our approach considers the nonlinear structure of positive definite tensors by means of the intrinsic riemannian metric. together with an intrinsic numerical scheme our approach overcomes a main drawback of former proposed anisotropic diffusion approaches, the so-called eigenvalue swelling effect. experiments on synthetic data as well as real dt-mri data demonstrate the value of a sound differential geometric formulation of diffusion processes for tensor valued data.
differential spatial resection - pose estimation using a single local image feature. robust local image features have been used successfully in robot localization and camera pose estimation; region tracking using affine warps is considered state of the art also for many years. although such correspondences provide a warp of the local image region and are quite powerful, in direct pose estimation they are so far only considered as points and therefore three of them are required to construct a camera pose. in this contribution we show how it is possible to directly compute a pose based upon one such feature, given the plane in space where it lies. this differential correspondence concept exploits the texture warp and has recently gained attention in estimation of conjugate rotations. the approach can also be considered as the limiting case of the well-known spatial resection problem when the three 3d points approach each other infinitesimally close. we show that the differential correspondence is more powerful than conic correspondences while its exploitation requires nothing more complicated than the roots of a third order polynomial. we give a detailed sensitivity analysis, a comparison against state-of-the-art pose estimators and demonstrate real-world applicability of the algorithm based on automatic region recognition.
simultaneous detection and registration for ileo-cecal valve detection in 3d ct colonography. object detection and recognition has achieved a significant progress in recent years. however robust 3d object detection and segmentation in noisy 3d data volumes remains a challenging problem. localizing an object generally requires its spatial configuration (i.e., pose, size) being aligned with the trained object model, while estimation of an object's spatial configuration is only valid at locations where the object appears. detecting object while exhaustively searching its spatial parameters, is computationally prohibitive due to the high dimensionality of 3d search space. in this paper, we circumvent this computational complexity by proposing a novel framework capable of incrementally learning the object parameters (ipl) of location, pose and scale. this method is based on a sequence of binary encodings of the projected true positives from the original 3d object annotations (i.e., the projections of the global optima from the global space into the sections of subspaces). the training samples in each projected subspace are labeled as positive or negative, according their spatial registration distances towards annotations as ground-truth. each encoding process can be considered as a general binary classification problem and is implemented using probabilistic boosting tree algorithm. we validate our approach with extensive experiments and performance evaluations for ileo-cecal valve (icv) detection in both clean and tagged 3d ct colonography scans. our final icv detection system also includes an optional prior learning procedure for ipl which further speeds up the detection.
dynamic integration of generalized cues for person tracking. we present an approach for the dynamic combination of multiple cues in a particle filter-based tracking framework. the proposed algorithm is based on a combination of democratic integration and layered sampling. it is capable of dealing with deficiencies of single features as well as partial occlusion using the very same dynamic fusion mechanism. a set of simple but fast cues is defined, which allow us to cope with limited computational resources. the system is capable of automatic track initialization by means of a dedicated attention tracker permanently scanning the surroundings.
image segmentation by branch-and-mincut. efficient global optimization techniques such as graph cut exist for energies corresponding to binary image segmentation from low-level cues. however, introducing a high-level prior such as a shape prior or a color-distribution prior into the segmentation process typically results in an energy that is much harder to optimize. the main contribution of the paper is a new global optimization framework for a wide class of such energies. the framework is built upon two powerful techniques: graph cut and branch-and-bound. these techniques are unified through the derivation of lower bounds on the energies. being computable via graph cut, these bounds are used to prune branches within a branch-and-bound search.we demonstrate that the new framework can compute globally optimal segmentations for a variety of segmentation scenarios in a reasonable time on a modern cpu. these scenarios include unsupervised segmentation of an object undergoing 3d pose change, category-specific shape segmentation, and the segmentation under intensity/color priors defined by chan-vese and grabcut functionals.
robust visual tracking based on an effective appearance model. most existing appearance models for visual tracking usually construct a pixel-based representation of object appearance so that they are incapable of fully capturing both global and local spatial layout information of object appearance. in order to address this problem, we propose a novel spatial log-euclidean appearance model (referred as slam) under the recently introduced log-euclidean riemannian metric [23]. slam is capable of capturing both the global and local spatial layout information of object appearance by constructing a block-based log-euclidean eigenspace representation. specifically, the process of learning the proposed slam consists of five steps--appearance block division, online log-euclidean eigenspace learning, local spatial weighting, global spatial weighting, and likelihood evaluation. furthermore, a novel online log-euclidean riemannian subspace learning algorithm (irsl) [14] is applied to incrementally update the proposed slam. tracking is then led by the bayesian state inference framework in which a particle filter is used for propagating sample distributions over the time. theoretic analysis and experimental evaluations demonstrate the promise and effectiveness of the proposed slam.
light-efficient photography. we consider the problem of imaging a scene with a given depth of field at a given exposure level in the shortest amount of time possible. we show that by (1) collecting a sequence of photos and (2) controlling the aperture, focus and exposure time of each photo individually, we can span the given depth of field in less total time than it takes to expose a single narrower-aperture photo. using this as a starting point, we obtain two key results. first, for lenses with continuously-variable apertures, we derive a closed-form solution for the globally optimal capture sequence, i.e., that collects light from the specified depth of field in the most efficient way possible. second, for lenses with discrete apertures, we derive an integer programming problem whose solution is the optimal sequence. our results are applicable to off-the-shelf cameras and typical photography conditions, and advocate the use of dense, wide-aperture photo sequences as a light-efficient alternative to single-shot, narrow-aperture photography.
priors for large photo collections and what they reveal about cameras. a large photo collection downloaded from the internet spans a wide range of scenes, cameras, and photographers. in this paper we introduce several novel priors for statistics of such large photo collections that are independent of these factors. we then propose that properties of these factors can be recovered by examining the deviation between these statistical priors and the statistics of a slice of the overall photo collection that holds one factor constant. specifically, we recover the radiometric properties of a particular camera model by collecting numerous images captured by it, and examining the deviation of this collection's statistics from that of a broader photo collection whose camera-specific effects have been removed. we show that using this approach we can recover both a camera model's non-linear response function and the spatially-varying vignetting of the camera's different lens settings. all this is achieved using publicly available photographs, without requiring images captured under controlled conditions or physical access to the cameras. we also apply this concept to identify bad pixels on the detectors of specific camera instances. we conclude with a discussion of future applications of this general approach to other common computer vision problems.
understanding camera trade-offs through a bayesian analysis of light field projections. computer vision has traditionally focused on extracting structure, such as depth, from images acquired using thin-lens or pinhole optics. the development of computational imaging is broadening this scope; a variety of unconventional cameras do not directly capture a traditional image anymore, but instead require the joint reconstruction of structure and image information. for example, recent coded aperture designs have been optimized to facilitate the joint reconstruction of depth and intensity. the breadth of imaging designs requires new tools to understand the tradeoffs implied by different strategies.this paper introduces a unified framework for analyzing computational imaging approaches. each sensor element is modeled as an inner product over the 4d light field. the imaging task is then posed as bayesian inference: given the observed noisy light field projections and a prior on light field signals, estimate the original light field. under common imaging conditions, we compare the performance of various camera designs using 2d light field simulations. this framework allows us to better understand the tradeoffs of each camera type and analyze their limitations.
passive reflectometry. different materials reflect light in different ways, so reflectance is a useful surface descriptor. existing systems for measuring reflectance are cumbersome, however, and although the process can be streamlined using cameras, projectors and clever catadioptrics, it generally requires complex infrastructure. in this paper we propose a simpler method for inferring reflectance from images, one that eliminates the need for active lighting and exploits natural illumination instead. the method's distinguishing property is its ability to handle a broad class of isotropic reflectance functions, including those that are neither radially-symmetric nor well-represented by low-parameter reflectance models. the key to the approach is a bi-variate representation of isotropic reflectance that enables a tractable inference algorithm while maintaining generality. the resulting method requires only a camera, a light probe, and as little as one hdr image of a known, curved, homogeneous surface.