nils-herrmann/clean_scirepeval huggingface dataset:
Two-view Matching

Establishing correspondences between images of the same scene is one of the most challenging and critical stcps in motion and scene analysis. Part of the difficulty is due to a wide variety of three-dimension structural discontinuities and occlusions that occur in real world scenes. This paper describes a computational approach to image matching that uses multiple attributes associated with a pixel to yield a generally overdetermined system of constraints, taking into account possible structural discontinuities and occlusions. In the algorithm implemented, intensity, edgeness, and comemess attributes are used in conjunction with the constraints arising from intraregional smoothness, field continuity and discontinuity, and occlusions to compute dense displacement fields and occlusion maps at pixel grids. A multiresolution multigrid structure is employed to deal with large disparities. Coarser level attributes are obtained by blurring the finer level attributes. The algorithms are tested on real world scenes containing depth discontinuities and occlusions. A special case of two-view matching is stereo matching where the motion between two images is known. The general algorithm given here can be easily spccialized to perfonn stereo matching using epipolar line constraint.
Necessary and sufficient conditions for a unique solution of plane motion and structure

This paper presents necessary and sufficient conditions for uniquely determining the motion and structure of a planar surface from any number of monocular images. The authors first show an uncertain situation for the two-view plane motion problem in which an infinite number of solutions may result. Although this situation rarely occurs in real situations, it does raise visual ambiguities under certain circumstances, e.g., when some images are observed through a mirror. The understanding of this situation therefore helps analyze reflected images which are often seen in daily life. Then, the authors present necessary and sufficient conditions for a unique solution of plane motion and structure from any number of views. These algorithm-independent conditions enhance understanding about the problem of estimating plane motion and structure from image sequences and may be used to determine the uniqueness of the solution of the motion and structure of a planar surface.
A uniformity criterion and algorithm for data clustering

We propose a novel multivariate uniformity criterion for testing uniformity of point density in an arbitrary dimensional point pattern. An unsupervised, nonparametric data clustering algorithm, using this criterion, is also presented. The algorithm relies on a relatively general notion of cluster so that it is applicable to clusters of relatively unrestricted shapes, densities and sizes. We define a cluster as a set of contiguous interior points surrounded by border points. We use our uniformity test to differentiate between interior and border points. We group interior points to form cluster cores, and then identify cluster borders as formed by the border points neighboring the cluster cores. The algorithm is effective in resolving clusters of different shapes, sizes and densities. It is relatively insensitive to outliers. We present results for experiments performed on artificial and real data sets.
Generating Octrees from Object Silhouettes in Orthographic Views

An algorithm to construct the octree representation of a three-dimensional object from silhouette images of the object is described. The images must be obtained from thirteen viewing directions corresponding to the three face views, six edge views, and four corner views of an upright cube. These views where chosen because they provide a simple relationship between pixels in the image and the octant labels in the octree, thus replacing the computation of detecting intersections between the octree space and the objects by a table lookup operation. The average ratio of the object volume to the octree volume is found to be greater than 90%. The sequential use made of the chosen viewing directions results in a coarse-to-fine acquisition of occupancy information. The number and order of the viewpoints used provides a mechanism for trading accuracy of the representation against the computational effort needed to obtain the representation.
A constant-space belief propagation algorithm for stereo matching

In this paper, we consider the problem of stereo matching using loopy belief propagation. Unlike previous methods which focus on the original spatial resolution, we hierarchically reduce the disparity search range. By fixing the number of disparity levels on the original resolution, our method solves the message updating problem in a time linear in the number of pixels contained in the image and requires only constant memory space. Specifically, for a 800 × 600 image with 300 disparities, our message updating method is about 30× faster (1.5 second) than standard method, and requires only about 0.6% memory (9 MB). Also, our algorithm lends itself to a parallel implementation. Our GPU implementation (NVIDIA Geforce 8800GTX) is about 10× faster than our CPU implementation. Given the trend toward higher-resolution images, stereo matching using belief propagation with large number of disparity levels as efficient as the small ones makes our method future-proof. In addition to the computational and memory advantages, our method is straightforward to implement1.
Region correspondence by global configuration matching and progressive Delaunay triangulation

In this paper, we present a novel algorithm for establishing region correspondences across images by first matching global region configuration and then propagating the matches locally constrained by Delaunay triangulation. We exploit a global configuration constraint, which has not been explicitly used in existing matching algorithms. The proposed algorithm is comprised of two stages: first, stable regions are matched by enforcing the global configuration constraint. This yields a set of global matches corresponding to stable regions distributed over the images. In the second stage, these matches are used to guide the matching of the remaining unmatched regions in the intervening spaces. This is done by enforcing local positioning constraint, which starts with the Delaunay triangulation defined by the global matches and performs progressive Delaunay triangulation for local matching. Experiments on both stereo and motion images are presented to show the effectiveness of the proposed algorithm.
Spatial and Fourier Error Minimization for Motion Estimation and Segmentation

We present a new approach to motion estimation by minimizing the squared error in both the spatial and frequency domains and we show that the spatially global nature of FT leads to a motion estimation error that is much lower than that obtained via spatial motion estimation. On the other hand, spatial analysis is useful for accurate segmentation. We describe a novel, hybrid approach combining the above two estimates of motion and segmentation. We examine the robustness of minimizing the error terms in both domains, both theoretically and experimentally. Experiments with real and synthetic sequences demonstrate the capabilities of the proposed algorithm
Active Online Anomaly Detection Using Dirichlet Process Mixture Model and Gaussian Process Classification

We present a novel anomaly detection (AD) system for streaming videos. Different from prior methods that rely on unsupervised learning of clip representations, that are usually coarse in nature, and batch-mode learning, we propose the combination of two non-parametric models for our task: i) Dirichlet process mixture models (DPMM) based modeling of object motion and directions in each cell, and ii) Gaussian process based active learning paradigm involving labeling by a domain expert. Whereas conventional clip representation methods adopt quantizing only motion directions leading to a lossy, coarse representation that are inadequate, our clip representation approach results in fine grained clusters at each cell that model the scene activities (both direction and speed) more effectively. For active anomaly detection, we adapt a Gaussian Process framework to process incoming samples (video snippets) sequentially, seek labels for confusing or informative samples and and update the AD model online. Furthermore, the proposed video representation along with a novel query criterion to select informative samples for labeling that incorporates both exploration and exploitation criteria is proposed, and is found to outperform competing criteria on two challenging traffic scene datasets.
Recovering shape and reflectance model of non-lambertian objects from multiple views

This paper proposes an algorithm to simultaneously estimate both the 3D shape and parameters of a surface reflectance model from multiple views of an object made of a single material. The algorithm is based on a multiple view shape from shading method. A triangular mesh represents the shape of the object. The Phong reflectance model is used to model the surface reflectance. We iteratively find the shape and reflectance parameters that best fit all input images. Subdividing triangles in the mesh into smaller ones gradually refines the estimates of shape and reflectance model. The estimation takes into account both self-occlusion and self-shadowing. Analysis shows that the accuracy of reflectance estimation is limited by the triangle size in the shape model. We also propose to use Richardson extrapolation to overcome this and further refine the reflectance model estimate. The estimated 3D shape and reflectance model can be used to render the same object from different viewing directions and under different lighting conditions. Experimental results on both synthetic and real objects are given.
Shape From Texture: Integrating Texture-Element Extraction and Surface Estimation

A method is presented for identifying texture elements while simultaneously recovering the orientation of textured surfaces. A multiscale region detector, based on measurements in a Del /sup 2/G (Laplacian-of-Gaussian) scale space, is used to construct a set of candidate texture elements. True elements are selected from the set of candidate elements by finding the planar surface that best predicts the observed areas of the latter. Results are shown for a variety of natural textures, including waves, flowers, rocks, clouds, and dirt clods.
Extracting Texels in 2.1D Natural Textures

This paper proposes the problem of unsupervised extraction of texture elements, called texels, which repeatedly occur in the image of a frontally viewed, homogeneous, 2.1D, planar texture, and presents a solution. 2.1D texture here means that the physical texels are thin objects lying along a surface that may partially occlude one another. The image texture is represented by the segmentation tree whose structure captures the recursive embedding of regions obtained from a multiscale image segmentation. In the segmentation tree, the texels appear as subtrees with similar structure, with nodes having similar photometric and geometric properties. A new learning algorithm is proposed for fusing these similar subtrees into a tree-union, which registers all visible texel parts, and thus represents a statistical, generative model of the complete (unoccluded) texel. The learning algorithm involves concurrent estimation of texel tree structure, as well as the probability distributions of its node properties. Texel detection and segmentation are achieved simultaneously by matching the segmentation tree of a new image with the texel model. Experiments conducted on a newly compiled dataset containing 2.1D natural textures demonstrate the validity of our approach.
A unified model for activity recognition from video sequences

We propose an activity recognition algorithm that utilizes a unified spatial-frequency model of motion to recognize large-scale differences in action using global statistics, and subsequently distinguishes between motions with similar global statistics by spatially localizing the moving objects. We model the Fourier transforms of translating rigid objects in a video, since the Fourier domain inherently groups regions of the video with similar motion in high energy concentrations within its domain to make global motion detectable. Frequency-domain statistics can be used to isolate the frames that both adhere to our model and contain similar global motion, thus we can separate activities into broader classes based on their global motion. A least-squares solution is then solved to isolate the spatially discriminative object configurations that produce similar global motion statistics. This model provides a unified framework to form concise globally-optimal spatial and motion descriptors necessary for discriminating activities. Experimental results are demonstrated on a human activity dataset.
Wyner-Ziv encoded predictive multiple descriptions

The predictive multiple description coding problem can be posed as a variant of the well-known Wyner-Ziv side-information problem. Predictive MD coding in this framework (termed the WYZE-PMD) eliminates the problem of predictive mismatch without requiring restrictive channel assumptions or high latency. The performance of two-channel one-step predictive MD coding was analyzed within the WYZE-PMD framework. Achievable rate-distortion (R-D) regions were obtained for the problem of MD coding in the presence of correlated decoder side-information. These were used to obtain the operational R-D performance for predictive MD coding under certain restrictions. Practical code constructions were proposed within the WYZE-PMD framework. Performance comparisons between the proposed codes and conventional approaches were presented for communication of a first-order Gauss-Markov source over two erasure channels with independent failure probabilities. Results indicated that the proposed approach significantly out-performs conventional approaches in terms of R-D performance.
Estimating motion of constant acceleration from image sequences

Presents a model-based algorithm for estimating motion from monocular image sequences. The authors first present a two-view motion algorithm and then extend it to multiple views. The two-view algorithm requires generally 6 pairs of point correspondences to give unique solution of the motion parameters. However, when the used points lie on a Maybank quadric, the algorithm requires 7 pairs of point correspondences to give double solutions. Object-centered motion representations and a motion model of constant acceleration are used to estimate motion parameters from long image sequences. The algorithm guarantees globally optimal solution. Since the algorithm does not involve structure parameters, it contains the least number of unknowns and is hence more efficient and robust than the existing ones. Experimental results with real image data are presented. The same method can be applied to solve for motions described by second or higher orders of polynomials.<<ETX>>
Automated registration of multimodality images by maximization of a region similarity measure

This paper presents a robust algorithm for automated registration of images related by rigid-body transformations. This algorithm uses a new region-based similarity metric, which enables accurate registration of images of large contrast differences. Region segmentation required by the metric is accomplished using a multiscale segmentation algorithm, and minimization of this metric is done using the Powell direction set method. Experimental results are presented to demonstrate that the algorithm is effective for aligning images from single or multiple imaging modalities without the use of any fiducial markers.
A Uniform Framework for Estimating Illumination Chromaticity, Correspondence, and Specular Reflection

Based upon a new correspondence matching invariant called illumination chromaticity constancy, we present a new solution for illumination chromaticity estimation, correspondence searching, and specularity removal. Using as few as two images, the core of our method is the computation of a vote distribution for a number of illumination chromaticity hypotheses via correspondence matching. The hypothesis with the highest vote is accepted as correct. The estimated illumination chromaticity is then used together with the new matching invariant to match highlights, which inherently provides solutions for correspondence searching and specularity removal. Our method differs from the previous approaches: those treat these vision problems separately and generally require that specular highlights be detected in a preprocessing step. Also, our method uses more images than previous illumination chromaticity estimation methods, which increases its robustness because more inputs/constraints are used. Experimental results on both synthetic and real images demonstrate the effectiveness of the proposed method.
On generating seamless mosaics with large depth of field

Imaging cameras have only finite depth of field and only those objects within that depth range are simultaneously in focus. The depth of field of a camera can be improved by mosaicing a sequence of images taken under different focal settings. In conventional mosaicing schemes, a focus measure is computed for every scene point across the image sequence and the point is selected from that image where the focus measure is highest. We have, however, proved in this paper that the focus measure is not the highest in the best focussed frame for a certain class of scene points. The incorrect selection of image frames for these points, causes visual artifacts to appear in the resulting mosaic. We have also proposed a method to isolate such scene points, and an algorithm to compose large depth of field mosaics without the undesirable artifacts.
Wireless video conferencing using multiple description coding

The transmission of video over wireless is a particularly involved problem-requiring as it does the transmission of a variable length highly compressed, delay-sensitive stream over an error prone medium. In this paper, this problem is addressed within the multiple description coding paradigm. In particular the problem of wireless video conferencing over a constant bit-rate channel is addressed. A low-complexity, low-delay adaptive coder that uses multiple description coding to combat channel errors is proposed. Error propagation is curtailed by using a leaky prediction mechanism. A low-complexity rate control scheme based on the LMS algorithm is also proposed.
Low bit-rate video coding with implicit multiscale segmentation

Discusses a multiscale segmentation based video compression algorithm aimed at very low bit-rate applications such as video teleconferencing and video phones. We introduce novel techniques for multiscale segmentation based motion compensation and residual coding. Our region based forward motion compensation strategy (in terms of direction of motion vector, which is from the previous frame to the current frame) regulates the size and number of regions used, by pruning a multiscale segmentation of video frames. Since regions used for motion compensation are obtained by segmenting the previously decoded frame, the shape of the regions need not be transmitted to the decoder. Furthermore, our hierarchical motion compensation strategy refines an initial region level, coarse motion field to obtain a dense motion field which provides pixel level motion vectors. The refinement procedure does not require any additional information to be transmitted. This motion compensation technique effectively addresses the problem of dealing with "holes" and "overlapping regions" which are inherent to forward motion compensation. Residual coding is performed using a novel method which exploits the fact that the energy of the residual resulting from motion compensation is concentrated in a priori predictable positions. We show that this residual coding technique can also be extrapolated to improve the performance of coders using a block based motion compensation strategy. A fusion of these concepts leads to a gain of 2-3 dB in peak signal-to-noise ratio, apart from significant perceptual improvement, over a generic video coding algorithm using a block based motion compensation strategy (such as H.261 or H.263) for a variety of test sequences.
Unsupervised Category Modeling, Recognition, and Segmentation in Images

Suppose a set of arbitrary (unlabeled) images contains frequent occurrences of 2D objects from an unknown category. This paper is aimed at simultaneously solving the following related problems: 1) unsupervised identification of photometric, geometric, and topological properties of multiscale regions comprising instances of the 2D category, 2) learning a region-based structural model of the category in terms of these properties, and 3) detection, recognition, and segmentation of objects from the category in new images. To this end, each image is represented by a tree that captures a multiscale image segmentation. The trees are matched to extract the maximally matching subtrees across the set, which are taken as instances of the target category. The extracted subtrees are then fused into a tree union that represents the canonical category model. Detection, recognition, and segmentation of objects from the learned category are achieved simultaneously by finding matches of the category model with the segmentation tree of a new image. Experimental validation on benchmark data sets demonstrates the robustness and high accuracy of the learned category models when only a few training examples are used for learning without any human supervision.
3-D Motion Estimation, Understanding, and Prediction from Noisy Image Sequences

This paper presents an approach to understanding general 3-D motion of a rigid body from image sequences. Based on dynamics, a locally constant angular momentum (LCAM) model is introduced. The model is local in the sense that it is applied to a limited number of image frames at a time. Specifically, the model constrains the motion, over a local frame subsequence, to be a superposition of precession and translation. Thus, the instantaneous rotation axis of the object is allowed to change through the subsequence. The trajectory of the rotation center is approximated by a vector polynomial. The parameters of the model evolve in time so that they can adapt to long term changes in motion characteristics. The nature and parameters of short term motion can be estimated continuously with the goal of understanding motion through the image sequence. The estimation algorithm presented in this paper is linear, i.e., the algorithm consists of solving simultaneous linear equations. Based on the assumption that the motion is smooth, object positions and motion in the near future can be predicted, and short missing subsequences can be recovered. Noise smoothing is achieved by overdetermination and a leastsquares criterion. The framework is flexible in the sense that it allows both overdetermination in number of feature points and the number of image frames.