nils-herrmann/clean_scirepeval huggingface dataset:
Stereo Matching Using Epipolar Distance Transform

In this paper, we propose a simple but effective image transform, called the epipolar distance transform, for matching low-texture regions. It converts image intensity values to a relative location inside a planar segment along the epipolar line, such that pixels in the low-texture regions become distinguishable. We theoretically prove that the transform is affine invariant, thus the transformed images can be directly used for stereo matching. Any existing stereo algorithms can be directly used with the transformed images to improve reconstruction accuracy for low-texture regions. Results on real indoor and outdoor images demonstrate the effectiveness of the proposed transform for matching low-texture regions, keypoint detection, and description for low-texture scenes. Our experimental results on Middlebury images also demonstrate the robustness of our transform for highly textured scenes. The proposed transform has a great advantage, its low computational complexity. It was tested on a MacBook Air laptop computer with a 1.8 GHz Core i7 processor, with a speed of about 9 frames per second for a video graphics array-sized image.
Single camera stereo using planar parallel plate

A system of using a planar parallel plate to achieve single camera stereo has been proposed by Nishimoto and Shirai . Their work was based on an assumption that the optical axis of the camera was equally displaced by a tilted planar plate with two fixed tilt angles. Such assumption is invalid in general. We propose a general framework to more accurately model such a single camera system using a planar parallel plate and to deal with arbitrary orientation of the plate. Our model has no limitation on the FOV of the camera. The proposed framework includes: a mathematical formulation to calculate the displacement of scene points; a plate calibration method to determine the plate intrinsic parameters as well as its extrinsic pose, and a generalized correspondence method which is capable of dealing with arbitrary number of images captured from different plate poses. Experimental result is included to validate the proposed framework.
Iterative 3D surface modelling from a sparse set of matched feature points

We present an iterative algorithm to reconstruct a 3D object surface from a sparse set of matched feature points on the input stereo images of the object. The initial matches are sparse and do not have to be accurate. The reconstructed 3D surface is represented in terms of triangular polygons whose vertices are initially the 3D points corresponding to these matched feature points. In order to render photorealistic images of the surface, these feature points are iteratively updated. New feature points are added into the feature point set as well as the depth estimates of the feature points are refined. Experimental results showing the updated correspondences, reconstructed surfaces and virtual views rendered from new directions are presented.
Shape regularized active contour using iterative global search and local optimization

Recently, nonlinear shape models have been shown to improve the robustness and flexibility of segmentation. In this paper, we propose shape regularized active contour (ShRAC) that incorporates existing nonlinear shape models into the classical active contour approach. ShRAC uses a discrete representation of the contour to allow efficient combinatorial search. The search for optimal contour is performed by coarse-to-fine algorithm that iterates between combinatorial search and gradient-based local optimization. First, multi-solution dynamic programming (MSDP) is used to generate initial candidates by minimizing only the image energy. In the second step, a combination of image energy and shape energy determined by a given prior shape model is minimized for the initial candidates using a local optimization method and the best one is selected. To have diverse initial candidates, we employ a clustered solution pruning procedure in the MSDP search space. Finally, local shape regularization is used to feed shape constraints back into the new MSDP search space of the next iteration. Our search strategy combines the advantages of global combinatorial search and local optimization, and has shown excellent robustness to local minima caused by distracting suboptimal segmentations. Experimental results on segmentation of different anatomical structures using ShRAC are provided.
High capacity data embedding in the wavelet domain

A novel solution to the problem of data embedding in images is proposed in this paper The proposed algorithm allows high capacity data embedding and is robust to JPEG image compression data is embedded in the wavelet domain which provides better perceptual masking compared to the DCT domain set partitioning in hierarchical trees (SPIHT) is used to control the distortion (in the sense of PSNR) in the embedded host. Unlike other data embedding algorithms available in literature, the proposed algorithm provides control over the BER of the embedded data by appropriately choosing the JPEG quantization matrix. Preliminary results of an implementation of the algorithm are also presented.
A testbed for precise registration, natural occlusion and interaction in an augmented environment using a head-mounted projective display (HMPD)

A head-mounted projective display (HMPD) consists of a pair of miniature projection lenses, beam splitters and displays mounted on the helmet and retro-reflective sheeting materials placed strategically in the environment. This has recently been proposed as an alternative to existing 3D visualization devices. In this paper, we first briefly review HMPD technology, including its featured capabilities and the recent development in both display implementations and applications. Then the implementation of a testbed for playing a "Go" game with a remote opponent in a 3D augmented environment is described. The testbed not only demonstrates the capabilities of virtual-real augmentation and registration, the natural occlusion of virtual objects by real ones, interaction with augmented environments and networking collaboration, but also embodies part of our long-term objective to develop a collaborative framework in 3D augmented environments. Through the testbed, major calibration issues, such as accommodation/convergence considerations and determination of viewing transformations, are studied and discussed in detail. Both calibration methods and results are included, which are applicable to other applications. Finally, experimental results of the testbed implementation are presented.
A Vision System for Monitoring Intermodal Freight Trains

We describe the design and implementation of a vision based Intermodal Train Monitoring System (ITMS) for extracting various features like length of gaps in an intermodal (IM) train which can later be used for higher level inferences. An intermodal train is a freight train consisting of two basic types of loads - containers and trailers. Our system first captures the video of an IM train, and applies image processing and machine learning techniques developed in this work to identify the various types of loads as containers and trailers. The whole process relies on a sequence of following tasks -robust background subtraction in each frame of the video, estimation of train velocity, creation of mosaic of the whole train from the video and classification of train loads into containers and trailers. Finally, the length of gaps between the loads of the IM train is estimated and is used to analyze the aerodynamic efficiency of the loading pattern of the train, which is a critical aspect of freight trains. This paper focusses on the machine vision aspect of the whole system
Automatic Recognition of Offensive Team Formation in American Football Plays

Compared to security surveillance and military applications, where automated action analysis is prevalent, the sports domain is extremely under-served. Most existing software packages for sports video analysis require manual annotation of important events in the video. American football is the most popular sport in the United States, however most game analysis is still done manually. Line of scrimmage and offensive team formation recognition are two statistics that must be tagged by American Football coaches when watching and evaluating past play video clips, a process which takes many man hours per week. These two statistics are also the building blocks for more high-level analysis such as play strategy inference and automatic statistic generation. In this paper, we propose a novel framework where given an American football play clip, we automatically identify the video frame in which the offensive team lines in formation (formation frame), the line of scrimmage for that play, and the type of player formation the offensive team takes on. The proposed framework achieves 95% accuracy in detecting the formation frame, 98% accuracy in detecting the line of scrimmage, and up to 67% accuracy in classifying the offensive team's formation. To validate our framework, we compiled a large dataset comprising more than 800 play-clips of standard and high definition resolution from real-world football games. This dataset will be made publicly available for future comparison.
Facial expression decomposition

In this paper, we propose a novel approach for facial expression decomposition - higher-order singular value decomposition (HOSVD), a natural generalization of matrix SVD. We learn the expression subspace and person subspace from a corpus of images showing seven basic facial expressions, rather than resort to expert-coded facial expression parameters. We propose a simultaneous face and facial expression recognition algorithm, which can classify the given image into one of the seven basic facial expression categories, and then other facial expressions of the new person can be synthesized using the learned expression subspace model. The contributions of this work lie mainly in two aspects. First, we propose a new HOSVD based approach to model the mapping between persons and expressions, used for facial expression synthesis for a new person. Second, we realize simultaneous face and facial expression recognition as a result of facial expression decomposition. Experimental results are presented that illustrate the capability of the person subspace and expression subspace in both synthesis and recognition tasks. As a quantitative measure of the quality of synthesis, we propose using gradient minimum square error (GMSE) which measures the gradient difference between the original and synthesized images.
Recovering frontal-pose image from a single profile image

In appearance based face recognition, lip reading, etc., eigen face and eigen lip are used for recognition. The pose changes of the human head in a video sequence often cause errors in the eigen space comparison stage, because the frontal-pose assumption has been violated. We propose a new method to compensate the pose changes by exploiting the general symmetry of human face. From the imaging geometry we show that a frontal pose can be recovered from only one profile view. The resulting pose compensation method has the following advantages: (1) it only requires one profile image; (2) it does not need any 3D model; (3) it does not need accurate feature detection. Experimental results in the context of lip images are given to show the effectiveness of our method.
Scalable predictive coding as the Wyner-Ziv problem

An alternative to scalable predictive coding of first order Gauss-Markov processes is proposed in this paper. It is shown that conventional scalable predictive coding is inherently suboptimal. An alternative to scalable predictive coding, which achieves the rate-distortion performance of predictive coding for first-order Gauss-Markov processes is then proposed. The proposed approach is posed as a variant of the well-known Wyner-Ziv (1976) problem. By using coset codes with nested lattices, the present paper proves that the proposed approach achieves the predictive coding bound asymptotically at all scales while simultaneously providing the functionality of scalable coding.
A fast scheme for altering resolution in the compressed domain

Given a video frame or image in terms of its 8/spl times/8 block-DCT coefficients we wish to obtain a downsized (lower resolution) or upsized (higher resolution) version of this frame also in terms of 8/spl times/8 block -DCT coefficients. We propose an algorithm for achieving this directly in the compressed domain which is computationally much faster, produces visually sharper images and gives significant improvements in PSNR (typically 4 dB better compared to other compressed domain methods based on bilinear interpolation). The downsampling and upsampling schemes combined together preserve all the low-frequency DCT coefficients of the original signal. This implies tremendous savings for coding the difference between the original (unsampled image) and its prediction (the upsampled image). This is desirable for many applications based on scalable encoding of video.
Path planning using the Newtonian potential

Newtonian potential function is used to represent polygonal objects and obstacles. The closed-form expression of this potential field and other gradient-related quantities are derived. Such results not only eliminate the problems associated with the discretization of the object and obstacles in evaluating the risk of collision, but also make the search for the optimal object configurations efficient. The object skeleton, a shape description of the moving object, is introduced to guide the moving object through narrow regions while the search is done at different stages. The free space can then be divided by the narrow regions where the path planning takes place-a very simple free space decomposition scheme. Successful global strategies are developed to connect the local plans into a safe and smooth global path.<<ETX>>
Object segmentation using graph cuts based active contours

In this paper we present a graph cuts based active contours (GCBAC) approach to object segmentation problems. Our method is a combination of active contours and the optimization tool of graph cuts and differs fundamentally from traditional active contours in that it uses graph cuts to iteratively deform the contour. Consequently, it has the following advantages. (1) It has the ability to jump over local minima and provide a more global result. (2) Graph cuts guarantee continuity and lead to smooth contours free of self-crossing and uneven spacing problems. Therefore, the internal force, which is commonly used in traditional energy functions to control the smoothness, is no longer needed, and hence the number of parameters is greatly reduced. (3). Our approach easily extends to the segmentation of three and higher dimensional objects. In addition, the algorithm is suitable for interactive correction and is shown to always converge. Experimental results and analyses are provided.
Segmentation, matching and estimation of structure and motion of textured piecewise planar surfaces

The authors present an algorithm which segments and matches the regions and then estimates non-iteratively 3D motion and structure of a moving piecewise planar textured surface from two perspective views. The algorithm has two major steps. The first step is coarse. Here, the local planar nature of the surface is used to obtain polynomial expressions for image plane displacements of features. Using regions as moving features, the image is segmented using Hough transform such that the regions in each segment have the same polynomial coefficients. The values of these coefficients and region properties (e.g., area) are then used to identify region correspondences. In the second step, for each planar surface, the region correspondences are used to compute the corresponding motion parameters and surface orientation in closed form. The second step uses a finer model of motion than the first step. Experimental results are presented for three image pairs.<<ETX>>
Predictive multiple description coding using coset codes

Multiple Description (MD) coding of predictively coded sources is of practical interest in several multimedia applications such as redundant storage of video/audio data, and real-time video/audio telephony. A key problem associated with predictive MD coding is the occurrence of predictive mismatch. In the present paper, we pose the problem of predictive MD coding as a variant of the Wyner-Ziv decoder side-in formation problem. We propose an approach based on the use of coset codes for predictive MD coding, which avoids predictive mismatch without requiring restrictive channel assumptions or high latency. We specifically consider two-channel predictive MD coding of a first-order Gauss-Markov process. Results indicate that the proposed approach significantly out-performs alternative approaches in terms of rate-distortion performance.
Coding the displaced frame difference for video compression

Popular techniques employed to code the displaced frame difference (DFD) treat it no differently from an ordinary image for coding purposes. Since the DFD is generated by the process of motion compensation, such methods do not fully exploit the underlying redundancies. This paper proposes a DFD coding method which exploits such redundancies while incurring negligible information overhead. The key idea is to predict locations of high DFD concentration which occupy small portions of the image and use this predicted information (which is also available to the decoder without additional information transmission) to improve the quality of the decoded image. Two key features of the proposed approach are its compatibility with any transform based DFD coding scheme and negligible information overhead. Tests with a fully functional video coder show the efficacy of the proposed approach.
Superpixel Hierarchy

Superpixel segmentation has been one of the most important tasks in computer vision. In practice, an object can be represented by a number of segments at finer levels with consistent details or included in a surrounding region at coarser levels. Thus, a superpixel segmentation hierarchy is of great importance for applications that require different levels of image details. However, there is no method that can generate all scales of superpixels accurately in real time. In this paper, we propose the superhierarchy algorithm which is able to generate multi-scale superpixels as accurately as the state-of-the-art methods but with one to two orders of magnitude speed-up. The proposed algorithm can be directly integrated with recent efficient edge detectors to significantly outperform the state-of-the-art methods in terms of segmentation accuracy. Quantitative and qualitative evaluations on a number of applications demonstrate that the proposed algorithm is accurate and efficient in generating a hierarchy of superpixels.
Robust visual tracking via multi-task sparse learning

In this paper, we formulate object tracking in a particle filter framework as a multi-task sparse learning problem, which we denote as Multi-Task Tracking (MTT). Since we model particles as linear combinations of dictionary templates that are updated dynamically, learning the representation of each particle is considered a single task in MTT. By employing popular sparsity-inducing ℓp, q mixed norms (p ∈ {2, ∞} and q = 1), we regularize the representation problem to enforce joint sparsity and learn the particle representations together. As compared to previous methods that handle particles independently, our results demonstrate that mining the interdependencies between particles improves tracking performance and overall computational complexity. Interestingly, we show that the popular L1 tracker [15] is a special case of our MTT formulation (denoted as the L11 tracker) when p = q = 1. The learning problem can be efficiently solved using an Accelerated Proximal Gradient (APG) method that yields a sequence of closed form updates. As such, MTT is computationally attractive. We test our proposed approach on challenging sequences involving heavy occlusion, drastic illumination changes, and large pose variations. Experimental results show that MTT methods consistently outperform state-of-the-art trackers.
Motion and structure from point correspondences with error estimation: planar surfaces

The determination of motion and structure of a planar scene from monocular image sequences, is studied. A new and simpler linear algorithm, that gives closed-form solutions for motion and structure parameters using point correspondences between two images, assuming that the coplanar points undergo a rigid motion in 3-D space, is presented. A series of analytical results is established. From two perspective views of a planar scene which is undergoing a rigid motion, there are generally two (normalized) interpretations for motion parameters and the positions of the object plane. These two interpretations, one vertical and the other illusive, are both valid in the sense that they render the same pair of images. The authors identify all the special cases in which the number of interpretations is not two, and derive necessary and sufficient geometrical conditions for those special cases to occur. The approach to error estimation is based on the first-order perturbation. The estimated errors provide quantitative assessment for the accuracy of the solutions. They also indicate degenerate or nearly degenerate configurations in the presence of noise.
Multiscale image segmentation by integrated edge and region detection

This paper is concerned with the detection of low-level structure in images. It describes an algorithm for image segmentation at multiple scales. The detected regions are homogeneous and surrounded by closed edge contours. Previous approaches to multiscale segmentation represent an image at different scales using a scale-space. However, structure is only represented implicitly in this representation, structures at coarser scales are inherently smoothed, and the problem of structure extraction is unaddressed. This paper argues that the issues of scale selection and structure detection cannot be treated separately. A new concept of scale is presented that represents image structures at different scales, and not the image itself. This scale is integrated into a nonlinear transform which makes structure explicit in the transformed domain. Structures that are stable (locally invariant) to changes in scale are identified as being perceptually relevant. The transform can be viewed as collecting spatially distributed evidence for edges and regions, and making it available at contour locations, thereby facilitating integrated detection of edges and regions without restrictive models of geometry or homogeneity. In this sense, it performs Gestalt analysis. All scale parameters of the transform are automatically determined, and the structure of any arbitrary geometry can be identified without any smoothing, even at coarse scales.
Face recognition using feature extraction based on independent component analysis

We have explored a new method of feature extraction for face recognition. It is based on independent component analysis (ICA), but unlike original ICA, one of the unsupervised learning methods, it is developed to be well suited for classification problems by utilizing class information. By using ICA in solving supervised classification problems, we can obtain new features which are made as independent from each other as possible and which convey the class information faithfully. We have applied this method on Yale face databases and AT and T face databases and compared the performance with those of conventional methods such as principal component analysis (PCA), Fisher's linear discriminant (FLD), and so on. The experimental results show that for both databases the proposed method outperforms the others.