nils-herrmann/clean_scirepeval huggingface dataset:
Active stereo based surface reconstruction

The problem of surface reconstruction from stereo images for large scenes having large depth ranges is considered. The passive stereo paradigm is inadequate for this problem because of the need to aim cameras in different directions and to fixate on different objects. An active stereo approach, in which the scene is systematically scanned and image acquisition and surface reconstruction are integrated using a four-step process, is presented. First, a new fixation point is selected from among the nonfixated, low-resolution scene parts of current fixation. Then a reconfiguration of cameras is initiated for refixation. As reconfiguration progresses, the images of the new fixation point gradually deblur and the accuracy of the stereo estimate of the point improves. The improved stereo estimate is used to achieve accurate focus and vergence settings of the cameras for fixation. Finally, focus-based depth estimates are obtained at a grid near the fixation point whose density is determined by the local surface slope. These estimates are fused with those obtained from stereo using maximum likelihood, and are interpolated to the nongrid points.<<ETX>>
Adaptive polynomial modelling of the reflectance map for shape estimation from stereo and shading

This paper is concerned with estimation of the reflectance map and its use in surface shape estimation by a method that combines stereo and shading information. There are many advantages to the integrated approach. However, shape from shading algorithms are limited in their applicability by the assumption of idealized reflectance and lighting models involving many preset or hard to estimate parameters. In this paper, it is shown that the difficulties involved in estimation of the reflectance function and light source distribution can be eliminated through direct estimation of the reflectance map using adaptive polynomial models. The reflectance map estimate is used with a surface estimate provided by stereo in an integrated approach to surface shape estimation.<<ETX>>
Calibration of an HMPD-Based Augmented Reality System

In augmented reality (AR) applications, accurately registering a virtual object with its real counterpart is a challenging problem. The size, depth, and geometry, as well as the physical attributes of the virtual object, have to be rendered precisely relative to a physical reference. This paper presents a systematic calibration process to address the registration challenge in a custom-designed AR system, which is based upon recent head-mounted projective display (HMPD) technology. Following a concise review of the HMPD concept and our system configuration, we first present a computational model of the HMPD viewing system and requirements for the system calibration. Then, we describe, in detail, the calibration procedures to obtain estimates of unknown transformations, summarize the application of the estimates in a customized graphics rendering toolkit, and discuss the evaluation experiments and observations. Finally, the implementation of a testbed to demonstrate a successful registration is briefly described, and experimental results are presented
Robust multi-object tracking via cross-domain contextual information for sports video analysis

Multiple player tracking is one of the main building blocks needed in a sports video analysis system. In an uncalibrated camera setting, robust mutli-object tracking can be very difficult due to a number of reasons including the presence of noise, occlusion, fast camera motion, low-resolution image capture, varying viewpoints and illumination changes. To address the problem of multi-object tracking in sports videos, we go beyond the video frame domain and make use of information in a homography transform domain that is denoted the homography field domain. We propose a novel particle filter based tracking algorithm that uses both object appearance information (e.g. color and shape) in the image domain and cross-domain contextual information in the field domain to improve object tracking. In the field domain, the effect of fast camera motion is significantly alleviated since the underlying homography transform from each frame to the field domain can be accurately estimated. We use contextual trajectory information (intra-trajectory and inter-trajectory context) to further improve the prediction of object states within an particle filter framework. Here, intra-trajectory contextual information is based on history tracking results in the field domain, while inter-trajectory contextual information is extracted from a compiled trajectory dataset based on tracks computed from videos depicting the same sport. Experimental results on real world sports data show that our system is able to effectively and robustly track a variable number of targets regardless of background clutter, camera motion and frequent mutual occlusion between targets.
WYZE-PMD based multiple description video codec

The main hindrance to the development of efficient low-latency multiple description (MD) video coders are the problem of predictive mismatch. In this paper, we present a two-channel predictive MD video codec architecture based on the recently proposed WYZE-PMD framework. The proposed codec transmits coset information to curtail error-propagation caused by predictive mismatch, without requiring high latency or restrictive channel assumptions. MD scalar quantizers are used to generate multiple descriptions, low-density parity check (LDPC) codes are used to generate coset information, and the H.263 video coding standard is used for efficient motion compensation. The proposed codec is used to code descriptions of CIF video for communication over two erasure channels with independent failure probabilities. Results indicate that the proposed codec provides efficient, drift-free predictive MD coding.
Compact representation of multidimensional data using tensor rank-one decomposition

This paper presents a new approach for representing multidimensional data by a compact number of bases. We consider the multidimensional data as tensors instead of matrices or vectors, and propose a tensor rank-one decomposition (TROD) algorithm by decomposing Nth-order data into a collection of rank-1 tensors based on multilinear algebra. By applying this algorithm to image sequence compression, we obtain much higher quality images with the same compression ratio as principal component analysis (PCA). Experiments with gray-level and color video sequences are used to illustrate the validity of this approach.
SVM for edge-preserving filtering

In this paper, we propose a new method to construct an edge-preserving filter which has very similar response to the bilateral filter. The bilateral filter is a normalized convolution in which the weighting for each pixel is determined by the spatial distance from the center pixel and its relative difference in intensity range. The spatial and range weighting functions are typically Gaussian in the literature. In this paper, we cast the filtering problem as a vector-mapping approximation and solve it using a support vector machine (SVM). Each pixel will be represented as a feature vector comprising of the exponentiation of the pixel intensity, the corresponding spatial filtered response, and their products. The mapping function is learned via ∊-SVM regression using the feature vectors and the corresponding bilateral filtered values from the training image. The major computation involved is the computation of the spatial filtered responses of the exponentiation of the original image which is invariant to the filter size given that an IIR O(1) solution is available for the spatial filtering kernel. To our knowledge, this is the first learning-based O(1) bilateral filtering method. Unlike previous O(1) methods, our method is valid for both low and high range variance Gaussian and the computational complexity is independent of the range variance value. Our method is also the fastest O(1) bilateral filtering yet developed. Besides, our method allows varying range variance values, based on which we propose a new bilateral filtering method avoiding the over-smoothing or under-smoothing artifacts in traditional bilateral filter.
Integration of frequency and space for multiple motion estimation and shape-independent object segmentation

A video containing multiple objects undergoing independent translational and rotational motions is analyzed through a combination of spatial- and frequency-domain representations. The Fourier transform of the sequence is used to estimate the multiple translations and rotations in a computationally efficient manner, which is also robust to local inaccuracies and global illumination changes. A novel algorithm is presented for the simultaneous extraction of all objects undergoing translation and the background via a least squares technique that takes place entirely in the Fourier domain. Spatial information is combined with the frequency domain object extraction results, to further refine them. For the case of rotational or combined, rotational and translational motions, the moving objects are segmented using purely spatial information. We show that the combined analysis takes advantage of the strengths of both representations, by providing reliable and computationally efficient motion estimates and object segmentation. The proposed algorithm is shown to be robust to local noise and occlusion, because of its global nature. Experiments are performed on synthetic and real video sequences to demonstrate the capabilities of our approach.
Matching Point Features with Ordered Geometric, Rigidity, and Disparity Constraints

This correspondence presents a matching algorithm for obtaining feature point correspondences across images containing rigid objects undergoing different motions. First point features are detected using newly developed feature detectors. Then a variety of constraints are applied starting with simplest and following with more informed ones. First, an intensity-based matching algorithm is applied to the feature points to obtain unique point correspondences. This is followed by the application of a sequence of newly developed heuristic tests involving geometry, rigidity, and disparity. The geometric tests match two-dimensional geometrical relationships among the feature points, the rigidity test enforces the three dimensional rigidity of the object, and the disparity test ensures that no matched feature point in an image could be rematched with another feature, if reassigned another disparity value associated with another matched pair or an assumed match on the epipolar line. The computational complexity is proportional to the numbers of detected feature points in the two images. Experimental results with indoor and outdoor images are presented, which show that the algorithm yields only correct matches for scenes containing rigid objects.
Active surface reconstruction by integrating focus, vergence, stereo, and camera calibration

A method is described for estimating surfaces from stereo images. A single pair of stereo images can yield surface reconstruction for only a small volume of space. Since the scene may be large, images are obtained dynamically using different camera configurations through the control of focus and camera vergence. For smooth objects, this results in the reconstruction of surface patches that are contiguous and overlapping. These sequentially acquired surface maps are merged into a central. composite representation. During camera reconfiguration, unpredictable calibration errors may arise. Therefore, this method permits small changes in calibration parameter values to achieve agreement between adjacent reconstructed patches in the areas of overlap. To integrate the different sources of depth information, and to balance the effects they have on the resulting surface estimate, the surface reconstruction problem is formulated as one of optimization.<<ETX>>
A fast scheme for image size change in the compressed domain

Given a video frame in terms of its 8/spl times/8 block-DCT coefficients, we wish to obtain a downsized or upsized version of this frame also in terms of 8/spl times/8 block-DCT coefficients. The DCT being a linear unitary transform is distributive over matrix multiplication. This fact has been used for downsampling video frames in the DCT domain. However, this involves matrix multiplication with the DCT of the downsampling matrix. This multiplication can be costly enough to trade off any gains obtained by operating directly in the compressed domain. We propose an algorithm for downsampling and also upsampling in the compressed domain which is computationally much faster, produces visually sharper images, and gives significant improvements in PSNR (typically 4-dB better compared to bilinear interpolation). Specifically the downsampling method requires 1.25 multiplications and 1.25 additions per pixel of original image compared to 4.00 multiplications and 4.75 additions required by the method of Chang et al. (1995). Moreover, the downsampling and upsampling schemes combined together preserve all the low-frequency DCT coefficients of the original image. This implies tremendous savings for coding the difference between the original frame (unsampled image) and its prediction (the upsampled image). This is desirable for many applications based on scalable encoding of video. The method presented can also be used with transforms other than DCT, such as Hadamard or Fourier.
Automated Visual Inspection of Railroad Tracks

Thousands of miles of railroad track must be inspected twice weekly by a human inspector to maintain safety standards. A computer vision system, consisting of field-acquired video and subsequent analysis, could improve the efficiency of the current methods. Such a system is prototyped, and the following challenges are addressed: the detection, segmentation, and defect assessment of track components whose appearance vary across different tracks and the identification and inspection of special track areas such as track turnouts. An algorithm that utilizes the periodic manner in which track components repeat in an inspection video is developed. Spectral estimation and signal-processing methods are used to provide robust detection of the periodically occurring track components. Results are demonstrated on field-acquired images and video.
Object contour tracking using graph cuts based active contours

In this paper, we present an object contour tracking approach using graph cuts based active contours (GCBAC). Our proposed algorithm does not need any a priori global shape model, which makes it useful for tracking objects with deformable shapes and appearances. GCBAC are not sensitive to initial conditions and always converge to the optimal contour within the dilated neighborhood of itself. Given an initial boundary near the object in the first frame, GCBAC can iteratively converge to an optimal object boundary. In each frame thereafter, the resulting contour in the previous frame is taken as initialization and the algorithm consists of two steps. In the first step, GCBAC are applied to the difference between this frame and its previous one. The resulting contour is taken as initialization of the second step, which applies GCBAC to current frame directly. To evaluate the tracking performance, we apply the algorithm to several real world video sequences. Experimental results are provided.
Modelling Objects using Distribution and Topology of Multiscale Region Pairs

We propose a method for simultaneous detection, localization and segmentation of objects of a known category. We show that this is possible by using segments as features. To this end, we propose an object model in which the image is represented as a tree, that captures containment relationships among the segments. Using segments as features has the advantage that object detection and segmentation is done simultaneously, forgoing the need for a separate sophisticated model for object segmentation. A generative model of an object category is estimated in a supervised mode, in terms of the characteristics of its constituent regions, their relative locations, and their mutual containment. The novel aspect of this work lies in simplifying the description of the hierarchy in terms of constraints that apply to only pairs of nodes, instead of all nodes in the tree. We show that this indeed improves the speed of learning algorithm. Inference is done using graph cuts. We report the performance of the model on standard datasets.
Selecting objects with freehand sketches

We present in this paper the design of an interactive tool for selecting objects using simple freehand sketches. The objective is to extract object boundaries precisely while requiring little skill and time from the user. The tool proposed achieves this objective by integrating user input and image computation in a two-phase algorithm. In the first phase, the input sketch is used along with a coarse global segmentation of the image to derive an initial selection and a triangulation of the region around the boundary. The triangles are used to formulate subproblems of local finer-grained segmentation and selection. Each of the subproblems is processed independently in the second phase, where a linear approximation of the local boundary as well as a local, finer-grained segmentation are computed. The approximate boundary is then used with the local segmentation to compute a final selection, represented with an alpha channel to fully capture diffused object boundaries. Experimental results show that the tool allows very simple sketches to be used to select objects with complex boundaries. Therefore, the tool has immediate applications in graphics systems for image editing, manipulation, synthesis, retrieval and processing.
Single image super-resolution from transformed self-exemplars

Self-similarity based super-resolution (SR) algorithms are able to produce visually pleasing results without extensive training on external databases. Such algorithms exploit the statistical prior that patches in a natural image tend to recur within and across scales of the same image. However, the internal dictionary obtained from the given image may not always be sufficiently expressive to cover the textural appearance variations in the scene. In this paper, we extend self-similarity based SR to overcome this drawback. We expand the internal patch search space by allowing geometric variations. We do so by explicitly localizing planes in the scene and using the detected perspective geometry to guide the patch search process. We also incorporate additional affine transformations to accommodate local shape variations. We propose a compositional model to simultaneously handle both types of transformations. We extensively evaluate the performance in both urban and natural scenes. Even without using any external training databases, we achieve significantly superior results on urban scenes, while maintaining comparable performance on natural scenes as other state-of-the-art SR algorithms.
Robust video shot change detection

We present a novel improvement to existing schemes for abrupt shot change detection. Existing schemes declare a shot change whenever the frame to frame histogram difference (FFD) value is above a particular threshold. In such an approach, a high value for the threshold results in a small number of false alarms and a large number of missed detections while a low value for the threshold decreases the number of missed detections at the expense of increasing the false alarms. We attribute this situation to the fact that the FFD cannot be reliably used as the sole indicator for the presence of a shot change. In the proposed method a two-step shot detection strategy is used which selectively uses a likelihood ratio (computed directly from the frames and not from the histograms) to confirm the presence of a shot change. Such a two-step checking increases the probability of detection without increasing the probability of false alarm. The improvement proposed is simple and computationally cheap. Tests with a wide variety of video sequences prove the efficacy of the proposed approach.
Compression of lightfield rendered images using coset codes

Image-based rendering (IBR) and lightfield rendering (LFR) techniques aim to represent a 3D real-world environment by densely sampling it through a set of fixed viewpoint cameras. Remote digital walkthroughs of the 3D environment are facilitated by synthesizing novel viewpoints from the captured view-set. The large amount of data generated by the dense capture process makes the use of compression imperative for practical IBR/LFR systems. In the present paper, we consider the design of compression techniques for streaming of IBR data to remote viewers. The key constraints that a compression algorithm for IBR streaming is required to satisfy, are those of random access for interactivity, and precompression. We propose a compression algorithm based on the use of coset codes for this purpose. The proposed algorithm employs H.264 source compression in conjunction with LDPC coset codes to precompress the IBR data. Appropriate coset information is transmitted to the remote viewers to allow interactive view generation. Results indicate that the proposed compression algorithm provides good compression efficiency, while allowing client interactivity and server precompression.
A scheme for spatial scalability using nonscalable encoders

We describe a scheme that achieves spatially scalable coding of video by employing nonscalable video encoders (e.g., MPEG-2 main profile), along with a downsampler and an upsampler. The scheme is illustrated for the case of coding video at two resolutions. The enhancement layer is coded in two steps by first exploiting the spatial redundancy and then exploiting the temporal redundancy. Hence, the scheme has a separable implementation. Results are presented for five different sequences, coded for three different combinations of base and enhancement layer bit rates. When MPEG-2 main profile is used for the nonscalable encoders, the results obtained are comparable to the performance of MPEG-2 spatial scalability profile.
Camera center estimation

A fast camera calibration technique to estimate the center of perspective projection of an imaging camera has been described. The proposed technique requires a single image of two planar calibration charts arranged in a special manner. The special arrangement of the calibration charts simplifies the projection equations relating the 3-D scene coordinates to the 2-D image coordinates and they can be suitably combined to eliminate the unknown intrinsic parameters other than the desired center. We have analyzed the error in the center estimate due to various alignment errors in the experimental setup, and shown that the scheme is quite robust.
Estimation and segmentation of displacement field using multiple features

We present an approach for estimating and segmenting the displacement field (DF) between two frames. Our method is based on local affine (first-order) approximation of the displacement field which is derived under the assumption of locally rigid motion. Each distinct motion is represented in the image plane by a distinct set of values of affine coefficients. All sets of values supported by the feature locations in two frames are identified by exhaustive coarse-to-fine search. The integrated use of multiple features (points, regions and lines) increases the probability of finding well-supported sets. The sets of coefficients thus obtained are used to describe the DF. Two experimental results with real images are presented to demonstrate the feasibility of our approach.<<ETX>>