nils-herrmann/clean_scirepeval huggingface dataset:
A state-free causal video encoding paradigm

A commonly encountered problem in the communication of predictively encoded video is that of predictive mismatch or drift. The problem of predictive mismatch manifests itself in numerous communication scenarios, including on-demand streaming, real-time streaming and multicast streaming. This paper proposes a state-free video encoding architecture that alleviates this problem. The main benefit of state-free encoding is that there is no need for the encoder and the decoder to maintain the same state, or equivalently, predict using the same predictor. This facilitates robust communication of causally encoded media. The proposed approach is based on the Wyner-Ziv theorem in information theory. Consequently, it leverages the superior performance of coset codes for the Wyner-Ziv problem for predictive coding. A video codec, with state-free functionality, based on the H.26L encoding standard is proposed. The performance of the proposed codec is within 1-2.5 dB of the H.26L encoder.
Modeling Dynamic Textures Using Subspace Mixtures

In this paper, we aim at modeling video sequences that exhibit temporal appearance variation. The dynamic texture model proposed in S. Soatto et al., (2001) is effective to model simple dynamic scenes. However, because of its oversimplified appearance model and under-constrained dynamics model, the visual quality of its synthesized video sequences is often not satisfactory. This leads to our new model. We parameterize the nonlinear image manifold using mixtures of probabilistic principal component analyzers. We then align coefficients from different mixture components in a global coordinate system, and model the image dynamics in the global coordinate using an autoregressive process. The experimental results show that our method is capable of capturing complex temporal appearance variation and offers improved synthesis results over previous works
Feature guided pixel matching and segmentation in motion image sequences

The problem of feature correspondences and trajectory finding for a long image sequence has received considerable attention. Most attempts involve small numbers of features and make restrictive assumptions such as the visibility of features in all the frames. In this paper, a coarse-to-fine algorithm is described to obtain pixel trajectories through the sequence and to segment into subsets corresponding to distinctly moving objects. The algorithm uses a coarse scale point feature detector to form a 3-D dot pattern in the spatio-temporal space. The trajectories are extracted as 3-D curves formed by the points using perceptual grouping. Increasingly dense correspondences are obtained iteratively from the sparse feature trajectories. At the finest level, matching of all pixels is done using intensity correlation and the finest boundaries of the moving objects are obtained.
Reconstructing a dynamic surface from video sequences using graph cuts in 4D space-time

This paper is concerned with the problem of dynamically reconstructing the 3D surface of an object undergoing non-rigid motion. The problem is cast as reconstructing a continuous optimal 3D hyper-surface in 4D space-time from a set of calibrated video sequences. The imaging model of video cameras in 4D space-time is derived and a photo-inconsistency cost function is defined for a hyper-surface in the 4D space-time. We use a 4D node-cut algorithm to find a global minimum of the cost function and obtain the corresponding optimal hyper-surface. Experimental results show that the proposed algorithm is effective in recovering continuously changing shapes and exhibits good noise resistance.
Interference detection among objects for operator assistance in virtual cooperative workspace

We propose an efficient method for detecting interference and potential collisions among objects to facilitate cooperative work in a virtual space. The method consists of two main stages: 1) the coarse stage, an approximate test is performed to identify interfering objects in the entire workspace using octree representation of object shapes; and 2) the fine stage, polyhedral representation of object shapes is used to more accurately identify any object parts causing interference and collisions. For this purpose specific pairs of faces belonging to any of the interfering objects found in the first stage are tested, and detailed computation is performed on a reduced amount of data. The experimental results show a better efficiency for the proposed method especially when there are complicated objects in the environment, in comparison with the conventional collision detection method.<<ETX>>
Rank-R approximation of tensors using image-as-matrix representation

We present a novel multilinear algebra based approach for reduced dimensionality representation of image ensembles. We treat an image as a matrix, instead of a vector as in traditional dimensionality reduction techniques like PCA, and higher-dimensional data as a tensor. This helps exploit spatio-temporal redundancies with less information loss than image-as-vector methods. The challenges lie in the computational and memory requirements for large ensembles. Currently, there exists a rank-R approximation algorithm which, although applicable to any number of dimensions, is efficient for only low-rank approximations. For larger dimensionality reductions, the memory and time costs of this algorithm become prohibitive. We propose a novel algorithm, for rank-R approximations of third-order tensors, which is efficient for arbitrary R but for the important special case of 2D image ensembles, e.g. video. Both of these algorithms reduce redundancies present in all dimensions. Rank-R tensor approximation yields the most compact data representation among all known image-as-matrix methods. We evaluated the performance of our algorithm vs. other approaches on a number of datasets with the following two main results. First, for a fixed compression ratio, the proposed algorithm yields the best representation of image ensembles visually as well as in the least squares sense. Second, proposed representation gives the best performance for object classification.
Robust segmentation of freight containers in train monitoring videos

This paper is about a vision-based system that automatically monitors intermodal freight trains for the quality of how the loads (containers) are placed along the train. An accurate and robust algorithm to segment the foreground of containers in videos of the moving train is indispensable for this purpose. Given a video of a moving train consisting of containers of different types, this paper presents a method exploiting the information in both frequency and spatial domains to segment these containers. This method can accurately segment all types of containers under a variety of background conditions, e.g illumination variations and moving clouds, in the train videos shot by a fixed camera. The accuracy and robustness of the proposed method are substantiated through a large number of experiments on real data of train videos.
On the Equivalence of Moving Entrance Pupil and Radial Distortion for Camera Calibration

Radial distortion for ordinary (non-fisheye) camera lenses has traditionally been modeled as an infinite series function of radial location of an image pixel from the image center. While there has been enough empirical evidence to show that such a model is accurate and sufficient for radial distortion calibration, there has not been much analysis on the geometric/physical understanding of radial distortion from a camera calibration perspective. In this paper, we show using a thick-lens imaging model, that the variation of entrance pupil location as a function of incident image ray angle is directly responsible for radial distortion in captured images. Thus, unlike as proposed in the current state-of-the-art in camera calibration, radial distortion and entrance pupil movement are equivalent and need not be modeled together. By modeling only entrance pupil motion instead of radial distortion, we achieve two main benefits, first, we obtain comparable if not better pixel re-projection error than traditional methods, second, and more importantly, we directly back-project a radially distorted image pixel along the true image ray which formed it. Using a thick-lens setting, we show that such a back-projection is more accurate than the two-step method of undistorting an image pixel and then back-projecting it. We have applied this calibration method to the problem of generative depth-from-focus using focal stack to get accurate depth estimates.
Dot Pattern Processing Using Voronoi Neighborhoods

A sound notion of the neighborhood of a point is essential for analyzing dot patterns. The past work in this direction has concentrated on identifying pairs of points that are neighbors. Examples of such methods include those based on a fixed radius, k-nearest neighbors, minimal spanning tree, relative neighborhood graph, and the Gabriel graph. This correspondence considers the use of the region enclosed by a point's Voronoi polygon as its neighborhood. It is argued that the Voronoi polygons possess intuitively appealing characteristics, as would be expected from the neighborhood of a point. Geometrical characteristics of the Voronoi neighborhood are used as features in dot pattern processing. Procedures for segmentation, matching, and perceptual border extraction using the Voronoi neighborhood are outlined. Extensions of the Voronoi definition to other domains are discussed.
Region-based video coding using a multiscale image segmentation

This paper proposes a novel region-based video coding technique using a multiscale image segmentation method thus obtaining better quality at the same bit rate. In most of the previous region-based video coding techniques, occlusion caused degradation in terms of both the PSNR and perceptual video quality. We propose a new motion estimation and compensation algorithm which solves occlusion related problems effectively. The proposed motion estimation and compensation is a two stage procedure. The first stage uses a coarse motion model while the second stage uses a dense motion model. The coarse motion model generates region level motion vectors which are then fine tuned by the dense motion model which produces pixel level motion vectors. A fusion of these concepts leads to a gain of 2/spl sim/3 dB in PSNR over the block-based algorithm for a variety of test sequences using a fully functional video coder.
A syntax for image understanding

We consider one of the most basic questions in computer vision, that of finding a low-level image representation that could be used to seed diverse, subsequent computations of image understanding. Can we define a relatively general purpose image representation which would serve as the syntax for diverse needs of image understanding? What makes good image syntax? How do we evaluate it? We pose a series of such questions and evolve a set of answers to them, which in turn help evolve an image representation. For concreteness, we first perform this exercise in the specific context of the following problem.
High dynamic range panoramic imaging

Most imaging sensors have a limited dynamic range and hence can satisfactorily respond to only a part of illumination levels present in a scene. This is particularly disadvantageous for omnidirectional and panoramic cameras since larger fields of view have larger brightness ranges. We propose a simple modification to existing high resolution omnidirectional/panoramic cameras in which the process of increasing the dynamic range is coupled with the process of increasing the field of view. This is achieved by placing a graded transparency (mask) in front of the sensor which allows every scene point to be imaged under multiple exposure settings as the camera pans, a process anyway required to capture large fields of view at high resolution. The sequence of images are then mosaiced to construct a high resolution, high dynamic range panoramic/omnidirectional image. Our method is robust to alignment errors between the mask and the sensor grid and does not require the mask to be placed on the sensing surface. We have designed a panoramic camera with the proposed modifications and have discussed various theoretical and practical issues encountered in obtaining a robust design. We show with an example of high resolution, high dynamic range panoramic image obtained from the camera we designed.
Isotropic error diffusion halftoning

The inherently causal nature of conventional single-pass error diffusion (ED) halftoning results in asymmetric diffusion of error. This results in the introduction of directional artifacts in the output halftone. In this paper we propose a novel two-pass algorithm which achieves symmetric error diffusion by using a zero-phase signal transfer function. We determine conditions under which isotropic diffusion of error and noise suppression are achieved. Experimental results demonstrate that the proposed algorithm breaks up worms and randomizes their direction, thus making the output halftone more visually appealing as compared to conventional error diffusion.
Discrete multi-dimensional linear transforms over arbitrarily shaped supports

In order to apply a multi-dimensional linear transform, over an arbitrarily shaped support, the usual practice is to fill out the support to a hypercube by zero padding. This does not however yield a satisfactory definition for transforms in two or more dimensions. The problem that we tackle is: how do we redefine the transform over an arbitrary shaped region suited to a given application? We present a novel iterative approach to define any multi-dimensional linear transform over an arbitrary shape given that we know its definition over a hypercube. The proposed solution is (1) extensible to all possible shapes of support (whether connected or unconnected) (2) adaptable to the needs of a particular application. We also present results for the Fourier transform, for a specific adaptation of the general definition of the transform which is suitable for compression or segmentation algorithms.
Matching images under unstable segmentations

Region based features are getting popular due to their higher descriptive power relative to other features. However, real world images exhibit changes in image segments capturing the same scene part taken at different time, under different lighting conditions, from different viewpoints, etc. Segmentation algorithms reflect these changes, and thus segmentations exhibit poor repeatability. In this paper we address the problem of matching regions of similar objects under unstable segmentations. Merging and splitting of regions makes it difficult to find such correspondences using one-to-one matching algorithms. We present partial region matching as a solution to this problem. We assume that the high contrast, dominant contours of an object are fairly repeatable, and use them to compute partial matching cost (PMC) between regions. Region correspondences are obtained under region adjacency constraints encoded by region adjacency graph (RAG). We integrate PMC in a many-to-one label assignment framework for matching RAGs, and solve it using belief propagation. We show that our algorithm can match images of similar objects across unstable image segmentations. We also compare the performance of our algorithm with that of the standard one-to-one matching algorithm on three motion sequences. We conclude that our partial region matching approach is robust under segmentation irrepeatabilities.
Easy calibration of a head-mounted projective display for augmented reality systems

Augmented reality (AR) superimposes computer-generated virtual images on the real world to allow users exploring both virtual and real worlds simultaneously. For a successful augmented reality application, an accurate registration of a virtual object with its physical counterpart has to be achieved, which requires precise knowledge of the projection information of the viewing device. The paper proposes a fast and easy off-line calibration strategy based on well-established camera calibration methods. Our method does not need exhausting effort on the collection of world-to-image correspondence data. All the correspondence data are sampled with an image based method and they are able to achieve sub-pixel accuracy. The method is applicable for all AR systems based on optical see-through head-mounted display (HMD), though we took a head-mounted projective display (HMPD) as the example. We first review the calibration requirements for an augmented reality system and the existing calibration methods. Then a new view projection model for optical see through HMD is addressed in detail, and proposed calibration method and experimental result are presented. Finally, the evaluation experiments and error analysis are also included. The evaluation results show that our calibration method is fairly accurate and consistent.
Multiview panoramic cameras using mirror pyramids

A mirror pyramid consists of a set of planar mirror faces arranged around an axis of symmetry and inclined to form a pyramid. By strategically positioning a number of conventional cameras around a mirror pyramid, the viewpoints of the cameras' mirror images can be located at a single point within the pyramid and their optical axes pointed in different directions to effectively form a virtual camera with a panoramic field of view. Mirror pyramid-based panoramic cameras have a number of attractive properties, including single-viewpoint imaging, high resolution, and video rate capture. It is also possible to place multiple viewpoints within a single mirror pyramid, yielding compact designs for simultaneous multiview panoramic video rate imaging.  (1996) first described some of the basic ideas behind mirror pyramid cameras. We analyze the general class of multiview panoramic cameras, provide a method for designing these cameras, and present experimental results using a prototype we have developed to validate single-pyramid multiview designs. We first give a description of mirror pyramid cameras, including the imaging geometry, and investigate the relationship between the placement of viewpoints within the pyramid and the cameras' field of view (FOV), using simulations to illustrate the concepts. A method for maximizing sensor utilization in a mirror pyramid-based multiview panoramic camera is also presented. Images acquired using the experimental prototype for two viewpoints are shown.
Action Recognition Using Discriminative Structured Trajectory Groups

In this paper, we develop a novel framework for action recognition in videos. The framework is based on automatically learning the discriminative trajectory groups that are relevant to an action. Different from previous approaches, our method does not require complex computation for graph matching or complex latent models to localize the parts. We model a video as a structured bag of trajectory groups with latent class variables. We model action recognition problem in a weakly supervised setting and learn discriminative trajectory groups by employing multiple instance learning (MIL) based Support Vector Machine (SVM) using pre-computed kernels. The kernels depend on the spatio-temporal relationship between the extracted trajectory groups and their associated features. We demonstrate both quantitatively and qualitatively that the classification performance of our proposed method is superior to baselines and several state-of-the-art approaches on three challenging standard benchmark datasets.
A reconfigurable and hierarchical parallel processing architecture: performance results for stereo vision

A multiprocessor architecture called NETRA is discussed. It is highly reconfigurable and does not involve the use of complex interconnection schemes. The topology of this multiprocessor is recursively defined and is therefore easily scalable from small to large systems. It has a tree-type hierarchical architecture featuring leaf nodes that consist of a cluster of small but powerful processors connected via a programmable crossbar with selective broadcast capability. The architecture is simulated on a hypercube multiprocessor and the performance of one processor cluster is evaluated for stereo-vision tasks. The particular stereo algorithm selected for implementation requires computation of the two-dimensional fast Fourier transform (2-D FFT), template matching, histogram computation, and least-squares surface fitting. Static partitioning of data is used for the data-independent tasks such as 2-D FFT and dynamic scheduling, and load balancing is used for the data-dependent tasks of feature matching and disambiguation.<<ETX>>
Extraction and Analysis of Multiple Periodic Motions in Video Sequences

The analysis of periodic or repetitive motions is useful in many applications, such as the recognition and classification of human and animal activities. Existing methods for the analysis of periodic motions first extract motion trajectories using spatial information and then determine if they are periodic. These approaches are mostly based on feature matching or spatial correlation, which are often infeasible, unreliable, or computationally demanding. In this paper, we present a new approach, based on the time- frequency analysis of the video sequence as a whole. Multiple periodic trajectories are extracted and their periods are estimated simultaneously. The objects that are moving in a periodic manner are extracted using the spatial domain information. Experiments with synthetic and real sequences display the capabilities of this approach.
A fast scheme for downsampling and upsampling in the DCT domain

Given a video frame or image in terms of its 8×8 block-DCT coefficients, we wish to obtain a downsized or upsized (by factor of two) version of this frame also in terms of 8×8 block-DCT coefficients. We propose an algorithm for achieving this directly in the DCT domain which is computationally much faster, produces visually sharper images and gives significant improvements in PSNR (typically 4 dB better), compared to other compressed domain methods based on bilinear interpolation. The downsampling and upsampling schemes combined together preserve all the low-frequency DCT coefficients of the original image. This implies tremendous savings for coding the difference between the original (unsampled image) and its prediction (the upsampled image). This is desirable for many applications based on scalable encoding of video.
Rotation-invariant texture recognition by rotation compensation and wavelet analysis

A new rotation-invariant wavelet-based texture recognition scheme is proposed. In the previous rotation-invariant approaches, the focus is on adapting the wavelet transform or filter to rotated texture. In our approach, instead, we estimate the rotation of the texture with respect to some reference orientation, and then rotate the texture image back to the reference orientation before applying the wavelet analysis to extract features. With such rotation compensation, even very simple features (such as 1-level DWT and the subband energy) can be effective in achieving high classification accuracy as we demonstrate through our experiments.
Fusion of frequency and spatial domain information for motion analysis

This work presents an approach to the analysis of multiple motions in video, which combines frequency and spatial domain information in a new manner. The tasks of interest are finding the number of moving objects, velocity estimation, object tracking, and motion segmentation. We propose a novel, hybrid approach, which avoids problems of spatial domain methods, like sensitivity to illumination changes or great computational cost, but also uses spatial information for precise object localization. Experiments with synthetic and real sequences, show both the possibilities and limitations of this approach.