nils-herrmann/clean_scirepeval huggingface dataset:
Compression of MPEG-4 facial animation parameters for transmission of talking heads

The emerging MPEG-4 standard supports the transmission and composition of facial animation with natural video. The new standard will include a facial animation parameter (FAP) set that is defined based on the study of minimal facial actions and is closely related to muscle actions. The FAP set enables model-based representation of natural or synthetic talking-head sequences and allows intelligible visual reproduction of facial expressions, emotions, and speech pronunciations at the receiver. This paper addresses the data-compression issue of talking heads and presents three methods for bit-rate reduction of FAPs. Compression efficiency is achieved by way of transform coding, principal component analysis, and FAP interpolation. These methods are independent of each other in nature and thus can be applied in combination to lower the bit-rate demand of FAPs, making possible the transmission of multiple talking heads over band-limited channels. The basic methods described here have been adopted into the MPEG-4 Visual Committee Draft and are readily applicable to other articulation data such as body animation parameters. The efficacy of the methods is demonstrated by both subjective and objective results.
Shadow removal from natural images

In this paper, we present a system for estimating the shadow field from a single natural image. The user of our system is provided with a broad brush to roughly specify the shadow boundary. As the user finishes drawing a stroke, the system starts to estimate the shadow field around the stroke and generates pretty accurate result even if the underlying surface is highly textured. Since the strokes provided by the user may be sparse, an optimization scheme is proposed to propagate the estimated shadow field to the entire image. With this scheme, the amount of user interactions required for the system is reduced. The shadow field estimated by our system can be used to seamlessly remove the shadow from the image. It is also useful for other shadow editing tasks. Experimental results on a variety of photos are provided to show the effectiveness of the proposed system.
Transformation of focus profiles for digital autofocus

A focus profile model is used to estimate the in-focus lens position so that a sharp image or video can be captured. In this paper, we propose an approach that transforms a wide variety of focus profiles generated by different focus measurement techniques into an analytic form and thereby converts the search of in-focus lens position into a mathematically tractable problem, resulting in a wide effective range and efficient image acquisition. An important feature of the approach is that it does not require any knowledge of the focus measurement technique used in a camera. Experimental results, including demo videos, are provided to demonstrate the effectiveness of the approach.
Reciprocal Focus Profile

A focus profile having a steeper peak is more resistant to image noise in the autofocus (AF) process of a digital camera. However, a focus profile of such shape normally has a flatter out-of-focus region on either side of the profile, resulting in a slow AF process due to the lack of clue about where the lens should move when the lens is in such regions. To address the problem, we provide a statistical analysis of the focus profile and show that a strictly monotonic transformation of the focus profile preserves the accuracy of the AF. On the basis of this analysis, we propose a new focus profile representation that transforms the focus profile to the reciprocal domain in which the reciprocal focus profile is modeled by a polynomial function. This transformation makes the AF mathematically tractable and boosts the search speed. Experimental results are shown to demonstrate the advantage of the proposed representation .
Improving video coding quality by perceptual rate-distortion optimization

The goal of this work is to seek a feasible direction of video coding that can provide a significant quality improvement over H.264. In light of the well-known findings that the distortion metric for video quality has a profound impact on video coding performance and that traditional metrics such as mean square error are poorly correlated with human perception, we identify perceptual video coding, more specifically, perceptual-based rate-distortion optimization, as a sensible approach that has the potential to help drive the performance of video coding to a significantly higher quality level. This technology assessment is supported by experiments with various video sequences, bit-rates, and encoding profiles. The results show that the perceptual-based RDO can indeed bring significant quality improvement for H.264.
Wavelet-based image coding: comparison of MPEG-4 and JPEG-2000

Two major emerging standards, MPEG-4 and JPEG-2000, have recently adopted wavelet-based coding as the basic framework for coding still images and textures. Although both MPEG-4 and JPEG-2000 are based on wavelet coding, they are different. This paper presents an overview of these two emerging standards and highlights the differences in terms of coding scheme, performance, scope, functionalities, and applications. Both qualitative and quantitative comparisons are provided.
Faster R-CNN Features for Instance Search

Image representations derived from pre-trained Convolutional Neural Networks (CNNs) have become the new state of the art in computer vision tasks such as instance retrieval. This work explores the suitability for instance retrieval of image-and region-wise representations pooled from an object detection CNN such as Faster R-CNN. We take advantage of the object proposals learned by a Region Proposal Network (RPN) and their associated CNN features to build an instance search pipeline composed of a first filtering stage followed by a spatial reranking. We further investigate the suitability of Faster R-CNN features when the network is fine-tuned for the same objects one wants to retrieve. We assess the performance of our proposed system with the Oxford Buildings 5k, Paris Buildings 6k and a subset of TRECVid Instance Search 2013, achieving competitive results.
Multiscale Combinatorial Grouping

We propose a unified approach for bottom-up hierarchical image segmentation and object candidate generation for recognition, called Multiscale Combinatorial Grouping (MCG). For this purpose, we first develop a fast normalized cuts algorithm. We then propose a high-performance hierarchical segmenter that makes effective use of multiscale information. Finally, we propose a grouping strategy that combines our multiscale regions into highly-accurate object candidates by exploring efficiently their combinatorial space. We conduct extensive experiments on both the BSDS500 and on the PASCAL 2012 segmentation datasets, showing that MCG produces state-of-the-art contours, hierarchical regions and object candidates.
Prediction of image partitions using Fourier descriptors: application to segmentation-based coding schemes

This paper presents a prediction technique for partition sequences. It uses a region-by-region approach that consists of four steps: region parameterization, region prediction, region ordering, and partition creation. The time evolution of each region is divided into two types: regular motion and shape deformation. Both types of evolution are parameterized by means of the Fourier descriptors and they are separately predicted in the Fourier domain. The final predicted partition is built from the ordered combination of the predicted regions, using morphological tools. With this prediction technique, two different applications are addressed in the context of segmentation-based coding approaches. Noncausal partition prediction is applied to partition interpolation, and examples using complete partitions are presented. In turn, causal partition prediction is applied to partition extrapolation for coding purposes, and examples using complete partitions as well as sequences of binary images--shape information in video object planes (VOPs)--are presented.
Object matching based on partition information

This paper presents a new technique for object matching that exploits the information about transitions in the image obtained by means of a segmentation approach. Object matching is performed by comparing a transformed version of an object shape model (reference contour) to the contours in the image partition. The comparison is based on a distance map that measures the Euclidean distance between any point in the image to the partition contours. Examples using parametric and non-parametric reference contours are provided to assess the quality of the proposed technique.
Upper-bound assessment of the spatial accuracy of hierarchical region-based image representations

Hierarchical region-based image representations are versatile tools for segmentation, filtering, object detection, etc. The evaluation of their spatial accuracy has been usually performed assessing the final result of an algorithm based on this representation. Given its wide applicability, however, a direct supervised assessment, independent of any application, would be desirable and fair. A brute-force assessment of all the partitions represented in the hierarchical structure would be a correct approach, but as we prove formally, it is computationally unfeasible. This paper presents an efficient algorithm to find the upper-bound performance of the representation and we show that the previous approximations in the literature can fail at finding this bound.
Unsupervised image segmentation controlled by morphological contrast extraction

A novel approach for unsupervised image segmentation is described. This approach makes use of a Gaussian pyramid as multiresolution decomposition to analyze images. Compound random fields are used to model images at each resolution. The hierarchical image model is formed by a Strauss process in the lower level and a set of white Gaussian random fields in the upper level. This basic image model is adapted to the data present at each resolution. Segmentations at coarse resolutions are used to guide segmentations at finest resolutions. Segmentation quality is controlled, at each level, by means of morphological tools. The control procedure is based on the residue between the original image and a morphological center transform. This procedure checks whether the current segmentation contains all the relevant regions in the scene. If not, the algorithm introduces seeds into the segmented image in order to detect the new regions.<<ETX>>
Improving spatial codification in semantic segmentation

This paper explores novel approaches for improving the spatial codification for the pooling of local descriptors to solve the semantic segmentation problem. We propose to partition the image into three regions for each object to be described: Figure, Border and Ground. This partition aims at minimizing the influence of the image context on the object description and vice versa by introducing an intermediate zone around the object contour. Furthermore, we also propose a richer visual descriptor of the object by applying a Spatial Pyramid over the Figure region. Two novel Spatial Pyramid configurations are explored: Cartesian-based and crown-based Spatial Pyramids. We test these approaches with state-of-the-art techniques and show that they improve the Figure-Ground based pooling in the Pascal VOC 2011 and 2012 semantic segmentation challenges.
Performance of region-based matching techniques to compute the ocean surface motion

The study of the ocean circulation is the central core of all dynamical oceanography. The routine derivation of sea surface temperature or infrared brightness temperatures has been used to estimate the surface circulation by calculating the motion of the thermal features (coastal upwellings, filaments and eddies) in successive images. To that respect, a number of authors have developed different methodologies to recover the motion field, but the most straightforward methods match patterns (points, borders or regions) in all possible subwindows of one image with those in the next image. The maximization of the normalized cross- correlation coefficient, known as the Maximum Cross-Correlation (MCC) technique, is the most popular region-based matching metric applied to compute ocean circulation. In this paper a careful analysis of different region matching techniques has been conducted and the performance achieved for each approach is presented. The assessment methodology uses a database of synthetic sequences, real sequences and in-situ speed measurements. After the qualitative and quantitative analysis, we can conclude that the best performance is achieved by ZSAD, ZSSD, NZSSD and NCC metrics. These metrics achieve, when applied to synthetic sequences, mean angular errors around 30deg and magnitude errors around 30% for the worst case. In general the flow field recovered by the 4 previous metrics, perfectly models the motion of the structures in real sequences. Finally, results obtained with comparison with ground-truth data suggest an underestimation in the computed velocity between 35%-45% but with a higher angular accuracy, achieving global errors around 30deg-50deg. To conclude, it is important to emphasize that the prevalent MCC method provides acceptable results but with more errors when compared with the four previous metrics over the three databases.
Performance evaluation of probability density estimators for unsupervised information theoretical region merging

Information theoretical region merging techniques have been shown to provide a state-of-the-art unified solution for natural and texture image segmentation. Here, we study how the segmentation results can be further improved by a more accurate estimation of the statistical model characterizing the regions. Concretely, we explore four density estimators that can be used for pdf or joint pdf estimation. The first three are based on different quantization strategies: a general uniform quantization, an MDL-based uniform quantization, and a data-dependent partitioning and estimation. The fourth strategy is based on a computationally efficient kernel-based estimator (averaged shifted histogram). Finally, all estimators are objectively evaluated using a database with available ground truth partitions.
Precise classification of coastal benthic habitats using high resolution Worldview-2 imagery

The analysis of the seafloor in shallow waters using remote sensing imagery at very high spatial resolution is a very challenging topic due to the minimum signal level received; the presence of noisy contributions from the atmosphere, solar reflection, foam, turbidity and water column; and the limited spectral information available for the classification at such depths that impedes, for example, the extraction of vegetation indices. In this complex scenario we have developed a mapping methodology that involves the precise application of pre-processing techniques and the use of efficient classification algorithms. In particular, after a detailed assessment, support vector machines achieved the best performance using the appropriate kernel and parameters. Two natural areas located at the Canary Islands (Spain) have been selected for their benthic habitats richness and specially for their preservation of highly protected seagrass regions.
Automatic satellite image georeferencing using a contour-matching approach

Multitemporal and multisatellite studies or comparisons between satellite data and local ground measurements require nowadays precise and automatic geometric correction of satellite images. This paper presents a fully automatic geometric correction system capable of georeferencing satellite images with high accuracy. An orbital prediction model, which provides initial earth locations, is combined with the proposed automatic contour-matching technique. This combination allows correcting the low-frequency error component, mainly due to timing and orbital model errors, as well as the high-frequency error component, due to variations in the spacecraft's attitude. The approach aims at exploiting the maximum reliable information in the image to guide the matching algorithm. The contour-matching process has three main steps: 1) estimation of the gradient energy map (edges) and detection of the cloudless (reliable) areas; 2) initialization of the contours positions; 3) estimation of the transformation parameters (affine model) using a contour optimization approach. Three different robust and automatic algorithms are proposed for optimization, and their main features are discussed. Finally, the performance of the three proposed algorithms is assessed using a new error estimation technique applied to Advanced Very High Resolution Radiometer (AVHRR), Sea-viewing Wide Field of view Sensor (SeaWiFS), and multisensor AVHRR-SeaWiFS imagery.
Region-Based Hierarchical Representation for Object Detection

This paper proposes a region-based hierarchical image representation to be used in the context of object detection. The work uses a bottom-up segmentation algorithm and, specifically, the Binary Partition Tree implementation. The different problems that arise when creating and working with a hierarchical region-based representation are analyzed; namely, (i) the creation of the initial partition, that is, the merging and stopping criteria as well we the way to ensure a sufficient accuracy in the scene representation, (ii) the merging criteria used to build the hierarchical representation and (iii) the use of the hierarchical representation for object detection. For steps (i) and (ii), the proposed approach is assessed and compared with previous ones over a subset of the Corel database using well-established partition-based metrics. For (iii), the usefulness of the final region-based representation for object detection is exemplified in different scenarios.
Motion Estimation Techniques to Automatically Track Oceanographic Thermal Structures in Multisensor Image Sequences

The ocean involves a complex set of physical, chemical, biological, and geological processes, interacting with each other to influence our climate and natural environment. One of the most important disciplines in oceanography is the study of the ocean dynamics and, particularly, the ocean surface circulation. One can estimate this by the automated tracking of thermal infrared features in pairs of sequential satellite imagery. In this context, an extensive analysis of different motion estimation techniques has been performed by employing databases with synthetic sequences, real sequences, and in situ measurements. Four region- based metrics and two differential algorithms are proposed to estimate surface currents in multitemporal and multisensor AVHRR and MODIS image sequences. Once the appropriate motion estimation techniques have been selected, a new methodology to compute ocean currents is proposed. It includes a preliminary step to precisely segment the oceanographic structures and a second step to track its motion using additional modules (initialization, preprocessing, and postprocessing) to increase effectiveness. The information provided by the segmentation step reduces computing times, initializes the motion estimation parameters with appropriate values, and increases the overall performance. In summary, this two-stage approach combines image processing tools and physical oceanography knowledge to achieve a good ocean current estimation.
Region-Based Particle Filter for Video Object Segmentation

We present a video object segmentation approach that extends the particle filter to a region-based image representation. Image partition is considered part of the particle filter measurement, which enriches the available information and leads to a re-formulation of the particle filter. The prediction step uses a co-clustering between the previous image object partition and a partition of the current one, which allows us to tackle the evolution of non-rigid structures. Particles are defined as unions of regions in the current image partition and their propagation is computed through a single co-clustering. The proposed technique is assessed on the SegTrack dataset, leading to satisfactory perceptual results and obtaining very competitive pixel error rates compared with the state-of-the-art methods.
Face Recognition using Groups of Images in Smart Room Scenarios

In this paper, we present a technique for face recognition in smart environments. The technique takes advantage of the continuous monitoring of the scenario and combines the information of several images to perform the recognition. Appearance based face recognition techniques are used given that unobtrusive systems are required in this type of applications and that the scenario does not ensure high quality images. Models for the users are created on-the-fly and subsequent face images of the same individual are gathered into groups. Images within a group are jointly compared to the models for identification and verification purposes. Reliable face images are used to update the user's models. The proposed technique is assessed with the BANCA and CHIL databases.
Methodology for the estimation of ocean surface currents using region matching and differential algorithms

The ocean involves a complex set of physical, chemical, biological and geological processes, interacting each other to influence our climate and natural environment. One of the most important disciplines in oceanography is the study of the ocean dynamics. Particularly, ocean surface circulation can be recovered by the automated tracking of thermal features (coastal upwellings, filaments and eddies) in pairs of sequential satellite imagery. This paper presents a new methodology for the automatic estimation of ocean surface currents using AVHRR and MODIS imagery. The technique is based on a two-step process: precise structure detection followed by the structure tracking. The precise detection methodology is, as well, composed by two modules. The first one with the goal to obtain a coarse structure segmentation and the second achieving the maximum detail of the structure. The tracking methodology estimates motion fields using region matching and differential techniques with additional modules to guarantee the optimum performance. Twelve matching metrics and four differential algorithms were implemented and tested. Information from the previous structure segmentation stage is of fundamental importance to initialize the optimum motion estimation parameters and to generate the corresponding study area for each sequence. The proposed methodology, combining the segmentation and tracking steps, has been extensively tested and it has demonstrated an adequate performance in the estimation of flow fields in multitemporal and multisensorial sequences of the NW African coast.
Human face segmentation and tracking using connected operators and partition projection

A new technique for segmenting and tracking human faces in video sequences is presented. The algorithm uses a connected operator to extract the connected component that more likely belongs to a face. Such a connected operator is implemented by means of a binary partition tree. A set of connected regions (a node in the tree) is selected maximizing an estimation of the likelihood of being part of a face. Faces are tracked through the sequence based on the partition projection approach. A face and a non-face core component are obtained in the current image by projecting the previous partition. The technique has been successfully assessed using several test sequences from the MPEG-4 database (raw format) as well as from the MPEG-7 database (MPEG-1 format).