nils-herrmann/clean_scirepeval huggingface dataset:
Multiresolution Hierarchy Co-Clustering for Semantic Segmentation in Sequences with Small Variations

This paper presents a co-clustering technique that, given a collection of images and their hierarchies, clusters nodes from these hierarchies to obtain a coherent multiresolution representation of the image collection. We formalize the co-clustering as Quadratic Semi-Assignment Problem and solve it with a linear programming relaxation approach that makes effective use of information from hierarchies. Initially, we address the problem of generating an optimal, coherent partition per image and, afterwards, we extend this method to a multiresolution framework. Finally, we particularize this framework to an iterative multiresolution video segmentation algorithm in sequences with small variations. We evaluate the algorithm on the Video Occlusion/Object Boundary Detection Dataset, showing that it produces state-of-the-art results in these scenarios.
Automatic tool for the precise detection of upwelling and filaments in remote sensing imagery

The upward movement of cool and nutrient-rich waters toward the surface leads to horizontal alterations in the distribution of the physical, chemical, and biological properties. Remote sensing is being extensively applied to detect such coastal upwellings; however, the enormous amount of data daily generated obliges to develop automatic detection and prediction tools. The problem of identifying oceanographic mesoscale structures has been studied using a variety of image processing techniques; however, the outstanding difficulties encountered in the traditional approaches are the presence of noise, the fact that gradients are weak, the strong morphological variation, and the absence of a valid analytical model for the structures. In this context, the proposed automatic upwelling extraction methodology overcomes the preceding detection inconveniences and achieves a highly accurate structure extraction. This automatic technique is based on a coarse-segmentation methodology followed by a fine-detail growing process. The complete system has been validated over a database of 378 multisensorial images of years 2000 to 2003, and it has been applied to the detection and feature extraction of coastal upwellings and filaments in three areas with different characteristics, such as the Canary Islands, Cape Ghir, and the Alboran Sea, using imagery from the Advanced Very High Resolution Radiometer 2 and 3 sensors, the Sea-viewing Wide Field-of-view Sensor, and the Moderate Resolution Imaging Spectroradiometer sensor, demonstrating its effectiveness and robustness in a wide variety of climate conditions.
A morphological approach for segmentation and tracking of human faces

A new technique for segmenting and tracking human faces in video sequences is presented. The technique relies on morphological tools such as using connected operators to extract the connected component that more likely belongs to a face, and partition projection to track this component through the sequence. A binary partition tree (BPT) is used to implement the connected operator. The BPT is constructed based on the chrominance criteria and its nodes are analyzed so that the selected node maximizes an estimation of the likelihood of being part of a face. The tracking is performed using a partition projection approach. Images are divided into face and non-face parts, which are tracked through the sequence. The technique has been successfully assessed using several test sequences from the MPEG-4 (raw format) and the MPEG-7 databases (MPEG-1 format).
Measures and Meta-Measures for the Supervised Evaluation of Image Segmentation

This paper tackles the supervised evaluation of image segmentation algorithms. First, it surveys and structures the measures used to compare the segmentation results with a ground truth database, and proposes a new measure: the precision-recall for objects and parts. To compare the goodness of these measures, it defines three quantitative meta-measures involving six state of the art segmentation methods. The meta-measures consist in assuming some plausible hypotheses about the results and assessing how well each measure reflects these hypotheses. As a conclusion, this paper proposes the precision-recall curves for boundaries and for objects-and-parts as the tool of choice for the supervised evaluation of image segmentation. We make the datasets and code of all the measures publicly available.
Bayesian Approach for Morphology-Based 2-D Human Motion Capture

This paper presents a novel technique for 2D human motion capture using a single non calibrated camera. The user's five extremities (head, hands and feet) are extracted, labeled and tracked after silhouette segmentation. As they are the minimal number of points that can be used in order to enable whole body gestural interaction, we henceforth refer to these features as crucial points. The crucial point candidates are defined as the local maxima of the geodesic distance with respect to the center of gravity of the actor region that lie on the silhouette boundary. In order to disambiguate the selected crucial points into head, left and right foot and left and right hand classes, we propose a Bayesian framework that combines a MAP approach weighted by a prior model and the intensities of the tracked crucial points. Due to its low computational complexity, the system can run at real-time paces on standard personal computers, with an average error rate range between 2% and 7% in realistic situations, depending on the context and segmentation quality.
Segmentation-based video coding system allowing the manipulation of objects

This paper presents a generic video coding algorithm allowing the content-based manipulation of objects. This manipulation is possible thanks to the definition of a spatiotemporal segmentation of the sequences. The coding strategy relies on a joint optimization in the rate-distortion sense of the partition definition and of the coding techniques to be used within each region. This optimization creates the link between the analysis and synthesis parts of the coder. The analysis defines the time evolution of the partition, as well as the elimination or the appearance of regions that are homogeneous either spatially or in motion. The coding of the texture as well as of the partition relies on region-based motion compensation techniques. The algorithm offers a good compromise between the ability to track and manipulate objects and the coding efficiency.
Accurate and automatic NOAA-AVHRR image navigation using a global contour matching approach

The problem of precise and automatic AVHRR image navigation is tractable in theory, but has proved to be somewhat difficult in practice. The authors' work has been motivated by the need for a fully automatic and operational navigation system capable of geo-referencing NOAA-AVHRR images with high accuracy and without operator supervision. The proposed method is based on the simultaneous use of an orbital model and a contour matching approach. This last process, relying on an affine transformation model, is used to correct the errors caused by inaccuracies in orbit modeling, nonzero value for the spacecraft's roll, pitch and yaw, errors due to inaccuracies in the satellite positioning and failures in the satellite internal clock. The automatic global contour matching process is summarized as follows: i) Estimation of the gradient energy map (edges) in the sensed image and detection of the cloudless (reliable) areas in this map. ii) Initialization of the affine model parameters by minimizing the Euclidean distance between the reference and sensed images objects. iii) Simultaneous optimization of all reference image contours on the sensed image by energy minimization in the domain of the global transformation parameters. The process is iterated in a hierarchical way, reducing the parameter searching space at each iteration. The proposed image navigation algorithm has proved to be capable of geo-referencing a satellite image within 1 pixel.
Morphological detection based on size and contrast criteria application to cells detection

This paper deals with a detection algorithm relying on size and contrast criteria. It is suitable for a large range of applications where a priori information about the size and the contrast of the objects to detect is available. The detection is performed in three separate steps: the first one is a preprocessing which removes unuseful information with a size criterion. The second one performs a feature extraction based on contrast. Finally, the last step is the decision itself. All these steps make use of morphological transformations because of their ability to deal with the criteria of interest and of their low computational cost. As an example, this algorithm is applied to the automatic detection of spermatozoa.
Automatic structures detection and spatial registration using multisensor satellite imagery

Mesoscale processes such as upwellings, eddies, or thermal fronts are very energetic and their knowledge is very important not only to study oceanic circulation but also areas of applications that include acoustic propagation anomalies, fisheries management and exploitation, coastal monitoring and offshore or ocean oil detection and exploitation. A variety of techniques and algorithms have been developed to detect such structures. The foremost difficulties encountered in the preceding approaches are the presence of noise, mainly due to clouds and other atmospheric phenomena. In this context, the proposed methodology, due to its region-based nature, overcomes the edge detection inconveniences and obtains the proper structure identification. Moreover and in order to perform an exhaustive analysis of the structure dynamics, it is necessary to compare image sequences. In this context, the use of spatial registration techniques is necessary to achieve that pixels in different images correspond to the same geographic region. An automatic contour based approach for high accuracy registration of multisensor and multitemporal remote sensing images is presented. It avoids the use of ground control points, while exploiting the maximum reliable information in both images. These automatic tools, that combine structures detection techniques and multitemporal and multisensoral registration, have been applied to AVHRR, SeaWiFS and MODIS images of the Canary Island and Alboran Sea areas and have demonstrated that it can be a fundamental tool to validate marine and coastal dynamic studies using remote sensing data.
Multispectral Cooperative Partition Sequence Fusion for Joint Classification and Hierarchical Segmentation

In this letter, a region-based fusion methodology is presented for joint classification and hierarchical segmentation of specific ground cover classes from high-spatial-resolution remote sensing images. Multispectral information is fused at the partition level using nonlinear techniques, which allows the different relevance of the various bands to be fully exploited. A hierarchical segmentation is performed for each individual band, and the ensuing segmentation results are fused in an iterative and cooperative way. At each iteration, a consensus partition is obtained based on information theory and is combined with a specific ground cover classification. Here, the proposed approach is applied to the extraction and segmentation of vegetation areas. The result is a hierarchy of partitions with the most relevant information of the vegetation areas at different levels of resolution. This system has been tested for vegetation analysis in high-spatial-resolution images from the QuickBird and GeoEye satellites.
Region-based representations of image and video: segmentation tools for multimedia services

This paper discusses region-based representations of image and video that are useful for multimedia services such as those supported by the MPEG-4 and MPEG-7 standards. Classical tools related to the generation of the region-based representations are discussed. After a description of the main processing steps and the corresponding choices in terms of feature spaces, decision spaces, and decision algorithms, the state of the art in segmentation is reviewed. Mainly tools useful in the context of the MPEG-4 and MPEG-7 standards are discussed. The review is structured around the strategies used by the algorithms (transition based or homogeneity based) and the decision spaces (spatial, spatio-temporal, and temporal). The second part of this paper proposes a partition tree representation of images and introduces a processing strategy that involves a similarity estimation step followed by a partition creation step. This strategy tries to find a compromise between what can be done in a systematic and universal way and what has to be application dependent. It is shown in particular how a single partition tree created with an extremely simple similarity feature can support a large number of segmentation applications: spatial segmentation, motion estimation, region-based coding, semantic object extraction, and region-based retrieval.
Object detection and segmentation on a hierarchical region-based image representation

In this paper we present a general framework for object detection and segmentation. Using a bottom-up unsupervised merging algorithm, a region-based hierarchy that represents the image at different resolution levels is created. Next, top-down, object class knowledge is used to select and combine regions from the hierarchy, in order to define the exact object shape. We illustrate the usefulness of the approach with four different object classes: sky, caption text, traffic signs and faces.
Classified vector quantization using fuzzy theory

Classified vector quantization (CVQ) is a technique for coding images that achieves good perceptual results while reducing the computational load of the process. A method that uses fuzzy theory to perform the classification involved in the CVQ technique is presented. To formulate the membership function of different categories, the concept of surface representation by triangular elements is introduced. This method yields a more natural classification scheme. The problem of determining the size of each subcodebook was also solved. The subcodebooks were built using fuzzy classifiers and the degree of accuracy of each codeword was tested by computing its fuzzy entropy. The codewords with larger fuzzy entropy were further split until an optimal point was reached.<<ETX>>
An automated multisensor satellite imagery registration technique based on the optimization of contour features

Spatial registration of multidate or multisensor images is required for many applications in remote sensing. Automatic image registration, which has been extensively studied in other areas of image processing, is still a complex problem in the framework of remote sensing. This work explores an alternative strategy for a fully automatic and operational registration system capable of registering multitemporal and multisensor remote sensing satellite images with high accuracy and avoiding the use of ground control points, exploiting the maximum reliable information in both images (coastlines not occluded by clouds). The automatic feature-based approach is summarized as follows: (i) reference image coastline extraction; (ii) sensed image gradient energy map estimation and (iii) contour matching, mapping function estimation and transformation of the sensed images. Several experimental results for single sensor imagery (AVHRR/3) and multisensor imagery (AVHRR/3-SeaWiFS-MODIS-ATSR) from different viewpoints and dates have verified the robustness and accuracy of the proposed automatic registration algorithm, demonstrating its capability of registering satellite images of coastal areas within one pixel.
General region merging approaches based on information theory statistical measures

This work presents a new statistical approach to region merging where regions are modeled as arbitrary discrete distributions, directly estimated from the pixel values. Under this framework, two region merging criteria are obtained from two different perspectives, leading to information theory statistical measures: the Kullback-Leibler divergence and the Bhattacharyya coefficient. The developed methods are size-dependent, which assures the size consistency of the partitions but reduces their size resolution. Thus, a size-independent extension of the previous methods, combined with a modified merging order, is also proposed. Additionally, an automatic criterion to select the most statistically significant partitions from the whole merging sequence is presented. Finally, all methods are evaluated and compared with other state-of-the-art region merging techniques.
Object-based evaluation of hierarchical region-based representations based on information theory statistical measures

This work presents an evaluation in terms of object representation of the hierarchical region-based representations created by a family of general statistical region merging algorithms. These merging techniques are based on different versions of information theory statistical measures; concretely, the Kullback-Leibler divergence and the Bhattacharyya coefficient. Additionally, a significance index can be defined together with these techniques to extract the most statistically meaningful partitions at different levels of resolution. The first part of this object-based analysis evaluates the ability of the hierarchy of partitions created by each method to represent the objects in the image. The second set of experiments compares the quality of the object representation into the most significant partitions extracted from the hierarchical representations in two different cases: considering that the object may be represented by a union of regions (object oversegmentation is not penalized), and considering only the best single region representing it (oversegmentation is penalized).
Binary Partition Trees for Object Detection

This paper discusses the use of binary partition trees (BPTs) for object detection. BPTs are hierarchical region-based representations of images. They define a reduced set of regions that covers the image support and that spans various levels of resolution. They are attractive for object detection as they tremendously reduce the search space. In this paper, several issues related to the use of BPT for object detection are studied. Concerning the tree construction, we analyze the compromise between computational complexity reduction and accuracy. This will lead us to define two parts in the BPT: one providing accuracy and one representing the search space for the object detection task. Then we analyze and objectively compare various similarity measures for the tree construction. We conclude that different similarity criteria should be used for the part providing accuracy in the BPT and for the part defining the search space and specific criteria are proposed for each case. Then we discuss the object detection strategy based on BPT. The notion of node extension is proposed and discussed. Finally, several object detection examples illustrating the generality of the approach and its efficiency are reported.
Partition coding using multi-grid chain code and motion compensation

In this paper, a lossy partition coding technique is presented which leads to decoded partitions with unnoticeable losses. It uses a hexagonal grid for contour representation and it is based on the concept of multi-grid chain code. This coding technique can be used in intra-frame mode or, in combination with partition prediction techniques, in interframe mode. In intra-frame mode, it leads to an average saving of 25% of the coding cost with respect to chain code techniques. The savings on inter-frame mode depend on the motion prediction approach. Results of coding binary shapes (concept of video object plane) as well as partitions with an arbitrary set of regions are presented.
Hierarchical segmentation using compound Gauss-Markov random fields

The authors discuss an original approach for segmenting still images. In this approach, the image is initially decomposed in several levels of different resolution. The decomposition that has been chosen is a Gaussian pyramid. At each level of the pyramid, the image is modeled by a compound Gauss-Markov random field and the segmentation is obtained by using a maximum a posteriori criterion. The segmentation is carried out first at the top level of the pyramid. Once a level (l) has been segmented, this segmentation is projected onto the following level below it (l-1). The process is iterated until the segmentation at the bottom level (0) is performed.<<ETX>>
Region Merging Techniques Using Information Theory Statistical Measures

The purpose of the current work is to propose, under a statistical framework, a family of unsupervised region merging techniques providing a set of the most relevant region-based explanations of an image at different levels of analysis. These techniques are characterized by general and nonparametric region models, with neither color nor texture homogeneity assumptions, and a set of innovative merging criteria, based on information theory statistical measures. The scale consistency of the partitions is assured through i) a size regularization term into the merging criteria and a classical merging order, or ii) using a novel scale-based merging order to avoid the region size homogeneity imposed by the use of a size regularization term. Moreover, a partition significance index is defined to automatically determine the subset of most representative partitions from the created hierarchy. Most significant automatically extracted partitions show the ability to represent the semantic content of the image from a human point of view. Finally, a complete and exhaustive evaluation of the proposed techniques is performed, using not only different databases for the two main addressed problems (object-oriented segmentation of generic images and texture image segmentation), but also specific evaluation features in each case: under- and oversegmentation error, and a large set of region-based, pixel-based and error consistency indicators, respectively. Results are promising, outperforming in most indicators both object-oriented and texture state-of-the-art segmentation techniques.
Trajectory Tree as an Object-Oriented Hierarchical Representation for Video

This paper presents the trajectory tree as a hierarchical region-based representation for video sequences. Motion, as well as spatial features from multiple frames are used to generate a set of temporal regions structured within a hierarchy of scale and motion coherency. The resulting representations offer a global description of the entire video sequence and enhance semantic analysis potential. A multiscale segmentation strategy is proposed whereby region-merging criteria of progressively greater complexity are used to define partition layers of increasing aptitude for object detection. A novel data structure, called the trajectory adjacency graph, is defined for the long-term analysis of partition sequences. Furthermore, mechanisms for assessing connectivity, verifying temporal continuity, and proposing merging operations based on color, affine, and translational motion homogeneity characteristics over the entire sequence are also introduced. Finally, as demonstrated through experimental results, the trajectory tree offers a concise yet versatile support for video object segmentation, description and retrieval tasks.