nils-herrmann/clean_scirepeval huggingface dataset:
Hierarchical forest attributes for multimodal tumor segmentation on FDG-PET/contrast-enhanced CT

Accurate tumor volume delineation is a crucial step for disease assessment, treatment planning and monitoring of several kinds of cancers. However, this process is complex due to variations in tumors properties. Recently, some methods have been proposed for taking advantage of the spatial and spectral information carried by coupled modalities (e.g., PET-CT, MRI-PET). Simultaneously, the development of attributebased approaches has contributed to improve PET image analysis. In this work, we aim at developing a coupled multimodal / attribute-based approach for image segmentation. Our proposal is to take advantage of hierarchical image models for determining relevant and specific attribute from each modality. These attributes then allow us to define a unique, semantic vectorial image. Sequentially, this image can be processed by a standard segmentation method, in our case a random-walker approach, for segmenting tumors based on their intrinsic attribute-based properties. Experimental results emphasize the relevance of computing region-based attributes from both modalities.
Domain adaptation for the extraction of complex urban patterns from multiresolution satellite images

The extraction of complex urban patterns from Very High Spatial Resolution (VHSR) images presents several challenges related to the complexity of the data. Based on the availability of images of a same scene at various resolutions (Medium to Very High Spatial resolutions), a hierarchical approach has been recently proposed to segment/classify objects of interest in a top-down fashion in order to determine patterns from VHSR images. To perform, this method requires the interactive definition of segmentation examples for each considered resolution image. In the context of large dataset processing, such interactive task becomes time consuming. To deal with this issue, we propose in this article, an extension of the domain adaptation paradigm enabling the transfer of the segmentation examples defined on a source dataset to automatically process a target one. Experiments performed on urban images provide satisfactory results which may be further used for operational needs.
An interactive approach for extraction of urban patterns from multisource images

The extraction of urban patterns from Very High Spatial Resolution (VHSR) images presents several challenges related to the size, the accuracy and the complexity of the considered data. In order to assist the end-user to efficiently carry out this task, a new approach is proposed for hierarchically extracting segments of interest from lower resolution data and finally determining urban patterns in these segments from VHSR ones. Based on an intuitive modus operandi, it allows the end-user to progressively segment images based on low level -spatial and radiometric- criteria. Experiments performed on urban images datasets provide encouraging results which may be further used for objects detection and classification purpose.
Unsupervised Quantification of Under- and Over-Segmentation for Object-Based Remote Sensing Image Analysis

Object-based image analysis (OBIA) has been widely adopted as a common paradigm to deal with very high-resolution remote sensing images. Nevertheless, OBIA methods strongly depend on the results of image segmentation. Many segmentation quality metrics have been proposed. Supervised metrics give accurate quality estimation but require a ground-truth segmentation as reference. Unsupervised metrics only make use of intrinsic image and segment properties; yet most of them strongly depend on the application and do not deal well with the variability of objects in remote sensing images. Furthermore, the few metrics developed in a remote sensing context mainly focus on global evaluation. In this paper, we propose a novel unsupervised metric, which evaluates local quality (per segment) by analyzing segment neighborhood, thus quantifying under- and over-segmentation given a certain homogeneity criterion. Additionally, we propose two variants of this metric, for estimating global quality of remote sensing image segmentation by the aggregation of local quality scores. Finally, we analyze the behavior of the proposed metrics and validate their applicability for finding segmentation results having good tradeoff between both kinds of errors.
A histogram semantic-based distance for multiresolution image classification

Image classification methods based on histogram analysis generally require to use relevant distances for histogram comparison. In this article, we propose a new distance devoted to compare histograms associated to semantic concepts linked by (dis)similarity correlations. This distance, whose computation relies on a hierarchical strategy, captures the multilevel semantic relations between these concepts. It also inherits from the low complexity properties of standard bin-to-bin distances, thus leading to fast and accurate results in the context of multiresolution image classification. Experiments performed on satellite images emphasize the relevance and usefulness of the proposed distance.
Interactive 3D brain vessel segmentation from an example

Segmentation of cerebral vascular networks from 3D angiographic data remains a challenge. Automation generally induces a high computational cost and possible errors, while interactive methods are hard to use due to the dimension and complexity of images. This article presents a compromise between both approaches, by using the concept of example-based segmentation. Segmentation examples of vascular structures are involved in a scheme relying on connected filtering, in order to obtain an interactive -but strongly assisted-segmentation method. This strategy, which uses component-trees in a non-standard fashion, leads to promising results, when applied on cerebral MR angiographic data.
Connected Filtering Based on Multivalued Component-Trees

In recent papers, a new notion of component-graph was introduced. It extends the classical notion of component-tree initially proposed in mathematical morphology to model the structure of gray-level images. Component-graphs can indeed model the structure of any-gray-level or multivalued-images. We now extend the anti extensive filtering scheme based on component-trees, to make it tractable in the framework of component-graphs. More precisely, we provide solutions for building a component-graph, reducing it based on selection criteria, and reconstructing a filtered image from a reduced component-graph. In this paper, we first consider the cases where component-graphs still have a tree structure; they are then called multivalued component-trees. The relevance and usefulness of such multivalued component-trees are illustrated by applicative examples on hierarchically classified remote sensing images.
Segmentation of head bones in 3-D CT images from an example

We propose a new generic framework for segmentation of 3D digital data, based on knowledge contained in a segmentation example of similar data. The integration of prior knowledge is made by registering the image to segment on the segmentation example. Since the registration step relies on binary segmented data, segmentation and registration are performed jointly in a coarse-to-fine way using a multiscale parametric representation of a threshold map and of a deformation field. The threshold map is required by the segmentation procedure which is also devoted to recover topological details. The benefit of such an approach is illustrated in the context of head bone segmentation in 3-D computed tomography (CT) images.
Topology Preserving Warping of 3-D Binary Images According to Continuous One-to-One Mappings

The estimation of one-to-one mappings is one of the most intensively studied topics in the research field of nonrigid registration. Although the computation of such mappings can be now accurately and efficiently performed, the solutions for using them in the context of binary image deformation is much less satisfactory. In particular, warping a binary image with such transformations may alter its discrete topological properties if common resampling strategies are considered. In order to deal with this issue, this paper proposes a method for warping such images according to continuous and bijective mappings while preserving their discrete topological properties (i.e., their homotopy type). Results obtained in the context of the atlas-based segmentation of complex anatomical structures highlight the advantages of the proposed approach.
Segmentation of the cortex in fetal MRI using a topological model

The study of in utero fetal MR images is essential for the diagnosis of abnormal brain development and the study of the maturation of the brain structures. Because of the particular properties of these images, only a few automated segmentation methods have been developed so far compared to the numerous ones existing for the adult brain anatomy. In this paper, we propose a two-step cortex segmentation technique including anatomical priors and a topological model. Experiments performed on in utero MR data and validation by comparison to experts segmentation emphasize the relevance of the method.
Evaluating the quality of binary partition trees based on uncertain semantic ground-truth for image segmentation

The binary partition tree (BPT) is a hierarchical data-structure that models the content of an image in a multiscale way. In particular, a cut of the BPT of an image provides a segmentation, as a partition of the image support. Actually, building a BPT allows for dramatically reducing the search space for segmentation purposes, based on intrinsic (image signal) and extrinsic (construction metric) information. A large literature has been devoted to the construction on such metrics, and the associated choice of criteria (spectral, spatial, geometric, etc.) for building relevant BPTs, in particular in the challenging context of remote sensing. But, surprisingly, there exists few works dedicated to evaluate the quality of BPTs, i.e. their ability to further provide a satisfactory segmentation. In this paper, we propose a framework for BPT quality evaluation, in a supervised paradigm. Indeed, we assume that ground-truth segments are provided by an expert, possibly with a semantic labelling and a given uncertainty. Then, we describe local evaluation metrics, BPT nodes / ground-truth segments fitting strategies, and global quality score computation considering semantic information, leading to a complete evaluation framework. This framework is illustrated in the context of BPT segmentation of multispectral satellite images.
Multivalued Component-Tree Filtering

We introduce the new notion of multivalued component-tree, that extends the classical component-tree initially devoted to grey-level images, in the mathematical morphology framework. We prove that multivalued component-trees can model images whose values are hierarchically organized. We also show that they can be efficiently built from standard component-tree construction algorithms, and involved in antiextensive filtering procedures. The relevance and usefulness of multivalued component-trees is illustrated by an applicative example on hierarchically classified remote sensing images.
Recycled linear classifiers for multiclass classification

Many machine learning applications employ a multiclass classification stage that uses multiple binary linear classifiers as building blocks. Among these, commonly used strategies such as one-vs-one classification can require learning a large number of hyperplanes, even when the number of classes to be discriminated among is modest. Further, when the data being classified is inherently high-dimensional, the storage and computational complexity associated with the application of multiple linear classifiers can ignite critical resource management issues. This work describes a novel multiclass classification method based on efficient use of a single “recycled” linear classifier (or ReLiC), which addresses these storage and implementation complexity issues. The proposed approach amounts to constraining the entire collection of hyperplanes to be circularly-shifted versions of each other, enabling classification procedures that may be implemented with efficient operations, such as circular convolution (which can be efficiently computed using transform domain techniques), and simple sampling/thresholding operations. We show that the optimization task associated with our proposed approach can be formulated as a quadratic program, and we introduce an efficient distributed procedure for its solution based on an alternating direction method of multipliers. Simulation results demonstrate that the performance of the proposed approach is comparable with the more complex, traditional multiclass linear classification strategies, suggesting the proposed approach is a viable alternative in large-scale data classification tasks.
Covariance Tracking using Model Update Based on Lie Algebra

We propose a simple and elegant algorithm to track nonrigid objects using a covariance based object description and a Lie algebra based update mechanism. We represent an object window as the covariance matrix of features, therefore we manage to capture the spatial and statistical properties as well as their correlation within the same representation. The covariance matrix enables efficient fusion of different types of features and modalities, and its dimensionality is small. We incorporated a model update algorithm using the Lie group structure of the positive definite matrices. The update mechanism effectively adapts to the undergoing object deformations and appearance changes. The covariance tracking method does not make any assumption on the measurement noise and the motion of the tracked objects, and provides the global optimal solution. We show that it is capable of accurately detecting the nonrigid, moving objects in non-stationary camera sequences while achieving a promising detection rate of 97.4 percent.
Learning an Invariant Hilbert Space for Domain Adaptation

This paper introduces a learning scheme to construct a Hilbert space (i.e., a vector space along its inner product) to address both unsupervised and semi-supervised domain adaptation problems. This is achieved by learning projections from each domain to a latent space along the Mahalanobis metric of the latent space to simultaneously minimizing a notion of domain variance while maximizing a measure of discriminatory power. In particular, we make use of the Riemannian optimization techniques to match statistical properties (e.g., first and second order statistics) between samples projected into the latent space from different domains. Upon availability of class labels, we further deem samples sharing the same label to form more compact clusters while pulling away samples coming from different classes. We extensively evaluate and contrast our proposal against state-of-the-art methods for the task of visual domain adaptation using both handcrafted and deep-net features. Our experiments show that even with a simple nearest neighbor classifier, the proposed method can outperform several state-of-the-art methods benefitting from more involved classification schemes.
Multi-camera calibration, object tracking and query generation

An automatic object tracking and video summarization method for multi-camera systems with a large number of non-overlapping field-of-view cameras is explained. In this framework, video sequences are stored for each object as opposed to storing a sequence for each camera. Object-based representation enables annotation of video segments, and extraction of content semantics for further analysis. We also present a novel solution to the inter-camera color calibration problem. The transitive model function enables effective compensation for lighting changes and radiometric distortions for large-scale systems. After initial calibration, objects are tracked at each camera by background subtraction and mean-shift analysis. The correspondence of objects between different cameras is established by using a Bayesian belief network. This framework empowers the user to get a concise response to queries such as "which locations did an object visit on Monday and what did it do there?".
Multiple dictionary learning for blocking artifacts reduction

We present a structured dictionary learning method to remove blocking artifacts without blurring edges or making any assumption over image gradients. Instead of a single overcomplete dictionary, we build multiple subspaces and impose sparsity on nonzero reconstruction coefficients when we project a given texture sample on each subspace separately. In case the texture matches to the dataset with which the subspace is trained, the corresponding response will be stronger and that subspace will be chosen to represent the texture. In this manner we compute the representations of all patches in the image and aggregate these to obtain the final image. Since the block artifacts are small in magnitude in comparison to actual image edges, aggregation efficiently removes the artifacts but keep the image gradients. We discuss the choices of subspace parameterizations and adaptation to given data. Our results on a large dataset of benchmark images demonstrate that the presented method provides superior results in terms of pixel-wise (PSNR) and perceptual (SSIM) measures.
Additive noise removal by sparse reconstruction on image affinity nets

This paper presents a new image denoising method based on sparse reconstruction by dictionary learning and collaborative filtering. First, we form an affinity net, in which a node represents an image patch, for the given image by clustering similar patches. For each cluster, we learn an undercomplete dictionary and represent clusters nodes by imposing sparsity inducing norm as a combination of few atoms. Depending on its affinity to other nodes, a single node could be present in multiple clusters making the clusters overlapping. This enables a single global estimation for each filtered pixel to be obtained by collaboratively aggregating its reconstructed patches in the corresponding clusters. Extensive experimental results demonstrate superior performance for additive noise removal without requiring the correct noise variance.
Instance-Aware Detailed Action Labeling in Videos

We address the problem of detailed sequence labeling of complex activities in videos, which aims to assign an action label to every frame. Previous work typically focus on predicting action class labels for each frame in a sequence without reasoning action instances. However, such category-level labeling is inefficient in encoding the global constraints at the action instance level and tends to produce inconsistent results. In this work we consider a fusion approach that exploits the synergy between action detection and sequence labeling for complex activities. To this end, we propose an instance-aware sequence labeling method that utilizes the cues from action instance detection. In particular, we design an LSTM-based fusion network that integrates framewise action labeling and action instance prediction to produce a final consistent labeling. To evaluate our method, we create a large-scale RGBD video dataset on gym activities for sequence labeling and action detection called GADD. The experimental results on GADD dataset show that our method outperforms all the state-of-the-art methods consistently in terms of labeling accuracy.
CrossTrack: Robust 3D tracking from two cross-sectional views

One of the challenges in radiotherapy of moving tumors is to determine the location of the tumor accurately. Existing solutions to the problem are either invasive or inaccurate. We introduce a non-invasive solution to the problem by tracking the tumor in 3D using bi-plane ultrasound image sequences. We present CrossTrack, a novel tracking algorithm in this framework. We pose the problem as recursive inference of 3D location and tumor boundary segmentation in the two ultrasound views using the tumor 3D model as a prior. For the segmentation task, a robust graph-based approach is deployed as follows: First, robust segmentation priors are obtained through the tumor 3D model. Second, a unified graph combining information across time and multiple views is constructed with a robust weighting function. For the tracking task, an effective mechanism for recovery from respiration-induced occlusion is introduced. Our experiments show the robustness of CrossTrack in handling challenging tumor shapes and disappearance scenarios, with sub-voxel accuracy, and almost 100% precision and recall, significantly outperforming baseline solutions.
A Comprehensive Look at Coding Techniques on Riemannian Manifolds

Core to many learning pipelines is visual recognition such as image and video classification. In such applications, having a compact yet rich and informative representation plays a pivotal role. An underlying assumption in traditional coding schemes [e.g., sparse coding (SC)] is that the data geometrically comply with the Euclidean space. In other words, the data are presented to the algorithm in vector form and Euclidean axioms are fulfilled. This is of course restrictive in machine learning, computer vision, and signal processing, as shown by a large number of recent studies. This paper takes a further step and provides a comprehensive mathematical framework to perform coding in curved and non-Euclidean spaces, i.e., Riemannian manifolds. To this end, we start by the simplest form of coding, namely, bag of words. Then, inspired by the success of vector of locally aggregated descriptors in addressing computer vision problems, we will introduce its Riemannian extensions. Finally, we study Riemannian form of SC, locality-constrained linear coding, and collaborative coding. Through rigorous tests, we demonstrate the superior performance of our Riemannian coding schemes against the state-of-the-art methods on several visual classification tasks, including head pose classification, video-based face recognition, and dynamic scene recognition.
Robust License Plate Detection Using Covariance Descriptor in a Neural Network Framework

We present a license plate detection algorithm that employs a novel image descriptor. Instead of using conventional gradient filters and intensity histograms, we compute a covariance matrix of low-level pixel-wise features within a given image window. Unlike the existing approaches, this matrix effectively captures both statistical and spatial properties within the window. We normalize the covariance matrix using local variance scores and restructure the unique coefficients into a feature vector form. Then, we feed these coefficients into a multi-layer neural network. Since no explicit similarity or distance computation is required in this framework, we are able to keep the computational load of the detection process low. To further accelerate the covariance matrix extraction process, we adapt an integral image based data propagation technique. Our extensive analysis shows that the detection process is robust against noise, illumination distortions, and rotation. In addition, the presented method does not require careful fine tuning of the decision boundaries.