nils-herrmann/clean_scirepeval huggingface dataset:
Discriminative Basis Selection Using Non-negative Matrix Factorization

Non-negative matrix factorization (NMF) has proven to be useful in image classification applications such as face recognition. We propose a novel discriminative basis selection method for classification of image categories based on the popular term frequency-inverse document frequency (TF-IDF) weight used in information retrieval. We extend the algorithm to incorporate color, and overcome the drawbacks of using unaligned images. Our method is able to choose visually significant bases which best discriminate between categories and thus prune the classification space to increase correct classifications. We apply our technique to ETH-80, a standard image classification benchmark dataset. Our results show that our algorithm outperforms other state-of-the-art techniques.
Context-Aware Hypergraph Modeling for Re-identification and Summarization

Tracking and re-identification in wide-area camera networks is a challenging problem due to non-overlapping visual fields, varying imaging conditions, and appearance changes. We consider the problem of person re-identification and tracking, and propose a novel clothing context-aware color extraction method that is robust to such changes. Annotated samples are used to learn color drift patterns in a non-parametric manner using the random forest distance (RFD) function. The color drift patterns are automatically transferred to associate objects across different views using a unified graph matching framework . A hypergraph representation is used to link related objects for search and re-identification. A diverse hypergraph ranking technique is proposed for person-focused network summarization . The proposed algorithm is validated on a wide-area camera network consisting of ten cameras on bike paths. Also, the proposed algorithm is compared with the state of the art person re-identification algorithms on the VIPeR dataset .
Joint source-channel coding scheme for image-in-image data hiding

We consider the problem of hiding images in images. In addition to the usual design constraints such as imperceptible host degradation and robustness in presence of variety of attacks, we impose the condition that the quality of the recovered signature image should be better if the attack is milder. We present a simple hybrid analog-digital hiding technique for this purpose. The signature image is compressed efficiently (using JPEG) into a sequence of bits, which is hidden using a previously proposed digital hiding scheme. The residual error between the original and compressed signature image is then hidden using an analog hiding scheme. The results show (perceptual as well as mean-square error) improvement as the attack becomes milder.
An automatic method to learn and transfer the photometric appearance of partially overlapping images

The first major contribution of this paper is a robust method to learn the photometric mapping between the overlapping portions of two registered images acquired either under different lighting conditions or different sensor modalities. Then, once such mapping is learnt, we demonstrate how it generalizes so that the photometric appearance can be transferred from one image to the other out of their overlapping area. This task is fundamental in several different contexts, such as image colorization, seamless mosaicking or change detection. After introducing the theory and discussing the algorithms, we will present several examples that confirm the efficacy of the proposed method in dealing with different types of images.
Edge flow: A framework of boundary detection and image segmentation

A novel boundary detection scheme based on "edge flow" is proposed in this paper. This scheme utilizes a predictive coding model to identify the direction of change in color and texture at each image location at a given scale, and constructs an edge flow vector. By iteratively propagating the edge flow, the boundaries can be detected at image locations which encounter two opposite directions of flow in the stable state. A user defined image scale is the only significant control parameter that is needed by the algorithm. The scheme facilitates integration of color and texture into a single framework for boundary detection.
Unified hypergraph for image ranking in a multimodal context

Image ranking has long been studied, yet it remains a very challenging problem. Increasingly, online images come with additional metadata such as user annotations and geographic coordinates. They provide rich complementary information. We propose to combine such multimodal information through a unified hypergraph to improve image retrieval performance. Hypergraphs allow for the simultaneously capture of higher order relationships among images using different modalities, e.g. visual content, user tags, and geolocations. Each image is represented as a vertex in the hypergraph. Each hyperedge is formed by a vertex and it's k-nearest neighbors. Three types of hyperedges exist in our unified hypergraph, which are in correspondence to the three different modalities. Image ranking is then formulated as a ranking problem on a unified hypergraph. The proposed method can easily be extended to incorporate additional modalities as long as a similarity function exists to compare the features. Experimental results on large datasets are promising.
Precise localization of key-points to identify local regions for robust data hiding

We propose a novel data hiding system where data is embedded in local non-overlapping regions in an image. To survive cropping, the encoder embeds the same data in multiple regions of fixed dimensions, while the decoder's challenge is to independently retrieve these regions. Salient feature points are computed on an image and the local regions are centered around them. To obtain non-overlapping regions, the points are pruned based on their corner strength and the size of the region. The decoder can retrieve the data only if it can precisely identify one or more of the same key-points. We present suitable key-point pruning methods such that even after considering a reduced number of key-points, the receiver is successful in identifying the same key-point locations as the encoder. We perform experimental comparison of various corner detectors and also study the performance of segmentation methods to obtain robust key-points.
Unsupervised Segmentation of Color-Texture Regions in Images and Video

A method for unsupervised segmentation of color-texture regions in images and video is presented. This method, which we refer to as JSEG, consists of two independent steps: color quantization and spatial segmentation. In the first step, colors in the image are quantized to several representative classes that can be used to differentiate regions in the image. The image pixels are then replaced by their corresponding color class labels, thus forming a class-map of the image. The focus of this work is on spatial segmentation, where a criterion for "good" segmentation using the class-map is proposed. Applying the criterion to local windows in the class-map results in the "J-image," in which high and low values correspond to possible boundaries and interiors of color-texture regions. A region growing method is then used to segment the image based on the multiscale J-images. A similar approach is applied to video sequences. An additional region tracking scheme is embedded into the region growing process to achieve consistent segmentation and tracking results, even for scenes with nonrigid object motion. Experiments show the robustness of the JSEG algorithm on real images and video.
Estimating and undoing rotation for print-scan resilient data hiding

This paper proposes a method to hide information into images that achieves robustness against printing and scanning with blind decoding. A significant contribution of this paper is a technique to estimate and undo rotation. The method is based on the fact that laser printers use an ordered digital halftoning algorithm for printing. Using the proposed hiding method, several hundred information bits can be embedded into 512/spl times/512 images with perfect recovery against the print-scan operation. Moreover, the hidden images also survive other attacks such as Gaussian or median filtering, scaling or aspect ratio change, heavy JPEG compression, and rows and/or columns removal.
Modeling and Detection of Geospatial Objects Using Texture Motifs

We propose the use of texture motifs, or characteristic spatially recurrent patterns, for modeling and detecting geospatial objects. A method is proposed for learning a texture-motif model from object examples and detecting objects based on the learned model. The model is learned in a two-layered framework: the first learns the constituent "texture elements" of the motif and, the second, the spatial distribution of the elements. In the experimental session, we demonstrate the model training and selection methodology for objects given a set of training examples. The utility of such models for detecting the presence or absence of geospatial objects in large aerial image datasets comprising tens of thousands of image tiles is then emphasized
Balloon motion estimation using two frames

A computational vision approach is presented for the estimation of 2D translation, rotation, and scale from two partially overlapping images. An illuminant direction estimation method is first used to obtain an initial estimate of camera rotation. A small number of feature points are then located based on a Gabor wavelet model for detecting local curvature discontinuities. An initial estimate of scale and translation is obtained by pairwise matching of the feature points detected from both frames. Finally, hierarchical feature matching is performed to obtain an accurate estimate of translation, rotation, and scale. The approach results in a fast method that produces excellent results. Experiments with synthetic and real images show that this algorithm yields accurate results when the scale between the image pair differs by up to 10%, the overlap between the two frames is as small as 35%, and the camera rotation between the two frames is significant.<<ETX>>
Detection of hiding in the least significant bit

In this paper, we apply the theory of hypothesis testing to the steganalysis, or detection of hidden data, in the least significant bit (LSB) of a host image. The hiding rate (if data is hidden) and host probability mass function (PMF) are unknown. Our main results are as follows. a) Two types of tests are derived: a universal (over choices of host PMF) method that has certain asymptotic optimality properties and methods that are based on knowledge or estimation of the host PMF and, hence, an appropriate likelihood ratio (LR). b) For a known host PMF, it is shown that the composite hypothesis testing problem corresponding to an unknown hiding rate reduces to a worst-case simple hypothesis testing problem.  Using the results for a known host PMF, practical tests based on the estimation of the host PMF are obtained. These are shown to be superior to the state of the art in terms of receiver operating characteristics as well as self-calibration across different host images. Estimators for the hiding rate are also developed.
Graphical Model-Based Tracking of Curvilinear Structures in Bio-image Sequences

Tracking of curvilinear structures is a task of fundamental importance in the quantitative analysis of biological structures such as neurons, blood vessels, retinal interconnects, microtubules, etc. The state of the art HMM-based contour tracking scheme for tracking microtubules, while performing well in most scenarios, can miss the track if, during its growth, it intersects another microtubule in its neighbourhood. In this paper we present a graphical model-based tracking algorithm which propagates across frames information about the dynamics of all the microtubules. This allows the algorithm to faithfully differentiate the contour of interest from others that contribute to the clutter, and maintain tracking accuracy. We present results of experiments on real microtubule images captured using fluorescence microscopy, and show that our proposed scheme outperforms the existing HMM-based scheme.
Weakly Supervised Manifold Learning for Dense Semantic Object Correspondence

The goal of the semantic object correspondence problem is to compute dense association maps for a pair of images such that the same object parts get matched for very different appearing object instances. Our method builds on the recent findings that deep convolutional neural networks (DCNNs) implicitly learn a latent model of object parts even when trained for classification. We also leverage a key correspondence problem insight that the geometric structure between object parts is consistent across multiple object instances. These two concepts are then combined in the form of a novel optimization scheme. This optimization learns a feature embedding by rewarding for projecting features closer on the manifold if they have low feature-space distance. Simultaneously, the optimization penalizes feature clusters whose geometric structure is inconsistent with the observed geometric structure of object parts. In this manner, by accounting for feature space similarities and feature neighborhood context together, a manifold is learned where features belonging to semantically similar object parts cluster together. We also describe transferring these embedded features to the sister tasks of semantic keypoint classification and localization task via a Siamese DCNN. We provide qualitative results on the Pascal VOC 2012 images and quantitative results on the Pascal Berkeley dataset where we improve on the state of the art by over 5% on classification and over 9% on localization tasks.
A contour-based approach to multisensor image registration

Image registration is concerned with the establishment of correspondence between images of the same scene. One challenging problem in this area is the registration of multispectral/multisensor images. In general, such images have different gray level characteristics, and simple techniques such as those based on area correlations cannot be applied directly. On the other hand, contours representing region boundaries are preserved in most cases. The authors present two contour-based methods which use region boundaries and other strong edges as matching primitives. The first contour matching algorithm is based on the chain-code correlation and other shape similarity criteria such as invariant moments. Closed contours and the salient segments along the open contours are matched separately. This method works well for image pairs in which the contour information is well preserved, such as the optical images from Landsat and Spot satellites. For the registration of the optical images with synthetic aperture radar (SAR) images, the authors propose an elastic contour matching scheme based on the active contour model. Using the contours from the optical image as the initial condition, accurate contour locations in the SAR image are obtained by applying the active contour model. Both contour matching methods are automatic and computationally quite efficient. Experimental results with various kinds of image data have verified the robustness of the algorithms, which have outperformed manual registration in terms of root mean square error at the control points.
SPAM: Signal Processing to Analyze Malware [Applications Corner]

In this article, we explored orthogonal methods to analyze malware motivated by signal and image processing. Malware samples are represented as images or signals. Image and signal-based features are extracted to characterize malware. Our extensive experiments demonstrate the efficacy of our methods on malware classification and retrieval. We believe that our techniques will open the scope of signal-and image-based methods to broader fields in computer security.
Search and retrieval of multi-modal data associated with image-parts

We present a novel framework for querying multi-modal data from a heterogeneous database containing images, textual tags, and GPS coordinates. We construct a bi-layer graph structure using localized image-parts and associated GPS locations and textual tags from the database. The first layer graphs capture similar data points from a single modality using a spectral clustering algorithm. The second layer of our multi-modal network allows one to integrate the relationships between clusters of different modalities. The proposed network model enables us to use flexible multi-modal queries on the database.
Integrating multiple model views for object recognition

We present a new approach to appearance-based object recognition, which captures the relationships between multiple model views and exploits them to improve recognition performance. The basic building block is local, viewpoint invariant regions. We propose an efficient algorithm for partitioning a set of region matches into groups lying on smooth surfaces (GAMs). During modeling, the model views are connected by a large number of region-tracks, each aggregating image regions of a single physical region across the views. At recognition time, GAMs are constructed matching a test image to each model view. The consistency of configurations of GAMs is measured by exploiting the model connections. A genetic algorithm finds covering the object as completely as possible the most consistent configuration. Introducing GAMs as an intermediate grouping level facilitates decision-making and improves discriminative power. As a complementary application, we introduce a novel GAM-based two-view filter and demonstrate its effectiveness in recovering correct matches in the presence of up to 96% mismatches.
Representation of motion activity in hierarchical levels for video indexing and filtering

A method for video indexing and filtering based on motion activity characteristics in hierarchical levels is proposed. To extract motion activity information, an MPEG (MPEG-1/2) video is first adaptively segmented into hierarchical levels with fixed percentage of original video length based on P-frame macroblock motion information. Three motion activity characteristics - motion intensity which represents the degree of change in motion, motion intensity histogram which represents the temporal statistics of motion intensity, and spatial descriptor which represents the spatial attribute of motion, are then computed to represent different levels of video. The descriptors from different levels are used selectively in different steps of video indexing and filtering. Experimental results show the proposed method is fast and effective, and provides a powerful video indexing and filtering tool.
Texture Features for Browsing and Retrieval of Image Data

Image content based retrieval is emerging as an important research area with application to digital libraries and multimedia databases. The focus of this paper is on the image processing aspects and in particular using texture information for browsing and retrieval of large image data. We propose the use of Gabor wavelet features for texture analysis and provide a comprehensive experimental evaluation. Comparisons with other multiresolution texture features using the Brodatz texture database indicate that the Gabor features provide the best pattern retrieval accuracy. An application to browsing large air photos is illustrated.
Detecting path intersections in panoramic video

Given panoramic video taken along a self-intersecting path, we present a method for detecting the intersection points. This allows "virtual tours" to be synthesized by splicing the panoramic video at the intersection points. Spatial intersections are detected by finding the best-matching panoramic images from a number of nearby candidates. Each panoramic image is segmented into horizontal strips. Each strip is averaged in the vertical direction. The Fourier coefficients of the resulting 1-D data capture the rotation-invariant horizontal texture of each panoramic image. The distance between two panoramic images is calculated as the sum of the distances between their strip texture pairs at the same row positions. The intersection is chosen as the two candidate panoramic images that have the minimum distance.
Automatic video annotation through search and mining

Conventional approaches to video annotation predominantly focus on supervised identification of a limited set of concepts, while unsupervised annotation with infinite vocabulary remains unexplored. This work aims to exploit the overlap in content of news video to automatically annotate by mining similar videos that reinforce, filter, and improve the original annotations. The algorithm employs a two-step process of search followed by mining. Given a query video consisting of visual content and speech-recognized transcripts, similar videos are first ranked in a multimodal search. Then, the transcripts associated with these similar videos are mined to extract keywords for the query. We conducted extensive experiments over the TRECVID 2005 corpus and showed the superiority of the proposed approach to using only the mining process on the original video for annotation. This work represents the first attempt at unsupervised automatic video annotation leveraging overlapping video content.
Graph cut segmentation of neuronal structures from transmission electron micrographs

In many neurophysiological studies, understanding the neuronal circuitry of the brain requires detailed 3D models of the nerve cells and their synapses. Typically, researchers build the 3D models by manually tracing the 2D cross-sectional profiles of the 3D structures from serial electron micrograph (EM) stacks and then construct the models from these 2D contours. While current computer-aided techniques can reduce the tracing time, they often require extensive user interaction. We propose a segmentation framework to extract the 2D profiles that is both fast and requires a minimal amount of user interaction. The framework uses graph cuts to minimize an energy defined over the image intensity and the flux of the intensity gradient field. Furthermore, to correct segmentation errors, our framework allows for efficient and intuitive editing of the initial results.
Effects of nonuniformity factor on the impulse breakdown voltage characteristics of SF/sub 6/ gas and with a spacer

Investigations were carried out on the breakdown characteristics of a nonuniform field gap, with and without spacers, under impulse voltages in the SF/sub 6/ environment. Both lightning impulse (LI) and switching impulse (SI) voltages were considered. In a highly nonuniform field, the presence of the spacer did not change the breakdown voltage characteristics significantly. With SI, for positive polarity, the spacer efficiency was observed to be greater than one. However under a moderately nonuniform field, the presence of the spacer appeared to distort the field distribution considerably, which was manifested in the breakdown voltages characteristics. The surface charge which occurred on the insulating surface probably changed the moderately nonuniform field into highly nonuniform field conditions. The results of these investigations are discussed.<<ETX>>
Intra-class multi-output regression based subspace analysis

A common challenge when dealing with heterogenous tasks such as face expression analysis, face and object recognition is high dimensionality and extreme appearance variations within each class. To handle such scenarios, we formulate a supervised Non-negative Matrix Factorization (NMF) based subspace learning technique that simultaneously preserves the intra-class regression information (local) and enhances inter-class discrimination (global) in the low dimensional embedding. Our method leverages the multi-dimensional image labels that quantify the within class regression to learn the subspaces for recognition. In addition, our formulation includes a novel multi-output regression based NMF algorithm.