nils-herrmann/clean_scirepeval huggingface dataset:
Robust multiple object tracking by detection with interacting Markov chain Monte Carlo

This paper presents a novel and computationally efficient multi-object tracking-by-detection algorithm with interacting particle filters. The proposed online tracking methodology could be scaled to hundreds of objects and could be completely parallelized. For every object, we have a set of two particle filters, i.e. local and global. The local particle filter models the local motion of the object. The global particle filter models the interaction with the other objects and scene. These particle filters are integrated into a unified Interacting Markov Chain Monte Carlo (IMCMC) framework. The local particle filter improves its performance by interacting with the global particle filter while they both are run in parallel. We indicate the manner in which we bring in object interaction and domain specific information into account by using global filters without further increase in complexity. Most importantly, the complexity of the proposed methodology varies linearly in the number of objects. We validated the proposed algorithms on two completely different domains 1) Pedestrian Tracking in urban scenarios 2) Biological cell tracking (Melanosomes). The proposed algorithm is found to yield favorable results compared to the existing algorithms.
Beyond Spatial Auto-Regressive Models: Predicting Housing Prices with Satellite Imagery

When modeling geo-spatial data, it is critical to capture spatial correlations for achieving high accuracy. Spatial Auto-Regression (SAR) is a common tool used to model such data, where the spatial contiguity matrix (W) encodes thespatial correlations. However, the efficacy of SAR is limited by two factors. First, it depends on the choice of contiguity matrix, which is typically not learnt from data, but instead, is assumed to be known apriori. Second, it assumes that the observations can be explained by linear models. In this paper, we propose a Convolutional Neural Network (CNN) framework to model geo-spatial data (specifically housing prices), to learn the spatial correlations automatically. We show that neighborhood information embedded in satellite imagery can be leveraged to achieve the desired spatial smoothing. An additional upside of our framework is the relaxation of linear assumption on the data. Specific challenges we tackle while implementing our framework include, (i) how much of the neighborhood is relevant while estimating housing prices? (ii) what is the right approach to capture multiple resolutions of satellite imagery? and (iii) what other data-sources can help improve the estimation of spatial correlations? We demonstrate a marked improvement of 57% on top of the SAR baseline through the use of features from deep neural networks for the cities of London, Birmingham and Liverpool.
Category-based image retrieval

This work presents a novel approach to content-based image retrieval in categorical multimedia databases. The images are indexed using a combination of text and content descriptors. The categories are viewed as semantic clusters of images and are used to confine the search space. Keywords are used to identify candidate categories. Content-based retrieval is performed in these categories using multiple image features. Relevance feedback is used to learn the user's intent-query specification and feature-weighting-with minimal user-interface abstraction. The method is applied to a large number of images collected from a popular categorical structure on the World Wide Web. Results show that efficient and accurate performance is achievable by exploiting the semantic classification represented by the categories. The relevance feedback loop allows the content descriptor weightings to be determined without exposing the calculations to the user.
Simultaneous cell tracking and image alignment in 3D CLSM imagery of growing Arabidopsis thaliana sepals

In this research we propose a combined cell matching and image alignment method for tracking cells based on their nuclear locations in 3D fluorescent Confocal Laser Scanning Microscopy (CLSM) image sequences. We then apply it to study the cell division pattern in the developing sepal of the small plant Arabidopsis thaliana. The method is based on geometric hashing and inherits its invariance to rotation, translation and scale. The method consists of three steps. In the first step the centroids of nuclei are detected using a previously developed cell detection algorithm, reducing the CLSM volumes to 3D point clouds, wherein every point represents a nuclear centroid with an associated confidence level. In the second step centroids between images are matched in two phases. First geometric hashing is used to find an initial set of centroid matches, then using the initial matches a dense matching is obtained through a novel iterative point matching algorithm. In the last step centroid matches are used to estimate transformations and register all input images to a common frame. Our algorithm has successfully aligned 12 volumes encompassing 72 hours data set and matched 258 nuclear lifelines.
Probabilistic motion parameter models for human activity recognition

A novel method for human activity recognition is presented. Given a video sequence containing human activity, the motion parameters of each frame are first computed using different motion parameter models. The likelihood of these observed motion parameters is optimally approximated, based directly on a multivariate Gaussian probabilistic model. The dynamic change of motion parameter likelihood in a video sequence is characterized using a continuous density hidden Markov model. Activity recognition is then posed as a motion parameter maximum likelihood estimation problem. Experimental results show that the method proposed here works well in recognizing such complex human activities as sitting, getting up from a chair, and some martial art actions.
A compact image signature for RTS-invariant image retrieval

In this paper we propose a novel framework to obtain a very compact image signature (32 bits) which is invariant to rotation, translation, scaling and other minor perturbations like smoothing, random noise addition, JPEG compression etc. The framework involves Fourier-Mellin transform, conventional PCA and non-uniform scalar quantization. The high retrieval efficiency and low space consumption demonstrates the significance of our signature in duplicate image retrieval and large image database indexing.
Region of interest extraction and virtual camera control based on panoramic video capturing

We present a system for automatically extracting the region of interest (ROI) and controlling virtual cameras' control based on panoramic video. It targets applications such as classroom lectures and video conferencing. For capturing panoramic video, we use the FlyCam system that produces high resolution, wide-angle video by stitching video images from multiple stationary cameras. To generate conventional video, a region of interest can be cropped from the panoramic video. We propose methods for ROI detection, tracking, and virtual camera control that work in both the uncompressed and compressed domains. The ROI is located from motion and color information in the uncompressed domain and macroblock information in the compressed domain, and tracked using a Kalman filter. This results in virtual camera control that simulates human controlled video recording. The system has no physical camera motion and the virtual camera parameters are readily available for video indexing.
Generalized subspace based high dimensional density estimation

Our paper presents a novel high dimensional probability density estimation technique using any dimensionality reduction method. Our method first performs subspace reduction using any matrix factorization algorithm and estimates the density in the low-dimensional space using sample-point variable bandwidth kernel density estimation. Subsequently, the high dimensional density is approximated from the low dimensional density parameters. The reconstruction error due to dimensionality reduction process is also modeled in a principled and efficient manner to obtain the high dimensional density estimate. We show the effectiveness of our technique by using two popular dimensionality reduction tools, principal component analysis and non-negative matrix factorization. This technique is applied to AT&T, Yale, Pointing'04 and CMU-PIE face recognition datasets and improved performance compared to other dimensionality reduction and density estimation algorithms is obtained.
Unsupervised Texture Segmentation Using Markov Random Field Models

The problem of unsupervised segmentation of textured images is considered. The only explicit assumption made is that the intensity data can be modeled by a Gauss Markov random field (GMRF). The image is divided into a number of nonoverlapping regions and the GMRF parameters are computed from each of these regions. A simple clustering method is used to merge these regions. The parameters of the model estimated from the clustered segments are then used in two different schemes, one being all approximation to the maximum a posterior estimate of the labels and the other minimizing the percentage misclassification error. The proposed approach is contrasted with the algorithm of S. Lakshamanan and H. Derin (1989), which uses a simultaneous parameter estimation and segmentation scheme. The results of the adaptive segmentation algorithm of Lakshamanan and Derin are compared with a simple nearest-neighbor classification scheme to show that if enough information is available, simple techniques could be used as alternatives to computationally expensive schemes.
Recording the region of interest from FlyCam panoramic video

A novel method for region of interest tracking and recording video is presented. The proposed method is based on the FlyCam system (Foote et al., 2000), which produces high-resolution and wide-angle video sequences by stitching the video frames from multiple stationary cameras. The method integrates tracking and recording processes, and targets applications such as classroom lectures and videoconferencing. First, the region of interest (which typically covers the speaker) is tracked using a Kalman filter. Then, the Kalman filter estimation results are used for virtual camera control and to record the video. The system has no physical camera motion and the virtual camera parameters are readily available for video indexing. The proposed system has been implemented for real-time recording of lectures and presentations.
High-volume data hiding in images: Introducing perceptual criteria into quantization based embedding

Information-theoretic analyses for data hiding prescribe embedding the hidden data in the choice of quantizer for the host data. In this paper, we consider a suboptimal implementation of this prescription, with a view to hiding high volumes of data in images with low perceptual degradation. Our two main findings are as follows: (a) In order to limit perceptual distortion while hiding large amounts of data, the hiding scheme must use perceptual criteria in addition to information-theoretic guidelines. (b) By focusing on “benign” JPEG compression attacks, we are able to attain very high volumes of embedded data, comparable to information-theoretic capacity estimates for the more malicious Additive White Gaussian Noise (AWGN) attack channel, using relatively simple embedding techniques.
NeTra-V: toward an object-based video representation

We present a prototype video analysis and retrieval system, called NeTra-V, that is being developed to build an object-based video representation for functionalities such as search and retrieval of video objects. A region-based content description scheme using low-level visual descriptors is proposed. In order to obtain regions for local feature extraction, a new spatio-temporal segmentation and region-tracking scheme is employed. The segmentation algorithm uses all three visual features: color, texture, and motion in the video data. A group processing scheme similar to the one in the MPEG-2 standard is used to ensure the robustness of the segmentation. The proposed approach can handle complex scenes with large motion. After segmentation, regions are tracked through the video sequence using extracted local features. The results of tracking are sequences of coherent regions, called "subobjects". Subobjects are the fundamental elements in our low-level content description scheme, which can be used to obtain meaningful physical objects in a high-level content description scheme. Experimental results illustrating segmentation and retrieval are provided.
Provably Secure Steganography: Achieving Zero K-L Divergence using Statistical Restoration

In this paper, we present a framework for the design of steganographic schemes that can provide provable security by achieving zero Kullback-Leibler divergence between the cover and the stego signal distributions, while hiding at high rates. The approach is to reserve a number of host symbols for statistical restoration: host statistics perturbed by data embedding are restored by suitably modifying the symbols from the reserved set. A dynamic embedding approach is proposed, which avoids hiding in low probability regions of the host distribution. The framework is applied to design practical schemes for image steganography, which are evaluated using supervised learning on a set of about 1000 natural images. For the presented JPEG steganography scheme, it is seen that the detector is indeed reduced to random guessing.
Towards automated bioimage analysis: from features to semantics

Recent advances in bio-molecular imaging have afforded biologists a more thorough understanding of cellular functions in complex tissue structures. For example, high resolution fluorescence images of the retina reveal details about tissue restructuring during detachment experiments. Time sequence imagery of microtubules provides insight into subcellular dynamics in response to cancer treatment drugs. However, technological progress is accompanied by a rapid proliferation of image data. Traditional analysis methods, namely manual measurements and qualitative assessments, become time consuming and are often nonreproducible. Computer vision tools can efficiently analyze these vast amounts of data with promising results. This paper provides an overview of several challenges faced in bioimage processing and our recent progress in addressing these issues
Distortion estimation techniques in solving visual CAPTCHAs

This paper describes two distortion estimation techniques for object recognition that solve EZ-Gimpy and Gimpy-r, two of the visual CAPTCHAs ("completely automated public turing test to tell computers and humans apart") with high degrees of success. A CAPTCHA is a program that generates and grades tests that most humans can pass but current computer programs cannot pass. We have developed a correlation algorithm that correctly identifies the word in an EZ-Gimpy challenge image 99% of the time and a direct distortion estimation algorithm that correctly identifies the four letters in a Gimpy-r challenge image 78% of the time.
Search Tracker: Human-Derived Object Tracking in the Wild Through Large-Scale Search and Retrieval

Humans use context and scene knowledge to easily localize moving objects in conditions of complex illumination changes, scene clutter, and occlusions. In this paper, we present a method to leverage human knowledge in the form of annotated video libraries in a novel search and retrieval-based setting to track objects in unseen video sequences. For every video sequence, a document that represents motion information is generated. Documents of the unseen video are queried against the library at multiple scales to find videos with similar motion characteristics. This provides us with coarse localization of objects in the unseen video. We further adapt these retrieved object locations to the new video using an efficient warping scheme. The proposed method is validated on in-the-wild video surveillance data sets where we outperform state-of-the-art appearance-based trackers. We also introduce a new challenging data set with complex object appearance changes.
The multiRANSAC algorithm and its application to detect planar homographies

A RANSAC based procedure is described for detecting inliers corresponding to multiple models in a given set of data points. The algorithm we present in this paper (called multiRANSAC) on average performs better than traditional approaches based on the sequential application of a standard RANSAC algorithm followed by the removal of the detected set of inliers. We illustrate the effectiveness of our approach on a synthetic example and apply it to the problem of identifying multiple world planes in pairs of images containing dominant planar structures.
Image segmentation using curve evolution and flow fields

An image segmentation scheme that utilizes image-based flow fields in a curve evolution framework is presented. Geometric curve evolution methods require an edge function and a vector field with certain characteristics that are obtained from the image itself. A vector field borrowed from the edgeflow segmentation (Ma and Manjunath 2000) method is utilized both to obtain an edge function and to guide the curve evolution towards the object boundaries. This vector field is computed from the image using intensity, texture and color features. The proposed method integrates well-tested image features with the well-studied curve evolution methods thus achieving better segmentation results.
Robust Simultaneous Registration and Segmentation with Sparse Error Reconstruction

We introduce a fast and efficient variational framework for Simultaneous Registration and Segmentation (SRS) applicable to a wide variety of image sequences. We demonstrate that a dense correspondence map (between consecutive frames) can be reconstructed correctly even in the presence of partial occlusion, shading, and reflections. The errors are efficiently handled by exploiting their sparse nature. In addition, the segmentation functional is reformulated using a dual Rudin-Osher-Fatemi (ROF) model for fast implementation. Moreover, nonparametric shape prior terms that are suited for this dual-ROF model are proposed. The efficacy of the proposed method is validated with extensive experiments on both indoor, outdoor natural and biological image sequences, demonstrating the higher accuracy and efficiency compared to various state-of-the-art methods.
Saccade gaze prediction using a recurrent neural network

We present a model that generates close-to-human gaze sequences for a given image in the free viewing task. The proposed approach leverages recent advances in image recognition using convolutional neural networks and sequence modeling with recurrent neural networks. Feature maps from convolutional neural networks are used as inputs to a recurrent neural network. The recurrent neural network acts like a visual working memory that integrates the scene information and outputs a sequence of saccades. The model is trained end-to-end with real-world human eye-tracking data using back propagation and adaptive stochastic gradient descent. Overall, the proposed model is simple compared to the state-of-the-art methods while offering favorable performance on a standard eye-tracking data set.
Dimensionality reduction using multi-dimensional scaling for content-based retrieval

There has been much interest recently in image content based retrieval, with applications to digital libraries and image database accessing. One approach to this problem is to base retrieval from the database upon feature vectors which characterize the image texture. Since feature vectors are often high dimensional, multi-dimensional scaling, or non-linear principal components analysis (PCA) may be useful in reducing feature vector size, and therefore computation time. We have investigated a variant of the non-linear PCA algorithm described by Webb (see Pattern Recognition, vol.28, no.6, p.753-9, 1995) and its usefulness in the database retrieval problem. The results are quite impressive; in an experiment using an aerial photo database, the feature vector length was reduced by a factor of 10 without significantly reducing the retrieval performance.
Estimation of optimum coding redundancy and frequency domain analysis of attacks for YASS - a randomized block based hiding scheme

Our recently introduced JPEG steganographic method called yet another steganographic scheme (YASS) can resist blind steganalysis by embedding data in the discrete cosine transform (DCT) domain in randomly chosen image blocks. To maximize the embedding rate for a given image and a specified attack channel, the redundancy factor used by the repeat- accumulate (RA) code based error correction framework in YASS is optimally chosen by the encoder. An efficient method is suggested for the decoder to accurately compute this redundancy factor. We also show experimentally which DCT coefficients are better suited for hiding and detection under various attacks. The effectiveness of YASS for robust steganography is demonstrated for certain attacks.
Using texture to annotate remote sensed datasets

Texture remains largely underutilized in the analysis of remote sensed datasets compared to descriptors based on the orthogonal spectral dimension. This paper describes our recent efforts towards using texture to automate the annotation of remote sensed imagery. Two applications are described that use the homogeneous texture descriptor recently standardized by MPEG-7. In the first, higher-level access to remote sensed imagery is enabled by using the texture descriptor to model geo-spatial objects. In particular, the common textures, or texture motifs, are characterized as Gaussian mixtures in the high-dimensional feature space. In the second application, the texture descriptor is used to label regions in a large collection of aerial videography in a perceptually meaningful wax. Gaussian mixtures are used to model the distribution of feature vectors for a variety of semantic classes. Frame level similarity retrieval based on semantic layout and semantic histogram is enabled by modeling the spatial arrangement of the labeled regions as a Markov random field.
A feature based approach to face recognition

A feature-based approach to face recognition in which the features are derived from the intensity data without assuming any knowledge of the face structure is presented. The feature extraction model is biologically motivated, and the locations of the features often correspond to salient facial features such as the eyes, nose, etc. Topological graphs are used to represent relations between features, and a simple deterministic graph-matching scheme that exploits the basic structure is used to recognize familiar faces from a database. Each of the stages in the system can be fully implemented in parallel to achieve real-time recognition. Experimental results for a 128*128 image with very little noise are evaluated.<<ETX>>
Video region segmentation by spatio-temporal watersheds

In this paper, we propose a video region segmentation scheme combining spatio-temporal edges and watershed techniques. We consider the video sequence as a 3-D volume and compute color edges within this volume. These color edges form a vector field that is in turn used to obtain an edge function. This edge function is used as a topological surface for a watershed grouping stage. Considering the video as a 3-D volume results in a batch segmentation instead of the traditional frame-by-frame segmentation. The main advantages of this approach are: 1) exploiting the time continuity in the frame sequence , 2) avoiding problems in tracking regions from frame to frame, 3) using a fast watershed-based method informing the final video regions. Preliminary experimental results are very promising.