nils-herrmann/clean_scirepeval huggingface dataset:
Scene cut detection using the colored pattern appearance model

In this paper, we propose to use the colored pattern appearance model (CPAM) as a content representation for video scene break detection. This model represents a scene by means of global statistics of the local visual appearance, and was originally motivated by studies in human color vision. The performance of this method is compared to several histogram-based approaches. An adaptive thresholding technique, namely entropic thresholding, is applied to determine the respective optimal threshold values for each of the approaches. In the experiments, the two video sequences in the MPEG-7 content set are used to evaluate the performances of the CPAM and the histogram-based methods. Experimental results show that our proposed model outperforms other histogram-based approaches in scene break detection.
Habitat classification using random forest based image annotation

Habitat classification is an important ecological activity used to monitor environmental biodiversity. Current classification techniques rely heavily on human surveyors and are laborious, time consuming, expensive and subjective. In this paper, we approach habitat classification as an automatic image annotation problem. We have developed a novel method for annotating ground-taken photographs with the habitats present in them using random projection forests. For this purpose, we have collected and manually annotated a geo-referenced habitat image database with over 1000 ground photographs. We compare the use of two different types of input (blocks within images and the whole images) to classify habitats. We also compare our approach with a popular random forest implementation. Results show that our approach has a lower error rate and it is able to classify three habitats (Woodland and scrub, Grassland and marsh, and Miscellaneous) with a high recall.
A new key frame representation for video segment retrieval

In this paper, we propose an optimal key frame representation scheme based on global statistics for video shot retrieval. Each pixel in this optimal key frame is constructed by considering the probability of occurrence of those pixels at the corresponding pixel position among the frames in a video shot. Therefore, this constructed key frame is called temporally maximum occurrence frame (TMOF), which is an optimal representation of all the frames in a video shot. The retrieval performance of this representation scheme is further improved by considering the k pixel values with the largest probabilities of occurrence and the highest peaks of the probability distribution of occurrence at each pixel position for a video shot. The corresponding schemes are called k-TMOF and k-pTMOF, respectively. These key frame representation schemes are compared to other histogram-based techniques for video shot representation and retrieval. In the experiments, three video sequences in the MPEG-7 content set were used to evaluate the performances of the different key frame representation schemes. Experimental results show that our proposed representations outperform the alpha-trimmed average histogram for video retrieval.
Multi-scale visual attention & saliency modelling with decision theory

Recently, an information-based saliency technique which is biologically plausible and computationally feasible called Discriminant Saliency (DIS) has been proposed. While DIS successfully defines discriminant saliency in the information theoretic sense, its implementation restraints the sampled features to a single fixed-size window and creates a bias towards objects with distinctive features fitted in the window size. This paper proposes a multi-scale discriminant saliency (MDIS) technique for visual attention which uses the wavelet transform for the multi-resolution framework. MDIS utilizes mutual information between classes and feature distribution to quantify classifying discriminant power as saliency value in multiple dyadic-scale structures. The paper will present simulations on Neil Bruce's image database with quantitative and qualitative results showing the advantages of MDIS over DIS. For quantitative comparisons, numerical tests AUC, NSS, LCC are measured and several plots are generated to visualized differences between simulation modes; meanwhile, qualitative evaluation is a visual examination of synthesized saliency maps of general natural scenes.
Efficient coarse-to-fine near-duplicate image detection in riemannian manifold

This paper presents an efficient coarse-to-fine strategy for near duplicate image detection in a Riemannian space. At the coarse level, we use the faster but less accurate log-Euclidean Riemannian metric to search the entire database to retrieve a subset of the images that are likely to contain the near duplicates of the querying image; and at the fine level, we use the more accurate but computationally more demanding affine-invariant Riemannian metric to search the coarse level results to accurately identify near-duplicates. We present experimental results to show that the new coarse to fine strategy can be over 20 times faster than existing techniques using affine-invariant Riemannian metric without sacrificing accuracy.
Functional optimization properties of median filtering

In this letter, we use a new approach for studying the properties of median filtering. Specifically, using threshold decomposition, it is shown that median filtering operation minimizes a two-term cost function of the output state of the median filter. The first term of the cost function measures the smoothness between the median filter output and its neighbor points within the operation window, and the second term measures the discrepancy between the filter output and its original signal. The results from this study have provided us with a new tool to analyze and understand some of the properties of the median filtering operation, including weighted median filtering.
Video buffer reduction via global and local colour quantization

This paper investigates video buffer reduction techniques for faithful colour display by CRT monitors. Based on an understanding of the properties of the human visual system and statistics of natural image data, a scheme combining global and local colour quantization to achieve efficient use of the video buffer and faithful visual colour display has been developed. Two different but related realisations of the scheme are implemented. A colour image quality metric, the S-CIELAB system (Zhang and Wandell, 1996), which was developed based on psychophysical measurements is used to evaluate the performance of the techniques. Experimental results are presented to demonstrate the effectiveness of the new scheme.
A portable real-time high-dynamic range video system

Using a single sensor to capture videos of scenes containing very high and very low luminance contents at the same time is still a challenge. An alternative solution is to use multiple sensors and specially designed optics. Such approach also faces some significant technical challenges including the manufacture of precision optics, accurate alignment of different sensors, recovering high dynamic range video frames and rendering them for display in real-time. In this work, we present a new real-time high dynamic range video system which is able to produce high quality live video up to 30 frames per second. The innovative features of the system include novel optics, new high dynamic range radiance map recovery methods and GPU based real-time HDR video processing and display. A video recorded live by the new system can be found in this vimeo link: http://vimeo.com/97595588. We will bring the system to the conference and perform live demo.
A colour histogram based approach to human face detection

We introduce a novel colour histogram based approach to human face detection in colour images. We reason that different regions of a human face contain different facial features and that the colour distributions of these regions may have certain characteristics that are unique to human faces. By concatenating colour histograms of different regions of a human face, we can form a vector that captures the spatial relationship of these unique regional characteristics, which in turn enables the development of effective face detection schemes. To improve efficiency and detection speed, we use principal component analysis (PCA) to reduce the dimensionality of the histograms and apply skin detection as a pre-processing step to reduce the search space. We use a support vector machine for both skin detection and face detection. We present experimental results to demonstrate the effectiveness of the new method.
Geometric Consistent Tree Partitioning Min-Hash for Large-Scale Partial Duplicate Image Discovery

This paper introduces a novel method that effectively and efficiently encodes the spatial geometric information of bag of visual words (BoW) to boost the performance of large scale partial duplicate image discovery and clustering. The loose cyclic spatial verification (LCSV) technique projects the locations of BoWs onto the perimeter of a circle centred around their geometric centroid and encodes their geometric relations in a simple ordered sequence of scalar values. We then treat the problem of validating the geometric consistencies of the BoWs from two separate image patches as the longest common cyclic subsequence (LCCS) problem and solve it using dynamic programming. By embedding the LCSV technique in a modified tree partitioning min-Hash framework, we introduce a geometric consistent tree partitioning min-Hash (gcTmH) technique for partial duplicate image discovery and clustering. We show that gcTmH is invariant to rotation and scaling, robust against noisy conditions, and is able to handle multiple duplicate models. We show that gcTmH can boost the accuracy of partial duplicate image discovery by deleting the random and false matching image pairs in a very efficient way. We present experimental results on two datasets and show that our method can boost partial duplicate image discovery performances of state of the art techniques.
Interactive skin condition recognition

It is believed that there are between 1000 to 2000 skin conditions, and about 20% are difficult to diagnose. An intelligent system capable of making accurate diagnosis not only helps patients in places where access to health services are scarce, but also benefits typical general practitioners who have received minimal dermatology training. In this paper, we introduce a challenging dataset developed by gathering 2309 images from 44 different skin conditions, and collecting answers to simple perceptual questions from 361 “Amazon Mechanical Turk” workers. We also propose a method based on random forest technology that combines visual features of the skin lesion images with user provided answers to achieve promising recognition rates. We believe that our solution can be potentially improved and installed on smart phones and tablets to enhance quality of life in patients across the world.
Image coding based on visual vector quantization

The authors present a novel image coding technique which explicitly aims to retain perceptually important features of the image. The system exploits the psychovisual properties of the human visual system, in that it identifies perceptually important vector patterns (referred to as visual vectors) within small blocks of the image, and codes these using a scheme similar to vector quantization (VQ). Thus the system is referred to as visual vector quantization (VVQ). One of the main advantages of the new VVQ scheme over traditional VQ is that it is computationally much simpler, and yet its performance is comparable to that of traditional VQ. Experimental results are presented which demonstrate that the new technique can achieve visually satisfactory image reconstruction at a compression ratio as high as 17:1. Comparison of peak signal to noise ratio (PSNR) values shows that the performance of the new technique is comparable to that of other VQ-based image coders at a similar bit rate.
HEp-2 cell classification based on a Deep Autoencoding-Classification convolutional neural network

In this paper, we present a novel deep learning model termed Deep Autoencoding-Classification Network (DACN) for HEp-2 cell classification. The DACN consists of an autoencoder and a normal classification convolutional neural network (CNN), while the two architectures shares the same encoding pipeline. The DACN model is jointly optimized for the classification error and the image reconstruction error based on a multi-task learning procedure. We evaluate the proposed model using the publicly available ICPR2012 benchmark dataset. We show that this architecture is particularly effective when the training dataset is small which is often the case in medical imaging applications. We present experimental results to show that the proposed approach outperforms all known state of the art HEp-2 cell classification methods.
Improved Image Compression Using Backpropagation Networks

This paper describes an improved image compression scheme using backpropagation networks. The new scheme is aimed at improving the networks' generalisation capabilities, thereby enabling them to effectively compress a wide range of novel images. The networks operate, and are trained on, residual image blocks, thus eliminating the problem of varying average image intensities highlighted by Cottrell et al. Tabulated experimental results and example reconstructed novel images, for the method of  and the new technique, are presented, which demonstrate the improved image compression performance gained using this new technique. 131.
Salient covariance for near-duplicate image and video detection

This paper introduces the covariance matrix of visually salient image features as a compact and robust descriptor for near duplicate image and video copy detection. We make two novel contributions. We first present a fast method for computing information theoretic based visual saliency maps using a data independent fast transform to replace the conventional data dependent computationally demanding transforms. We then introduce salient covariance (SCOV) — the covariance matrix of various image features within the visually salient regions and use SCOV for near duplicate image and video copy detection. We present experimental results to show that our new fast visual saliency computation technique improves efficiency without compromising performances. We demonstrate that SCOV is a very compact and robust feature for near duplicate image and video copy detection. Compared to popular features such as GIST, SCOV is not only more robust against various manipulations but also can be over 20 times more compact whilst achieving the same or better performances.
Spatio-temporal prediction of shopping behaviours using taxi trajectory data

Taxi trajectory data (GPS data collected for 15,000 taxis at intervals of 30 seconds across three million journeys over eight days) is used to generate a spatio-temporal prediction of shopping behaviours in the emerging metropolitan city of Shenzhen, China. Two approaches are compared: time-series forecasting using ARIMA; and a gravity model approach, using the Huff model calibrated with Geographical Weighted Regression. Results demonstrate that ARIMA performs with significantly higher accuracy than the more traditional Huff model method. Further, it is demonstrate that while the accuracy of the Huff model is constrained by model assumptions, applying time-series methods to the underlying data directly (i.e., the ARIMA method) has no such constraints, and is limited only by the amount of data available. This suggests that, as richer data sets become available, spatio-temporal modelling of this kind will become more accurate.
A Color Prediction System for Interactive Drawing Based Image Retrieval on Mobile Devices

Query suggestion plays a key role in improving the usability of image search. Textual Query Suggestion, widely used in existing search text-based image retrieval engines, is able to suggest a list of textual query terms based on users' query input. This paper presents a color prediction system dedicated to interactive drawing based image retrieval system on mobile devices. Usually, such systems retrieve the results after a complete sketch is submitted as input. We propose a color prediction system that recommends colors in the palette and predict drawings to assist users in creating queries. The predicted drawings can be fed to the retrieval system so it can retrieve results as soon as a user draws a stroke on the canvas. We conduct user studies to compare our system with a general drawing based image retrieval interface and show that our system can help users retrieve target image with an improved efficiency.
Adapting indexing features to scene structures for relevant image retrieval

Scene structural matrices (SSMs) are 2D tables that can be used conveniently to capture the organization of a scene in a colour image when it has been adaptively partitioned and represented by a simplified binary space partitioning (BSP) tree. In this paper, we present a number of methods to populate the cells of SSMs. Well-established and popular image indexing features are used. Since the SSMs implicitly contain scene organization information and intermediate level concept, indexing the cells with these features is equivalent to adapting them to the structure of the scene. To show the effectiveness of our method, we conducted an experiment to retrieve landscape images, a type of image which can be described using an intermediate-level concept. Our results show that the new methods significantly outperform the traditional methods.
An improved recursive median filtering scheme for image processing

In a recent publication, it was shown that median filtering is an optimization process in which a two-term cost function is minimized. Based on this functional optimization property of the median filtering process, a new approach for designing the recursive median filter for image processing applications is introduced in this paper. We prove that the new approach is guaranteed to converge to root within a finite number of iterations. The new method is applied to process a real image corrupted by pseudorandom impulsive noise, and the results show that the new scheme provides improved mean square error (RISE) performance over the standard recursive median filters.
Near-Duplicate Image Recognition and Content-based Image Retrieval using Adaptive Hierarchical Geometric Centroids

In this paper, we present a new feature extraction method that simultaneously captures the global and local characteristics of an image by adaptively computing hierarchical geometric centroids of the image. We show that these hierarchical centroids have some very interesting properties such as illumination invariant and insensitive to scaling. We have applied the method for near-duplicate image recognition and for content-based image retrieval. We present experimental results to show that our method works effectively in both applications
Draw-and-Guess: A mobile game with a purpose for drawing base image retrieval

The availability of finger-touch mobile devices has provided a feasible platform for drawing based image retrieval. While text labels represent a sematic description of an image, drawings represent a visual description of the image. In this paper, we describe the development and preliminary design of a game with a purpose that with potential to build a dataset for drawing based image retrieval field. To achieve this, we 1) develop a user-friendly interface to allow users to easily color sketches; 2) propose an entertaining game, Draw-and-Guess, which employs a drawing based image retrieval paradigm in the gameplay; 3) carry out a user study to evaluate the usability of this application and several design guidelines were concluded from the phases of development and evaluation; 4) shows how the collected dataset can be used for evaluating performance of algorithm in DBIR.
A Human Vision System based Flash Picture Coding Method for Video Coding

Flash light due to photographing widely appears in video sequences especially in those obtained from news interviews, conferences and sports matches. When a flash picture is encoded, the intensity changes drastically and non-uniformly so that motion estimation can not find a well-matching block in reference pictures. Accordingly, much more bits are generated than the neighboring non-flash pictures. In this paper, a novel flash picture coding method which represents a new concept of video coding is proposed. Based on human vision system (HVS) property, rather than encoding the original flash picture, it encodes an artificial flash picture which is made up of an artificial non-flash (or de-flashed) picture and some parameters for flash effect modeling. Experiments show the proposed flash picture coding method greatly reduces the coded bits of the flash picture while still provides a competitive subjective quality.
A Hybrid Probabilistic Model for Unified Collaborative and Content-Based Image Tagging

The increasing availability of large quantities of user contributed images with labels has provided opportunities to develop automatic tools to tag images to facilitate image search and retrieval. In this paper, we present a novel hybrid probabilistic model (HPM) which integrates low-level image features and high-level user provided tags to automatically tag images. For images without any tags, HPM predicts new tags based solely on the low-level image features. For images with user provided tags, HPM jointly exploits both the image features and the tags in a unified probabilistic framework to recommend additional tags to label the images. The HPM framework makes use of the tag-image association matrix (TIAM). However, since the number of images is usually very large and user-provided tags are diverse, TIAM is very sparse, thus making it difficult to reliably estimate tag-to-tag co-occurrence probabilities. We developed a collaborative filtering method based on nonnegative matrix factorization (NMF) for tackling this data sparsity issue. Also, an L1 norm kernel method is used to estimate the correlations between image features and semantic concepts. The effectiveness of the proposed approach has been evaluated using three databases containing 5,000 images with 371 tags, 31,695 images with 5,587 tags, and 269,648 images with 5,018 tags, respectively.
Generating Novel Information Salient Maps for Foreground Object Detection in Video

The conceptual model of visual saliency in human vision system has been employed in extracting salient features from images and multimedia data in the last decade. This paper proposes to employ the visual saliency for moving object detection. The crucial factor is to compute a saliency map such that visual attention can be performed. This paper proposes a new method for saliency map construction based on information theory and spatio-temporal model, called information saliency map (ISM). The ISM provides rich information content of the video. Moving object detection are then performed based on the ISM. Two popular and publicly available visual surveillance databases from CAVIAR and PETS are selected for evaluation. Experimental results show that the proposed method is robust for moving object detection in complex background and illumination changes. The average detection rate is 90.35% while the false alarm rate is 2.46% in CAVIAR (INRIA entrance hall) dataset with ground truth data, and it has shown merits comparing with the current state of the art.