nils-herrmann/clean_scirepeval huggingface dataset:
Visualising the Query Space of the Image Collection

In this paper, we propose an information visualisation solution for multimedia retrieval based on semantic concepts defined in an image database. The proposed visualisation approach utilises concept map, Venn diagram and Fisheye distortion to enable effective and efficient visualisation of image database. In addition, the proposed approach enables displaying the local and global views of the collection subset, selected as relevant by the user. The proposed solution is evaluated on Corel 700 dataset with 10 semantic concepts for the following user actions: exploratory browsing, querying and detecting patterns.
Combining a fuzzy rule-based classifier and illumination Invariance for improved building detection

The problem of edge-based classification of natural video sequences containing buildings and captured under changing lighting conditions is addressed in this letter. The introduced approach is derived from two empiric observations: In static regions the likelihood of finding features that match the patterns of "buildings" is high because buildings are rigid static objects; and misclassification can be reduced by filtering out image regions changing or deforming in time. These regions may contain objects semantically different to buildings but with a highly similar edge distribution, e.g., high frequency of vertical and horizontal edges. Using these observations a strategy is devised in which a fuzzy rule-based classification technique is combined with a method for changing region detection in outdoor scenes. The proposed approach has been implemented and tested with sequences showing changes in the lighting conditions. Selected results from the experimental evaluation are reported.
Navigation in REVERIE's virtual environments

This work presents a novel navigation system for social collaborative virtual environments populated with multiple characters. The navigation system ensures collision free movement of avatars and agents. It supports direct user manipulation, automated path planning, positioning to get seated, and follow-me behaviour for groups. In follow-me mode, the socially aware system manages the mise en place of individuals within a group. A use case centred around on an educational virtual trip to the European Parliament created for the REVERIE FP7 project, also serves as an example to bring forward aspects of such navigational requirements.
A concise survey for 3D reconstruction of building façades

3D facade modeling consists of representing architectural elements of a building in detail so that both geometry and appearance are generated in a photo-realistic 3D scene. In this article, we review four main techniques used to solve this problem: photogrammetry, sparse and dense reconstruction based on SfM (Structure from Motion) and procedural modeling. We provide a comparison of several methods in these categories by considering their main advantages and limitations.
A complexity-scalable transcoder from H.264/AVC to the new HEVC codec

The emerging video coding standard, HEVC, is currently approaching the final stage of development prior to standardization. However, the current H.264/AVC standard is very successful, and it has been widely adopted for many applications. Thus, transcoding between these codecs will be highly needed once the HEVC codec is finalised. This paper studies the performance of one of the most common techniques for heterogeneous transcoding, motion vector (MV) reuse, in a H.264/AVC to HEVC transcoder. Furthermore, it proposes a new transcoder that is capable of complexity scalability, trading off rate-distortion performance for complexity reduction. The proposed transcoder is based on a new metric to compute the similarity of the H.264/AVC MVs, which is used to decide which HEVC partitions are tested on the transcoder.
Semantic Object Based Retrieval from Surveillance Videos

In recent years, due to technological developments, the use of Closed-Circuit Television monitoring has been widely used not only in public areas but also in confined and/or private spaces for improved personal safety and security. The increased data acquisition has naturally resulted in the critical need for multimedia analysis for semantic object and event detection. Addressing this research problem, in this paper we present an novel architecture for extracting and indexing semantic objects with Scale Invariant Feature Transform features. The proposed approach exploits the developments of motion tracking and video indexing algorithms. The proposed framework is an ongoing development with the objective to enable the semantic retrieval of objects. The preliminary performance analysis of the proposed approach has been evaluated on a set of surveillance videos.
The privacy challenges of in-depth video analytics

The increasing need for both automated and privacy-respecting CCTV systems adds many challenges to the tasks of Video Analytics. The growing capabilities of automated surveillance systems lead to the automatic extraction and processing of complex and potentially personal data related to individuals who may not even be aware of it. This paper discusses the issues related to the processing of potentially sensitive information extracted from various computer vision algorithms and techniques such as person tracking, soft biometrics, behaviour analysis or multi-modal person identification, and delivers insights regarding technical solutions and guidelines for achieving a higher level of privacy compliance.
How accurate should be disparities estimated for image-based rendering?

Disparity estimation is the basis for the generation of a virtual view from a small set of real reference images. Much research on this area has been conducted by the computer vision community over the last decade. Currently, most problems involved in this technology are well understood and there exist several well-established algorithms to render virtual views from perspectively different images of the same scene. However, most work has been oriented towards high-accuracy disparity estimation to produce high-quality virtual images, often using sophisticated purpose-built hardware accelerators to achieve real-time results. Two disparity estimators with different complexity degrees are described and used to examine how much disparity inaccuracies influence image rendering quality. The objective of this study is to design software-based image synthesis in real-time on conventional PC platforms. Basically, this work looks at the opposite end of the cost-complexity curve by making very restrained demands on the disparity estimator. It is shown empirically that in many cases the effect of disparity accuracy on the quality of virtual views is almost imperceptible and that, for many applications requiring real-time processing, reasonably good results can be achieved with less computational cost.
A low access latency video portal

Using a meaningful combination of advanced video processing techniques, the development of a low-access latency video portal is presented. The main objective is to overcome the speed limitations of existing video indexing and retrieval systems. Due to the linear and data intensive nature of video, the temporal segmentation of video sequences is the first step towards the implementation of the system. Video is first partitioned into basic units called shots. Thereafter, each single shot can he identified by a key-frame containing the most relevant scene information. For the sake of efficiency these processing steps are carried out in the compressed domain. Once key frames have been extracted a family of successively simplified image features is defined using colour histograms. The image colour distribution is quantised according to a scale-parameter obtained by convolving the colour distribution function with Gaussians of increasing size. The generated family of histograms is then used to define descriptors at various levels of detail. Using the hierarchical descriptor structure, irrelevant details and noise are removed in a very early processing step.
Asymmetric and Normalized Cuts for Image Clustering and Segmentation

Over the last few years spectral clustering has emerged as a powerful model for data partitioning and segmentation. Spectral clustering techniques use eigenvalues and eigenvectors of the matrix representation of a suitable graph representing the original data. In this paper a new spectral clustering method is proposed: the asymmetric cut. It allows extraction of relevant information from a dataset by making just one cut over the database. The approach is tailored to the image classification task where a given image class is to be extracted from an image database containing an unknown number of classes. The main goal of this paper is to show that the proposed technique outperforms standard spectral methods under given circumstances. The technique is compared against the conventional and well-known normalized cut algorithm
Steadiness analysis for optimal GOP size selection in HEVC

High Efficiency Video Coding is the latest and most advanced video coding standard. It supports various Group Of Pictures (GOP) sizes and types such as low delay and random access. The size of the GOP substantially influences the temporal coding process. Therefore, a suitable GOP selection strategy can have a significant impact in the compression efficiency. In this paper, a strategy for GOP selection is proposed. It is derived from a new, low complexity measure of the temporal “steadiness” in the video content. Contrasting conventional approaches the proposed technique does not relies on previous estimation of motion vectors or motion information in the video sequence. As such, it can be used for automated encoding parameter optimization at the start of the coding process. The proposed technique leads to improved encoding efficiency at a negligible computational cost, when compared with the standard coding settings. A comprehensive experimental evaluation confirm that on average -6.69%, BD-rate gain and 14.18% time savings can be achieved.
The sparse image representation for automated image retrieval

We describe a novel sparse image representation for full automated content-based image retrieval using the latent semantic indexing (LSI) approach and also a novel statistical-based model for the efficient dimensional reduction of sparse data. Although images can be represented sparsely for instance by the discrete cosine transform (DCT) coefficients, this sparsity character is destroyed during the LSI-based dimension reduction process. In our approach, we keep the memory limit of the decomposed data by a statistical model of the sparse data. The aim is to find a small but "important" sub-set of coefficients, which represent semantics of images efficiently. The effectiveness of our novel approach is demonstrated by the large scale image similarity task of the NIST TrecVid 2007 benchmark.
A novel linear technique to estimate the epipolar geometry

The accurate reconstruction of the 3D scene structure from two different projections and the estimation of the camera scene geometry is of paramount importance in many computer vision tasks. Most of the information about the camera-scene geometry is encapsulated in the fundamental matrix. Estimating the fundamental matrix has been an object of research for many years and continues to be a challenging task in current computer vision systems. While nonlinear iterative approaches have been successful in dealing with the high instability of the underlying problem, their inherent large workload makes these approaches inappropriate for real-time applications. Practical aspects of highly efficient linear methods are studied and a novel low-cost and accurate linear algorithm is introduced. The performance of the proposed approach is assessed by several experiments on real images.
Fuzzy ensembles for embedding adaptive behaviours in semi-autonomous avatars in 3D virtual worlds

Semi-autonomous avatars should be both realistic and believable. The goal is to learn from and reproduce the behaviours of the user-controlled input to enable semi-autonomous avatars to plausibly interact with their human-controlled counterparts. A powerful tool for embedding autonomous behaviour is learning by imitation. Hence, in this paper an ensemble of fuzzy inference systems cluster the user input data to identify natural groupings within the data to describe the users movement and actions in a more abstract way. Multiple clustering algorithms are investigated along with a neuro-fuzzy classifier; and an ensemble of fuzzy systems are evaluated.
An efficient video super-resolution approach based on sparse representation

Block-wise super-resolution methods, and in particular sparse representation based approaches, often focus on spatial upsampling of still images. Applying such models to videos is an extremely time consuming process due to the expensive sparse coding process for every block of each frame, and the conventional exhaustive overlapping blocks processing for reducing the blocking artifacts. In this paper, we introduce an approach that enables us to skip the sparse representation process for the static portions of a video by predicting the high resolution blocks from previously super-resolved frames, exploiting the significant amount of correlation between the adjacent frames in a video. Our approach is also enhanced by adaptive in-loop filters for removing the blocking and pixel-wise artifacts, replacing the overlapping blocks structure of super-resolution. Our method provides comparable results in terms of image quality with respect to the state of the art, and can reduce the processing time of the super-resolution methods by 63% in average. It enables the block based super-resolution methods to be applied in upsampling of videos to very high resolutions with a reasonable processing time.
Deep Co-Space: Sample Mining Across Feature Transformation for Semi-Supervised Learning

Aiming at improving the performance of visual classification in a cost-effective manner, this paper proposes an incremental semi-supervised learning paradigm called deep co-space (DCS). Unlike many conventional semi-supervised learning methods usually performed within a fixed feature space, our DCS gradually propagates information from labeled samples to unlabeled ones along with deep feature learning. We regard deep feature learning as a series of steps pursuing feature transformation, i.e., projecting the samples from a previous space into a new one, which tends to select the reliable unlabeled samples with respect to this setting. Specifically, for each unlabeled image instance, we measure its reliability by calculating the category variations of feature transformation from two different neighborhood variation perspectives and merged them into a unified sample mining criterion deriving from Hellinger distance. Then, those samples keeping stable correlation to their neighboring samples (i.e., having small category variation in distribution) across the successive feature space transformation are automatically received labels and incorporated into the model for incrementally training in terms of classification. Our extensive experiments on standard image classification benchmarks (e.g., Caltech-256 and SUN-397) demonstrate that the proposed framework is capable of effectively mining from large-scale unlabeled images, which boosts image classification performance and achieves promising results compared with other semi-supervised learning methods.
Masking of transformed intra-predicted blocks for high quality image and video coding

Many professional applications for image and video coding require very high levels of quality of the decoded signal. Under these conditions, even state-of-the-art standards such as High Efficiency Video Coding (HEVC) may not provide sufficient compression efficiency. Large values of the residual samples, especially at high frequency components, are difficult to encode and result in high bitrates of the coded signal. A novel scheme for image and video coding is presented in this paper to enhance compression at high quality levels, based on separate transform to the frequency domain of original and prediction signals. Selected frequency components of the prediction signal are discarded by means of appropriate masking patterns prior to the residual computation. The optimal masking pattern is selected for each transformed block and signalled in the bitstream. The approach is shown achieving gains against conventional HEVC under high quality constraints when coding both still images and video sequences.
On the influence of motion vector precision limiting in scalable video coding

Recent studies on scalable video coding have not only substantiated the need for such technology but also made evident that many related problems remain open and need to be tackled if truly scalable video coding is to be achieved. One of these challenges relates to the coding of motion vectors. In conventional coders motion vectors are treated and coded in a nonprogressive manner. Since scalable video coding targets decoding at several resolutions and a wide range of quality levels, the motion information needs to be encoded in an adaptive way. We propose a simple, yet efficient, strategy for scalable motion vector coding. The results show improvements of resolution scalability performance at lower-bit rates, while overcoming any negative influence at high resolutions and bit-rates.
Robust Feature Matching in Long-Running Poor-Quality Videos

We describe a methodology that is designed to match key point and region-based features in real-world images, acquired from long-running security cameras with no control over the environment. We detect frame duplication and images from static scenes that have no activity to prevent processing saliently identical images, and describe a novel blur-sensitive feature detection method, a combinatorial feature descriptor, and a distance calculation that efficiently unites texture and color attributes to discriminate feature correspondence in low-quality images. Our methods are tested by performing key point matching on real-world security images such as outdoor closed-circuit television videos that are low quality and acquired in uncontrolled conditions with visual distortions caused by weather, crowded scenes, emergency lighting, or the high angle of the camera mounting. We demonstrate an improvement in accuracy of matching key points between images compared with state-of-the-art feature descriptors. We use key point features from a Harris corner detector, scale-invariant feature transform, speeded-up robust features, binary robust invariant scalable keypoints, and features from accelerated segment test as well as MSER and MSCR region detectors to provide a comprehensive analysis of our generic method. We demonstrate feature matching using a 138D descriptor that improves the matching performance of a state-of-the-art 384D color descriptor with just 36% of the storage requirements.
Semantic labeling of images combining color, texture and keywords

Content-based image retrieval systems combine perceptual features such as color, texture and shape with semantic concepts for improving the quality of the query's results. In this paper, an annotation technique that combines color and texture with keywords is presented. A method based on color similarity along with a keyword mining technique is used to propagate keywords extracted from a sub-set of annotated images into a large-scale database. A method based on texture properties is applied to link keywords with regions within the images. Finally, an approach for semantic labeling of images is described. In this approach, accuracy of the annotations is estimated and the relationships among keywords are identified. The presented annotation technique is useful for labeling images with keywords construing the underlying semantic content.
Improved indoor scene geometry recognition from single image based on depth map

Interpreting 3D structure from 2D images is a constant problem to be solved in the field of computer vision. Prior work has been made to tackle this issue mainly in two different ways - depth estimation from multiple-view images based on geometric triangulation and depth reasoning from single image depending on monocular depth cues. Both solutions do not involve direct depth map information. In this work, we captured a RGBD dataset using Microsoft Kinect depth sensor. Approximate depth information is acquired as the fourth channel and employed as an extra reference for 3D scene geometry reasoning. It helps to achieve better estimation accuracy. We define nine basic geometric models for general indoor restricted-view scenes. Then we extract low/medium level colour and depth features from all four of the RGBD channels. Sequential Minimal Optimization SVM is used in this work as efficient classification tool. Experiments are implemented to compare the result of this approach with previous work that does not have the depth channel as input.
Data-driven nonlinear diffusion for object segmentation

We propose a novel technique for effective object segmentation, which is based on the combination of image simplification via data-driven nonlinear diffusion and subsequent efficient segmentation of the simplified image. In particular, the data, taking the form of a disparity field from a stereo analysis, has been used to modulate the diffusion process. The strength of this strategy consists of an ability to smooth considerably the details of the imaging scene within the objects' boundaries while inhibiting the diffusion across the boundaries, preserving and even enhancing the object borders. As such, from the simplified image, a simple but efficient histogram-based thresholding and labeling technique can be used to extract precisely an object boundary in its entirety.
Spatially Adaptive Wavelet Transform for Video Coding with Multi-Scale Motion Compensation

In this paper a technique that enables efficient synthesis of the prediction signal for application in multi-scale motion compensation is presented. The technique targets prediction of high-pass spatial subbands for motion compensation at higher scales. Since in the targeted framework these subbands are obtained by high-pass filtering of prediction signal in pixel domain, an adaptive approach for filtering is proposed to support decomposition of differently predicted frame areas. In this way an efficient application of different prediction modes at all scales used for compensation is enabled. Experimental results show that for fast sequences where such an application of different prediction modes is crucial, the proposed adaptive transform introduces significant objective and visual improvements.
Robust feature matching in the wild

Finding corresponding key points in images from security camera videos is challenging. Images are generally low quality and acquired in uncontrolled conditions with visual distortions caused by weather, crowded scenes, emergency lighting or the high angle of the camera mounting. We describe a methodology to match features between images that performs especially well with real-world images. We introduce a novel blur sensitive feature detection method, a combinatorial feature descriptor and a distance calculation that efficiently unites texture and colour attributes to discriminate feature correspondence in low quality images. Our methods are tested by performing key point matching on real-world security images such as outdoor CCTV videos, and we demonstrate an improvement in the ability to match features between images compared with the standard feature descriptors extracted from the same set of feature points. We use key point features from Harris Corners, SIFT, SURF, BRISK and FAST as well as MSER and MSCR region detectors to provide a comprehensive analysis of our generic method. We demonstrate feature matching using a 138-dimensional descriptor that improves the matching performance of a state-of-the-art 384-dimension colour descriptor with just 40% of the storage requirements.