nils-herrmann/clean_scirepeval huggingface dataset:
Aspect coherence for graph-based image labelling

Semantic image labelling is the task of assigning each pixel of an image to a semantic category. To this end, in low-level image labelling, a labelled training set is available. In such a situation, structural information about the correlation between different image parts is particularly important. When a part- based inference algorithm is used to perform the association of semantic classes to pixels, however, a good choice on how to use structural information is crucial for learning an efficient and generalisable probabilistic model for the labelling task. In this paper we introduce an efficient way to take into account correlation between different image parts, embedding the parts relationships in a graph built according to aspect coherence of neighbouring image patches.
Inter-prediction optimisations for fast HEVC encoding of ultra high definition content

Thanks to technological advances in devices for video signal capturing and displaying, the usage of large resolution video formats is becoming more and more popular in consumer and professional applications. The new Ultra High Definition (UHD) format aims to provide enhanced viewing experience with its spatial resolution of at least 3840 × 2160 luma samples per frame and temporal resolution of up to 120 frames per second. Such a large amount of raw data requires extremely high levels of compression efficiency. The latest state-of-the-art High Efficiency Video Coding (HEVC) standard was developed also to respond to these needs. HEVC outperforms previous video coding standards in terms of coding efficiency, especially while encoding high resolution video content. However, this comes at very high computational costs, making current HEVC encoder implementations of very limited usage. In this paper, a fast inter-prediction scheme is proposed targeting fast mode decision based on reverse Coding Unit (CU) visiting order, and early termination of both the integer- and fractional-pel motion estimation processes. Experimental results show that the proposed algorithm achieves on average 72.9% time savings, for limited coding losses.
A probabilistic model for flood detection in video sequences

In this paper we propose a new image event detection method for identifying flood in videos. Traditional image based flood detection is often used in remote sensing and satellite imaging applications. In contrast, the proposed method is applied for retrieval of flood catastrophes in newscast content, which present great variation in flood and background characteristics, depending on the video instance. Different flood regions in different images share some common features which are reasonably invariant to lightness, camera angle or background scene. These features are texture, relation among color channels and saturation characteristics. The method analyses the frame-to-frame change in these features and the results are combined according to the Bayes classifier to achieve a decision (i.e. flood happens, flood does not happen). In addition, because the flooded region is usually located around the lower and middle parts of an image, a model for the probability of occurrence of flood as a function of the vertical position is proposed, significantly improving the classification performance. Experiments illustrated the applicability of the method and the improved performance in comparison to other techniques.
Fast HEVC coding using reverse CU visiting

The High Efficiency Video Coding (HEVC) standard makes use of flexible partitioning to achieve high compression ratios. Each frame is divided in Coding Tree Units (CTUs) of fixed size, which are further partitioned into Coding Units (CUs) following a recursive quadtree structure. Typically, CUs at each level of recursion are tested to select the optimal coding configuration, hence this process is extremely demanding in terms of computational complexity. In this paper, a method to reduce complexity of HEVC quadtree configuration selection and mode decision is presented, based on a reverse bottom-to-top visiting order of CUs in the quadtree. By visiting smallest CUs first, information can be extracted to make decisions on larger CUs. The encoder adaptively selects whether a CTU is encoded using the reverse CU visiting, allowing for considerably faster encoding under all conditions. Experimental results show that the algorithm achieves on average 21% speedups over previous state-of-the-art fast HEVC algorithms, and up to 36% for some sequences, at very limited efficiency losses.
Different non-linear diffusion filters combined with triangle method used for noise removal from polygonal shapes

A two-step process for removing noise from polygonal shapes is presented in this study. The authors present a polygonal shape as its turning function and then apply a non-linear diffusion filter and a triangle method on it. In the first step the authors apply several different non-linear diffusion filters on the turning function and compare the performance of these filters later. Non-linear diffusion filters identify dominant vertices in a polygon and remove those vertices that are identified as noise or irrelevant features. The vertices in the turning function which diffuse until the sides that immediately surround them approach the same turning function are identified as noise and removed. The vertices that are enhanced are preserved without changing their coordinates and they are identified as dominant ones. After the authors carry this process as far as it will go without introducing noticeable shape distortion, and switch to the triangle method for further removal of vertices that are to be treated as noise. In the second step the authors remove the vertices that form the smallest area triangles. The authors submit experimental results of the tests that demonstrate that this two-step process successfully removes vertices that should be dismissed as noise while preserving dominant vertices that can be accepted as relevant features and give a faithful description of the shape of the polygon. In experimental tests of this procedure the authors demonstrate successful removal of noise and excellent preservation of shape, thanks to appropriate emphasis of dominant vertices.
Enriched access to digital audiovisual content

This paper presents access engine to digital audio and related content developed under IST FP6 project EASAIER. The main driving force for the project was the lack of qualitative solutions for access to digital sound archives. An innovative remote access system which extends beyond standard content management and retrieval systems, addresses a range of issues identified, such as inconsistent formats of archived materials with related media often in separate collections and related metadata given in non-standard specialist format, incomplete or even erroneous. The system focuses on sound archives, libraries, museums, broadcast archives, and music schools, but the tools may be used by anyone interested in accessing archived material; amateur or professional, regardless of the material involved. The system functionalities; enhanced cross media retrieval, multi-media synchronisation, audio and video processing, analysis and visualisation tools, enable the user to experiment with the materials in exciting new ways.
Examining visual attention: A method for revealing users' interest for images on screen

This report tries to measure users' interest in images that appear on the screen by monitoring their attention via eye-tracking. Our Gaze Inference System analyzes the gaze-movement features to assign a user interest level (UIL) from 0 to 1 to every image that appears on the screen. Because the properties of the gaze features for every user are different from others, the framework is designed to be user adaptive. This framework is capable of building a new processing system for every new user that starts experiencing it. The generated UILs can be used in different scenarios that use the users' interest as an input. The developed framework produces promising and reliable results where 10% of the target images that the users were searching for received UILs over 0.8 with precision of 100%.
An Object- and User-Driven System for Semantic-Based Image Annotation and Retrieval

In this paper, a system for object-based semi-automatic indexing and retrieval of natural images is introduced. Three important concepts underpin the proposed system: a new strategy to fuse different low-level content descriptions; a learning technique involving user relevance feedback; and a novel object based model to link semantic terms and visual objects. To achieve high accuracy in the retrieval and subsequent annotation processes several low-level image primitives are combined in a suitable multifeatures space. This space is modelled in a structured way exploiting both low-level features and spatial contextual relations of image blocks. Support vector machines are used to learn from gathered information through relevance feedback. An adaptive convolution kernel is defined to handle the proposed structured multifeature space. The positive definite property of the introduced kernel is proven, as essential condition for uniqueness and optimality of the convex optimization in support vector machines. The proposed system has been thoroughly evaluated and selected results are reported in this paper
Live scalable video streaming on peer-to-peer overlays with network coding

Scalable video coding is a paradigm that allows partial decoding of the video stream at reduced resolution, frame-rate or quality, adapting to display requirements and reception conditions of heterogeneous receivers. Transmission of scalable data with prioritization enhances the transmission performance, reducing the sensitivity to network congestions and exploiting the multirate characteristic of scalable coding. Network coding is a novel transmission technique that allows intermediate network nodes to perform coding operations on the information in transit, as opposed to traditional routing. This yields to maximization of the transmission rate and encoding of the information with spatial diversity. We employ an overlay network that uses network coding and delivers scalable video with prioritization. We test the performance of the scalable streaming against a nonscalable system, when the upload bandwidth of the nodes is not known.
HEVC coding optimisation for Ultra High Definition television services

Ultra High Definition TV (UHDTV) services are being trialled while UHD streaming services have already seen commercial débuts. The amount of data associated with these new services is very high thus extremely efficient video compression tools are required for delivery to the end user. The recently published High Efficiency Video Coding (HEVC) standard promises a new level of compression efficiency, up to 50% better than its predecessor, Advanced Video Coding (AVC). The greater efficiency in HEVC is obtained at much greater computational cost compared to AVC. A practical encoder must optimise the choice of coding tools and devise strategies to reduce the complexity without affecting the compression efficiency. This paper describes the results of a study aimed at optimising HEVC encoding for UHDTV content. The study first reviews the available HEVC coding tools to identify the best configuration before developing three new algorithms to further reduce the computational cost. The proposed optimisations can provide an additional 11.5% encoder speed-up for an average 3.1% bitrate increase on top of the best encoder configuration.
Query refinement and user relevance feedback for contextualized image retrieval

The motivation of this paper is to enhance the user perceived precision of results of content based information retrieval (CBIR) systems with query refinement (QR), visual analysis (VA) and relevance feedback (RF) algorithms. The proposed algorithms were implemented as modules into K-Space CBIR system. The QR module discovers hypernyms for the given query from a free text corpus (such as Wikipedia) and uses these hypernyms as refinements for the original query. Extracting hypernyms from Wikipedia makes it possible to apply query refinement to more queries than in related approaches that use static predefined thesaurus such as Wordnet. The VA Module uses the K-Means algorithm for clustering the images based on low-level MPEG - 7 Visual features. The RF Module uses the preference information expressed by the user to build user profiles by applying SOM- based supervised classification, which is further optimized by a hybrid Particle Swarm Optimization (PSO) algorithm. The experiments evaluating the performance of QR and VA modules show promising results.
Knowledge modeling for privacy-by-design in smart surveillance solution

As new information and communications systems are being equipped with more aggressive capabilities to enable smart surveillance, individuals' private and ethical data is more exposed to potential threats. Consequently, the attention of researchers and policy makers has become increasingly focused on controlling the emerging threats to privacy. In order to ensure that a surveillance system framework complies with the legal, ethical and privacy requirements of the law, in this paper we present a Surveillance Ontology extending the SKOS foundational ontology. The fundamental principles of privacy-by-design (PbD) demand that the surveillance framework consider data minimization, user control, accountability and data separation. Hence, the objective of this ontology is to translate the high-level linguistic rules into the information that can be processed and used to assess the compliance of the video analysis module with the rules defined.
Frequency-Domain Intra Prediction Analysis and Processing for High-Quality Video Coding

Most of the advances in video coding technology focus on applications that require low bitrates, for example, for content distribution on a mass scale. For these applications, the performance of conventional coding methods is typically sufficient. Such schemes inevitably introduce large losses to the signal, which are unacceptable for numerous other professional applications such as capture, production, and archiving. To boost the performance of video codecs for high-quality content, better techniques are needed especially in the context of the prediction module. An analysis of conventional intra prediction methods used in the state-of-the-art High Efficiency Video Coding (HEVC) standard is reported in this paper, in terms of the prediction performance of such methods in the frequency domain. Appropriately modified encoder and decoder schemes are presented and used for this paper. The analysis shows that conventional intra prediction methods can be improved, especially for high frequency components of the signal which are typically difficult to predict. A novel approach to improve the efficiency of high-quality video coding is also presented in this paper based on such analysis. The modified encoder scheme allows for an additional stage of processing performed on the transformed prediction to replace selected frequency components of the signal with specifically defined synthetic content. The content is introduced in the signal using feature-dependent lookup tables. The approach is shown to achieve consistent gains against conventional HEVC with up to -5.2% coding gains in terms of bitrate savings.
Computational experiments with area-based stereo for image-based rendering

In this paper two disparity estimators with different complexity degrees are described and used to examine how much disparity inaccuracies influence image rendering quality. The objective of this study is to design software-based image synthesis in real-time on conventional PC platforms. Basically, this work looks at the opposite end of the cost-complexity curve by making very restrained demands to the disparity estimator. It is empirically shown that in many cases the effect of disparity accuracy in the quality of virtual views is almost imperceptible and that for many applications requiring real-time processing reasonably good results can be achieved with less computational cost.
Two-pass rate control for UHDTV delivery with HEVC

Rate control has been regarded as an indispensable video coding tool for virtually any application involving video transmission. With the advent of many flexible tools introduced in the current state-of-the-art High Efficiency Video Coding (HEVC) standard, previous Rate-Distortion (RD) models used for rate control become insufficiently accurate. To overcome this issue, a new RD model has been recently proposed based on a robust correspondence between the rate and Lagrange multiplier λ. However, existing methods based on this model tend to perform sub-optimally after the scene change. In this paper, a two-pass rate control method is proposed, targeting Ultra High Definition Television (UHDTV) applications. In the first pass, a fast encoder with a reduced set of coding tools is used to obtain the data used for rate allocation and model parameter initialisation utilised during the second pass. Multiple encoding steps required to derive this information are avoided with the proposed variable quantization parameter framework. Experimental evaluation showed that the proposed two-pass rate control method achieves on average 4.4% BD-rate loss, compared with variable bit-rate encoding. That significantly outperforms the state-of-the-art HEVC rate control method with an average BD-rate loss of 8.8%.
Video composition by spatiotemporal object segmentation, 3D-structure and tracking

A stereo vision based system for composition of natural and computer generated images is presented. The system focuses on the solution of four essential tasks in computer vision: disparity estimation, object segmentation, modeling and tracking. These tasks are performed in direct interaction with each other using novel and available multiview analysis techniques and standard computer graphics algorithms. The system is assessed by processing natural video sequences. Selected results are reported.
Bayesian learning and reasoning for context exploitation in visual information retrieval

This paper presents a semantic context inference approach on the basis of a multi-feature based visual information retrieval framework. This approach aims at assisting effective retrieval of visual content by exploiting the context information in the digital database. Bayesian networks are used as an inference tool, which can be automatically constructed by learning from the multi-feature similarities and a small amount of training data. The idea is to model potential semantic descriptions of basic semantic concepts in the visual content, the dependencies between them, and the conditional probabilities involved in those dependencies. This information is then used to calculate the probabilities of the effects that those concepts have on each other in order to obtain more precise and meaningful semantic labels for the visual content. However, the proposed method is not restricted to the specific multi- feature based visual information retrieval framework used in this paper. Selected experimental results are presented to show how the proposed context inference approach could improve the retrieval performance.
Image based rendering using rational filters

Image based rendering is a powerful technique for 3D object visualization and representation. Using this approach, arbitrary object or scene views are generated automatically from a small set of real reference images. An algorithm for image based rendering using stereo image pairs is presented. The main goal is to produce a realistic 3D 'continuous look-around' effect along the stereo baseline. To minimize the distortion of the reconstructed pixels due to undefined disparity values, a nonlinear interpolator is proposed. It uses the sparse available disparity map to generate a dense field that partially reconstructs original disparity edge information, producing a sharper intermediate view. The identification of occluded and non-occluded areas is also used to aid the view synthesis process. A special treatment for occluded image areas is also considered in the proposed technique. Several computer experiments have been conducted to assess the performance of the presented method.
Stereo matching for enhanced telepresence in three-dimensional videocommunications

A robust approach for joint motion and disparity estimation in stereo sequences to synthesize arbitrary intermediate views is presented. The improved concept for stereo image analysis is based on a modified block matching algorithm. In which a cost function consisting of area-based correlation together with an appropriately weighted temporal smoothness term is applied. A confidence measure to evaluate the reliability of estimated correspondences is introduced. In occluded image areas and image points with unreliable motion or disparity assignments, considerable improvements are obtained applying an edge-assisted vector interpolation strategy. Two different image synthesis concepts are presented as well. The reported approach is verified by processing a set of sequences taken with stereo cameras having large interaxial distances. Computer simulations show that telepresence illusion with continuous motion parallax and good image quality can be obtained using the methods presented.
User Requirements for Multimedia Indexing and Retrieval of Unedited Audio-Visual Footage - RUSHES

Multimedia analysis and reuse of raw un-edited audio visual content known as rushes is gaining acceptance by a large number of research labs and companies. A set of research projects are considering multimedia indexing, annotation, search and retrieval in the context of European funded research, but only the FP6 project RUSHES is focusing on automatic semantic annotation, indexing and retrieval of raw and un-edited audio-visual content. Even professional content creators and providers as well as home-users are dealing with this type of content and therefore novel technologies for semantic search and retrieval are required. As a first result of this project, the user requirements and possible user-scenarios are presented in this paper. These results lay down the foundation for the research and development of a multimedia search engine particularly dedicated to the specific needs of the users and the content.
A generic video analysis and segmentation system

A generic video analysis system for supervised and unsupervised segmentation is described. The idea behind the presented concept is to integrate different advanced segmentation techniques to obtain a robust, efficient and modular segmentation system for natural video and still images. The system entails several independent modules. Each one of these modules encapsulates a complete video processing technique The intermediate results obtained from each single module are merged and further processed by a set of intelligent rules to achieve a highly accurate final segmentation. The modular structure of the system allows it to be extended continuously and with ease by adding new independent modules. The intermediate segmentation results of newly added modules are linked to the other system results via the rule processor. A user friendly graphical interface (GUI) is also provided. The functionality of the GUI is twofold: it serves as input interface to pass processing parameters to the system and as semi-automatic segmentation tool for user interaction and manually refinement of automatically generated segmentation masks. Selected results obtained with the current version of the video analysis system are reported.