nils-herrmann/clean_scirepeval huggingface dataset:
UHD Video Super-Resolution Using Low-Rank and Sparse Decomposition

Sparse coding-based algorithms have been successfully applied to the single-image super resolution problem. Conventional multi-image super-resolution (SR) algorithms incorporate auxiliary frames into the model by a registration process using subpixel block matching algorithms that are computationally expensive. This becomes increasingly important as super-resolving UHD video content with existing sparse-based SR approaches become less efficient. In order to fully utilize the spatio-temporal information, we propose a novel multi-frame video SR approach that is aided by a low-rank plus sparse decomposition of the video sequence. We introduce a group of pictures structure where we seek a rank-1 low-rank part that recovers the shared spatiotemporal information among the frames in the group of pictures (GOP). Then we super-resolve the low-rank frame and sparse frames separately. This assumption results in significant time reductions, as well as surpassing state-of-the-art performance both qualitatively and quantitatively.
Proactive Prioritized Mixing of Scalable Video Packets in Push-Based Network Coding Overlays

Network coding applied to end-system multicast is a viable solution for a multitude of issues related to on-demand video streaming. End-system multicast on network overlays is a desirable option for relieving the content server from bandwidth bottlenecks and computational load as well as allowing decentralized allocation of resources for terminals with different computational and display capabilities. Network coding has proven to be able to solve many issues related to content distribution and rate allocation on end-system overlays, one of them being the coupon-collection problems typical of P2P networks. In this paper we present a scalable video streaming system based on end-system multicast, where the network coding technique with push-based content distribution is extended to perform prioritized streaming with error and congestion control. We identify a problem of layer and rate selection due to the difficulty in estimating the max-flow in end-system overlays, which, with many previously proposed techniques, yields to bandwidth inefficiencies. We present a mechanism for selecting and encoding chunks of scalable video prior to forwarding, and a peer-selection technique, targeting increased efficiency with the available bandwidth, that also improves quality and continuity of service with better use of network rate. Simulated tests results are presented to prove the performance of our system.
A robust correlation measure for correspondence estimation

A median correlation for the estimation of corresponding points in stereovision is proposed. It is based on the normalised correlation coefficient using the median instead of the mean. Its performance appears to be superior than conventional correlation specially in depth discontinuities image areas. This conclusion is derived from an empirical evaluation in which the proposed correlation is compared with the normalised correlation coefficient and the sum of absolute difference. The results show that the median correlation produces higher scores and lower estimation errors.
A Probabilistic Approach for Vision-Based Fire Detection in Videos

Automated fire detection is an active research topic in computer vision. In this paper, we propose and analyze a new method for identifying fire in videos. Computer vision-based fire detection algorithms are usually applied in closed-circuit television surveillance scenarios with controlled background. In contrast, the proposed method can be applied not only to surveillance but also to automatic video classification for retrieval of fire catastrophes in databases of newscast content. In the latter case, there are large variations in fire and background characteristics depending on the video instance. The proposed method analyzes the frame-to-frame changes of specific low-level features describing potential fire regions. These features are color, area size, surface coarseness, boundary roughness, and skewness within estimated fire regions. Because of flickering and random characteristics of fire, these features are powerful discriminants. The behavioral change of each one of these features is evaluated, and the results are then combined according to the Bayes classifier for robust fire recognition. In addition, a priori knowledge of fire events captured in videos is used to significantly improve the classification results. For edited newscast videos, the fire region is usually located in the center of the frames. This fact is used to model the probability of occurrence of fire as a function of the position. Experiments illustrated the applicability of the method.
Global-to-local oriented perception on blurry visual information

A system for perception on blurry visual information is described and validated. Essential capture in global pathway and saliency highlight in local pathway are integrated for low-quality visual information applications. The system can differentiate scenes from various semantic meanings using a spatial layout of context information, which capture the "essential" of the scene. The system can even further discriminate the contents contained in the scene via local highlight. Distinct from previous frameworks, the system presents the entire scheme of being biologically plausible and application efficiency, thus offering a straightforward platform for rapid analysis and interpretation on blurry visual information.
Global-to-Local Oriented Rapid Scene Perception

An approach for rapid scene perception from global layout to local features is presented. The representation of a complex scene is initially built from a collection of global features from which properties related to the spatial layout of the scene and its semantic category can be estimated. The rapid perception of natural scenes relies partly on a global estimation of the features contained in the scene. Further analysis on the local essential areas is deployed on the basis. Such kind of integrated model guarantees the interactive processing between local and global features, thus enabling low-level features to initiate scene perception and categorization efficiently.
Inter-Prediction Optimizations for Video Coding Using Adaptive Coding Unit Visiting Order

The flexible partitioning scheme and increased number of prediction modes in the high efficiency video coding (HEVC) standard are largely responsible for both its high compression efficiency and computational complexity. In typical HEVC encoder implementations, coding units (CUs) in a coding tree unit (CTU) are visited from top to bottom at each level of recursion to select the optimal coding configuration. In this paper, a novel approach is presented in which CUs in a CTU can be adaptively visited also ina reverse, bottom to top visiting order. This reverse CU (RCU) visiting order allows for different algorithmic optimizations for further complexity reduction of many HEVC encoding steps, especially under challenging conditions, such as highly textured or fast moving content. In particular, algorithms to reduce complexity of HEVC depth selection, mode decision, and inter-prediction are presented here based on the coding information obtained from higher depths when using the RCU visiting order. Experimental results show that enabling different stages of the proposed algorithm can achieve average speedups from 16.3% to 36.6% compared to fast reference HEVC implementation with pre-built speed-ups enabled (up to 51.2% in some cases), for 0.3% to 2.2% BD-rate penalty.
Reading Users' Minds From Their Eyes: A Method for Implicit Image Annotation

This paper explores the possible solutions for image annotation and retrieval by implicitly monitoring user attention via eye-tracking. Features are extracted from the gaze trajectory of users examining sets of images to provide implicit information on the target template that guides visual attention. Our Gaze Inference System (GIS) is a fuzzy logic based framework that analyzes the gaze-movement features to assign a user interest level (UIL) from 0 to 1 to every image that appeared on the screen. Because some properties of the gaze features are unique for every user, our user adaptive framework builds a new processing system for every new user to achieve higher accuracy. The generated UILs can be used for image annotation purposes; however, the output of our system is not limited as it can be used also for retrieval or other scenarios. The developed framework produces promising and reliable UILs where approximately 53% of target images in the users' minds can be identified by the machine with an error of less than 20% and the top 10% of them with no error. We show in this paper that the existing information in gaze patterns can be employed to improve the machine's judgement of image content by assessment of human interest and attention to the objects inside virtual environments.
Document Image Processing for Paper Side Communications

This paper proposes the use of higher order statistical moments in document image processing to improve the performance of systems which transmit side information through the print and scan channel. Examples of such systems are multilevel 2-D bar codes and certification via text luminance modulation. These systems print symbols with different luminances, according to the target side information. In previous works, the detection of a received symbol is usually performed by evaluating the average luminance or spectral characteristics of the received signal. This paper points out that, whenever halftoning algorithms are used in the printing process, detection can be improved by observing that third and fourth order statistical moments of the transmitted symbol also change, depending on the luminance level. This work provides a thorough analysis for those moments used as detection metrics. A print and scan channel model is exploited to derive the relationship between the modulated luminance level and the higher order moments of a halftone image. This work employs a strategy to merge the different moments into a single metric to achieve a reduced detection error rate. A transmission protocol for printed documents is proposed which takes advantage of the resulting higher robustness achieved with the combined detection metrics. The applicability of the introduced document image analysis approach is validated by comprehensive computer simulations.
Enabling Geometry-Based 3-D Tele-Immersion With Fast Mesh Compression and Linear Rateless Coding

3-D tele-immersion (3DTI) enables participants in remote locations to share, in real time, an activity. It offers users interactive and immersive experiences, but it challenges current media-streaming solutions. Work in the past has mainly focused on the efficient delivery of image-based 3-D videos and on realistic rendering and reconstruction of geometry-based 3-D objects. The contribution of this paper is a real-time streaming component for 3DTI with dynamic reconstructed geometry. This component includes both a novel fast compression method and a rateless packet protection scheme specifically designed towards the requirements imposed by real time transmission of live-reconstructed mesh geometry. Tests on a large dataset show an encoding speed-up up to ten times at comparable compression ratio and quality, when compared with the high-end MPEG-4 SC3DMC mesh encoders. The implemented rateless code ensures complete packet loss protection of the triangle mesh object and a delivery delay within interactive bounds. Contrary to most linear fountain codes, the designed codec enables real-time progressive decoding allowing partial decoding each time a packet is received. This approach is compared with transmission over TCP in packet loss rates and latencies, typical in managed WAN and MAN networks, and heavily outperforms it in terms of end-to-end delay. The streaming component has been integrated into a larger 3DTI environment that includes state of the art 3-D reconstruction and rendering modules. This resulted in a prototype that can capture, compress transmit, and render triangle mesh geometry in real-time in realistic internet conditions as shown in experiments. Compared with alternative methods, lower interactive end-to-end delay and frame rates over three times higher are achieved.
Enhanced multi-view dancing videos synchronisation

This paper describes a system for automatically synchronising multi-view video sequences of Salsa dancing recorded with multimodal capturing platform. The multimodal capturing setup consists of audiovisual streams along with depth maps and inertial measurements. Part of the dataset was video sequences captured from machine vision cameras and Microsoft Kinect sensor that were not temporal synchronised during the capturing stage. As an essential step, we proposed efficient solutions for synchronisation of these data based on co-occurrence appearance changes. In order to improve the accuracy, the proposed system employed state-of-art body detection and tracking algorithm to obtain Region of Interest, within which the appearance changes are analysed. The accurately synchronised video set can then be further analysed and augmented for visualisation and evaluation of dancing performance.
Fast multiframe motion estimation for surveillance videos

We propose a fast multiple reference frames based motion estimation technique for surveillance videos. In the very first reference frame of each motion vector search, successive elimination algorithm is used to find the best motion vector in the search window. For the remaining reference frames, a difference between current and previous reference frames is performed to indentify candidate matching blocks in the current reference frame. Different block matching strategies are proposed to find the optimum motion vector. Experimental evaluation shows that significant reduction in computational complexity can be achieved by applying the proposed strategy.
Minimal Hough Forest training for pattern detection

This paper assesses Hough Forest configuration parameters with respect to their impact on runtime performance and precision of pattern detection, without large-scale training. The Hough Forest is trained using a very small training set of data and parameters are tuned in a number of experiments, assessing the impact on pattern detection accuracy. A novel method to improve training performance and precision by adaptive selection of the patch size and calculating the used number of patches is introduced. Results are presented using challenging street-scene videos, and demonstrate that the proposed method improves precision and performance over general-purpose Hough Forest parameters used in the literature.
Reconstruction for 3D immersive virtual environments

The future of tele-conferencing is towards multi-party 3D Tele-Immersion (TI) and TI environments that can support realistic inter-personal communications and virtual interaction among participants. In this paper, we address two important issues, pertinent to TI environments. The paper focuses on techniques for the real-time, 3D reconstruction of moving humans from multiple Kinect devices. The off-line generation of real-life 3D scenes from visual data, captured by non-professional users is also addressed. Experimental results are provided that demonstrate the efficiency of the methods, along with an example of mixing real with virtual in a shared space.
Transcoding From Hybrid Nonscalable to Wavelet-Based Scalable Video Codecs

Scalable video coding (SVC) enables low complexity adaptation of compressed video, providing an efficient solution for content delivery through heterogeneous networks and to diverse displays. However, legacy video and most commercially available content capturing devices use conventional nonscalable coding, e.g., H.264/AVC. This paper proposes an efficient transcoder from H.264/AVC to a wavelet-based SVC. It aims at exploiting the advantages offered by fine granularity SVC technology when dealing with conventional coders and legacy video. The proposed transcoder was developed to cope with important functionalities of H.264/AVC, such as flexible reference frame (RF) selection. It is able to work with different coding configurations of H.264/AVC, including IPP or IBBP with multiple RFs. Moreover, many of the techniques presented in this paper are generic in the sense that they can be used for transcoding with many popular wavelet-based and hybrid-based video coding architectures. To reduce the transcoder's complexity, motion information and residual data extracted from a compressed H.264/AVC stream are exploited. Experimental results show a very good performance of the proposed transcoder in terms of decoded video quality and system complexity.
Evaluation of Techniques for Modeling of Layered Motion Structure

Motion information scalability is important for scalable bit-stream adaptation on low bit-rates, when motion rate occupies a significant portion of the total bit-rate. This type of scalability can be achieved by layered representation of motion block partitioning and predictive coding of associated motion vectors across these layers. So far, several approaches for creating layered motion structure targeting quality scalability have been proposed and in this paper their accuracy is evaluated. For that purpose optimal motion models have been found. It has been shown that simple evaluation of reconstruction error at the encoder side improves suboptimal modeling techniques.
Efficient and accurate image based camera registration

A technique for efficient and accurate camera registration based on stereo image analysis is presented. Initially, few correspondences are estimated with high accuracy using a probabilistic relaxation technique. Accuracy is achieved by considering the continuous approximations of selected image areas using second order polynomials and a relaxation rule defined according to the likelihood that estimates obey stereoscopic constraints. The extrinsic camera parameters are then obtained using a novel efficient and robust approach derived from the classic eight point algorithm. Efficiency is achieved by solving a parametric linear optimization problem rather than a nonlinear one as more conventional methods attempt to do. Robustness is obtained by applying two novel strategies: normalization of the initial data via a simple but efficient diagonal scaling approach, and regularization of the underlying linear parametric optimization problem using meaningful constraints. The performance of the presented methods is assessed in several computer experiments using natural video data.
A fast error protection scheme for transmission of embedded coded images over unreliable channels and fixed packet size

Joint source-channel coding enables efficient transmission of embedded bitstreams over unreliable channels. We address channels with fixed packetisation and decoding without or with minimal delays. The computation of an optimal protection scheme for such bitstreams is generally an exponential complexity problem and hence not applicable in a straightforward implementation. Using the rate-distortion characteristics of the source bitstream and the dynamic programming approach we construct an efficient unequal error protection scheme for the predefined channels. Our algorithm is of linear complexity and thus applicable in real time scenarios.
A rule-based video annotation system

A generic system for automatic annotation of videos is introduced. The proposed approach is based on the premise that the rules needed to infer a set of high-level concepts from low-level descriptors cannot be defined a priori. Rather, knowledge embedded in the database and interaction with an expert user is exploited to enable system learning. Underpinning the system at the implementation level is preannotated data that dynamically creates signification links between a set of low-level features extracted directly from the video dataset and high-level semantic concepts defined in the lexicon. The lexicon may consist of words, icons, or any set of symbols that convey the meaning to the user. Thus, the lexicon is contingent on the user, application, time, and the entire context of the annotation process. The main system modules use fuzzy logic and rule mining techniques to approximate human-like reasoning. A rule-knowledge base is created on a small sample selected by the expert user during the learning phase. Using this rule-knowledge base, the system automatically assigns keywords from the lexicon to nonannotated video clips in the database. Using common low-level video representations, the system performance was assessed on a database containing hundreds of broadcasting videos. The experimental evaluation showed robust and high annotation accuracy. The system architecture offers straightforward expansion to relevance feedback and autonomous learning capabilities.
Scalable object-based image retrieval

Digital visual libraries have currently available huge amounts of content in unstructured, non-indexed form. Since these collections keep growing fast, retrieving specific images is becoming extremely difficult. It is too slow to linearly search all the stored feature vectors to find those that satisfy the query criteria. Scalability is crucial for an image retrieval system to be practical and realistic. In this paper a simple hierarchical object descriptor scheme, which is compact, flexible, and inherently suited for hierarchical search, is described. By integrating a suitable segmentation algorithm into the descriptor generation schema, the proposed approach becomes object oriented. Basically, features used for the extraction of image regions belonging to single physical objects are used in the definition of object descriptors. The resulting technique generates compact scalable descriptions for each object in the database. Experimental results show the performance of the presented schema in terms of accuracy and scalability.
Efficient image selection for concept learning

In semantic-based image classification, learning concepts from features is an ongoing challenge for researchers and practitioners in different communities such as pattern recognition, machine learning and image analysis, among others. Concepts are used to add knowledge to the image descriptions linking high- and low-level numerical interpretation of the image content. Augmented descriptions are useful to perform more 'intelligent' processing on large-scale image databases. The semantic component casts the classification into the supervised or learning-from-examples paradigm, in which the classifier obtains knowledge by generalising specific facts presented in a number of design samples (or training patterns). Consequently, selection of suitable samples becomes a critical design step. The introduced framework exploits the capability of support vector classifiers to learn from relatively small number of patterns. Classifiers make decisions based on low-level descriptions containing only some image content information (e.g. colour, texture, shape). Therefore there is a clear drawback in collecting image samples by just using random visual observation and ignoring any low-level feature similarity. Moreover, this sort of approach set-up could lead to sub-optimal training data sets. The presented framework uses unsupervised learning to organise images based on low-level similarity, in effort to assist a professional annotator in picking positive and negative samples for a given concept. Active learning to refine the classifier model follows this initial design step. The framework shows promising results as an efficient approach in selecting design samples for semantic image description and classification.