nils-herrmann/clean_scirepeval huggingface dataset:
Optimised Motion Estimation for Robust Video Coding in Packet Loss Environment

An improved motion estimation method for robust video coding is proposed. It is designed to enhance packet loss resilience in lossy environments. The proposed motion estimation is based on an end-to-end rate-distortion optimisation framework. Initially, the end-to-end reconstructed distortion is efficiently estimated considering the distortion caused by quantisation, error propagation, and error concealment. Then, the total bit rate for coding the residual is estimated using a quadratic rate-distortion model. The results are incorporated into the Lagrangian optimisation during motion estimation. Optimised motion vectors are selected in the sense of coding efficiency and robustness. A comparative evaluation with conventional robust video coding techniques was conducted. The experimental results demonstrate a superior performance of the proposed method.
Detection and enhancement of moving objects in surveillance centric coding

The coexistence of multiple cameras, especially in wireless video surveillance systems imposes severe constraints in terms of computational resources, power supply and bandwidth. These limitations hamper coding and transmission of high-quality video streams. In this paper, we propose a new approach for video coding and transmission of surveillance video. It integrates the coding sub-system and the motion detection module to enhance the quality of moving objects in low-bit rate streams. At the sender side, videos are downsampled and compressed. At the decoder, the video is upsampled and the foreground quality is enhanced by detecting meaningful edges of moving objects via the Hough transform. The quality of the background is also enhanced through progressive update of the background model.
A spectral technique for image clustering

Given an image i* and an image database V containing and unknown number of image classes, in this paper we propose a technique for finding the class A of V that contains i*. To solve this (1+x)-class clustering problem a novel spectral “asymmetric” formulation of the problem is introduced: The Asymmetric Cut. It permits the extraction of the required class regardless other classes in the data base. The actual goal is to find a spectral formulation of the (1+x)-class clustering problem and to propose an efficient numerical implementation of the approach for large image database. The proposed method finds a subset A that maximizes the similarities within the chosen cluster but it does not involve affinities or dissimilarities among remaining unknown clusters in the database. Asymmetric cuts seamlessly lead to a spectral representation which can be solved by finding the critical points of the corresponding Rayleigh quotient. Following the underlying spectral theoretical approach the critical points correspond to the eigenvectors of an affinity matrix derived from pair-wise similarities involving information related to a single image i* representing the image class of concern. Selected results from experimental evaluation are presented.
A Multi-Pattern Search Algorithm for Block Motion Estimation in Video Coding

In this paper, we propose a novel multi-pattern based search technique, TCon search, for fast block matching motion estimation. It starts with small cross shaped and small triangular shaped patterns. Afterwards, based on the previous step optimal motion vector, the search pattern for next step is selected. Except first and last step, each search step considers only three points thus reducing the number of search points significantly. Experimental results demonstrate that the proposed TCon search algorithm performs better than the well-known diamond search (DS) and cross-diamond-hexagonal search (CDHS) algorithms. Compared with the DS algorithm, the proposed TCon search performs up to 2.67 times faster in terms of search point computation and up to 1.67 times faster than CDHS algorithm while comparable quality of reconstructed sequence is maintained.
End-to-End Rate-Distortion Optimized Motion Estimation

An end-to-end rate-distortion optimized motion estimation method for robust video coding in lossy networks is proposed. In this method the expected reconstructed distortion after transmission and the total bit rate for displaced frame difference are estimated at the encoder. The results are fed into the Lagrangian optimization at the encoder to perform motion estimation. Here the encoder automatically finds an optimized motion compensated prediction by estimating the best trade off between coding efficiency and end-to-end distortion. Computer simulations in lossy channel environments were conducted to assess the performance of the proposed method. A comparative evaluation using other conventional techniques from the literature was also conducted.
Bandwidth-Efficient Packet Scheduling for Live Streaming With Network Coding

Network coding (NC) brings substantial improvements in terms of throughput and delay in collaborative media streaming applications. A key aspect of NC-driven live peer-to-peer streaming is the packet scheduling policy. Indeed, lack of synchronization among peers usually results in significantly redundant packet transmission, which in turn leads to severe bandwidth inefficiencies. In this paper, we address the problem of finding a suitable asynchronous packet scheduling policy that greatly helps to overcome this critical redundant transmission problem. We propose a bandwidth cost minimization technique under a full video packet recovery constraint. In order to add scalability and improved performance, we also further derive a distributed packet scheduling algorithm. Both implementation and analytical considerations of the proposed approaches are described in this paper. Experimental results confirm that the proposed algorithms deliver higher bandwidth efficiency with reduced redundancy and communication overhead rate and, consequently, better quality-of-service in terms of improved video quality and delivery ratio.
Enhanced Inter-Prediction Via Shifting Transformation in the H.264/AVC

Inter-prediction based on block-based motion estimation (ME) is used in most video codecs. The closer the prediction to the target block, the lower the residual, and thus more efficient compression can be achieved. In this paper, a new technique called enhanced inter-prediction (EIP) is proposed to improve the prediction candidates using an additional transformation acting while performing ME. A parametric transformation acts within the coding loop of each block to modify the prediction for each motion vector candidate. The EIP is validated in the particular case of a single-parameter shifting transformation. This paper presents an efficient algorithm to compute the best shift for each prediction candidate and a model to select the optimal prediction based on minimum cost integrating the approach with existing rate-distortion optimization techniques in the H.264/AVC video codec. Results show significant improvements with an average of 6% bit-rate reduction compared to the original H.264/AVC.
On the role of structure in part-based object detection

Part-based approaches in image analysis aim at exploiting the considerable discriminative power embedded in relations among image parts. Nonetheless, learning structural information is not always possible without the availability of a training set of classified parts, and taking into account this additional information can even degrade the performance of the system. In this paper, a discriminative graphical model for object detection is introduced and used in order to analyse and report results on the role of structural information in image classification tasks.
Visual word-based CAPTCHA using 3D characters

A secure login authentication for Web applications is presented. The proposed technique uses a CAPTCHA to discriminate between users and automated programs designed to access Web applications. The novelty of the proposed CAPTCHA resides on the use of 3D characters with 3D boundaries delimited by shadows. The robustness of the proposed CAPTCHA is increased by applying random distortions in each character according to geometric transformation in a 3D space. In particular, shadows obtained from different light effects are used to further enhance security against automatic character recognition tools. Experimental evaluations are reported to confirm the security of the approach. (5 pages)
A low cost hybrid diffusion technique for object segmentation

In this paper a low-complexity nonlinear filtering technique to smooth textures preserving object contours is presented. The approach is based on a hybrid combination of both isotropic and anisotropic recursive filtering. Using only intensity information the segment borders obtained by applying nonlinear filtering do not necessarily coincide with physical object contours, especially in the case of textured objects. To segment images into regions with physical meaning additional information extracted from disparity or motion is used to weight the filter coefficients. The presented technique has been successfully tested in the context of object segmentation of natural scenes and object-based disparity estimation for stereoscopic applications.
Discriminant Pairwise Local Embeddings

This paper introduces Discriminant Pairwise Local Embeddings (DPLE) a supervised dimensionality reduction technique that generates structure preserving discriminant subspaces. This objective is achieved through a convex optimization formulation where Euclidean distances between data pairs that belong to the same class are minimized, while those of pairs belonging to different classes are maximized. These pairwise relations are encoded in two matrices and weighted with the data affinity matrix to ensure local structure preservation. The discriminant efficiency of our technique is demonstrated in two popular applications, face and sketch recognition, where DPLE outperforms competitive manifold learning algorithms. A kernelized version of DPLE, that further enhances recognition accuracy, is also explained.
Optimised selection of structure of pictures for video coding

Encoders based on the High Efficiency Video Coding (HEVC) standard consider an input sequence as a succession of slices grouped in Structures of Pictures (SOP). The SOP used while encoding specifies many parameters, such as the coding order of frames, or the reference frames used during inter-prediction. Reference encoders typically make use of a fixed SOP structure of a given size, which is periodically repeated throughout the whole sequence. In this paper, the usage of unconventional SOP structures is first analysed, showing that most sequences benefit from usage of larger SOPs, and that the selection of the optimal SOP is highly content dependent. As a result, an algorithm is proposed to automatically select the optimal SOP size based on a low-complexity texture analysis of neighbouring frames. The algorithm is capable of adaptively changing SOP size during the encoding. Extensive evaluation shows that consistent bit-rate reductions are reported at the same objective quality as an effect of using the proposed algorithm.
Random Methods for Fast Exploration of the Raw Video Material

In this work we propose a visualization and access method for exploring a database of raw video material, where the span of information stored can be huge and difficult to understand at a glance. Developed visual mining tool presents the overview of the hierarchically structured repository in a intuitive way and provides interactive navigational support for the user. Browsing through the un-annotated content is enabled by two complementary interactive methods, a direct selection and a random access, both supported by visual display of the preview nodes. User evaluation aims at demonstrating how the hierarchical random visualization assists the process of accessing and retrieving content relevant to the user.
Turbo Coding Protection of Compressed Domain Watermarking Channel

The aim of compression in general is to reduce information spatial and temporal redundancy from the data. From a watermarking point of view, it reduces the watermark insertion space and power of the embedded signal. This paper describes a watermarking scheme based on the spread spectrum paradigm with capacity enhancement using a state-of-the-art error correction technique - turbo coding. A new watermark composition with random spreading of watermarking bits through the video frames, amplitude adjustment in DCT domain and bit-rate preserving is proposed. However, it was essential to introduce an error correction technique in order to achieve reasonable capacity levels and robustness. In addition, experimental results on perceptibility and robustness on transcoding are presented
Semi-automatic image annotation using frequent keyword mining

Research in content-based image retrieval is an expanding discipline with an accelerated growth in the last ten years. Advances in telecommunications and the huge demand of visual information on Internet and mobile devices is occupying the attention of the researchers in developing efficient systems to ease the task of useful visual information retrieval by the users. We present a semiautomatic image annotation process using the low-level image descriptor fuzzy color signature to extract the most similar images from an annotated database and frequent pattern mining to select the candidates keywords for annotating the new image. The idea is aimed at establishing a bridge between visual data and their interpretation using a weak semantic approach.
A robust and efficient scale-space based metric for the evaluation of MPEG-4 VOPs

New MPEG-4 functionalities require the segmentation of input video into different layers (VOPs). Usually, these layers contain arbitrarily shaped objects representing meaningful content of the video stream. With the introduction of the new functionalities in MPEG4, the need of objective and subjective assessment of segmented image quality has emerged. In this paper we introduce an efficient and reliable metric to evaluate segmentation results by comparing them with a given ground truth. Beyond this application the proposed technique can be used for real-time shape description and retrieval in the context of the emerging MPEG7, as well as in general pattern recognition tasks. Selected results obtained by using this metric within these application areas are reported.
Improved combined intra prediction for higher video compression efficiency

Intra prediction is an important component of video compression. The High Efficiency Video Coding (HEVC) standard supports advanced Intra prediction tools to achieve remarkable Intra coding performance. However, higher compression efficiency performance may be achieved using Combined Intra Prediction (CIP). CIP consists in combining conventional Intra reference samples, with samples extracted from within the current block being encoded. In this paper, an improvement to the original CIP approach is proposed to achieve higher compression efficiency. Due to the fact the encoder does not have access to reconstruction samples while compressing a block, a controlled drift occurs at the block level between the Intra prediction blocks generated at the encoder and decoder when using CIP. The proposed approach aims at reducing such a drift, hence improving CIP performance. This Improved CIP is able to achieve up to 1.4 % BD-rate savings with respect to the HEVC reference software. The paper also proposes the combination of Improved CIP with an additional tool for enhancing the accuracy of Intra prediction. Up to 1.9 % BD-rate savings can be achieved using this approach.
Foreground Segmentation with Tree-Structured Sparse RPCA

Background subtraction is a fundamental video analysis technique that consists of creation of a background model that allows distinguishing foreground pixels. We present a new method in which the image sequence is assumed to be made up of the sum of a low-rank background matrix and a dynamic tree-structured sparse matrix. The decomposition task is then solved using our approximated Robust Principal Component Analysis (ARPCA) method which is an extension to the RPCA that can handle camera motion and noise. Our model dynamically estimates the support of the foreground regions via a superpixel generation step, so that spatial coherence can be imposed on these regions. Unlike conventional smoothness constraints such as MRF, our method is able to obtain crisp and meaningful foreground regions, and in general, handles large dynamic background motion better. To reduce the dimensionality and the curse of scale that is persistent in the RPCA-based methods, we model the background via Column Subset Selection Problem, that reduces the order of complexity and hence decreases computation time. Comprehensive evaluation on four benchmark datasets demonstrate the effectiveness of our method in outperforming state-of-the-art alternatives.
Content-Adaptive Sketch Portrait Generation by Decompositional Representation Learning

Sketch portrait generation benefits a wide range of applications such as digital entertainment and law enforcement. Although plenty of efforts have been dedicated to this task, several issues still remain unsolved for generating vivid and detail-preserving personal sketch portraits. For example, quite a few artifacts may exist in synthesizing hairpins and glasses, and textural details may be lost in the regions of hair or mustache. Moreover, the generalization ability of current systems is somewhat limited since they usually require elaborately collecting a dictionary of examples or carefully tuning features/components. In this paper, we present a novel representation learning framework that generates an end-to-end photo-sketch mapping through structure and texture decomposition. In the training stage, we first decompose the input face photo into different components according to their representational contents (i.e., structural and textural parts) by using a pre-trained convolutional neural network (CNN). Then, we utilize a branched fully CNN for learning structural and textural representations, respectively. In addition, we design a sorted matching mean square error metric to measure texture patterns in the loss function. In the stage of sketch rendering, our approach automatically generates structural and textural representations for the input photo and produces the final result via a probabilistic fusion scheme. Extensive experiments on several challenging benchmarks suggest that our approach outperforms example-based synthesis algorithms in terms of both perceptual and objective metrics. In addition, the proposed method also has better generalization ability across data set without additional training.
Efficient scalable video transmission based on two-dimensional error protection scheme

An efficient approach for transmission of scalable video over erroneous channel is proposed. The proposed approach jointly optimises the bit allocation between a wavelet-based scalable video coding framework and a forward error correction codes. The forward error correction is based on the two-dimensional error protection scheme which applies unequal error protection in temporal-spatial layers as well as in quality layers. The scheme minimizes the reconstructed video distortion at the decoder subject to a constraint on the overall transmission bit-rate budget with limited complexity. This minimization is achieved by exploiting the combined scalability in temporal-spatial and quality domain. Experimental results clearly demonstrate the superiority of the proposed approach over conventional forward error correction techniques. It also significantly improves the performance of end to end scalable video transmission at all channel bit rates.
Estimating the essential matrix by efficient linear techniques

In the problem of recovering the 3D structure of a scene from its 2D projections, a fundamental low-level computer vision task is the estimation of the epipolar geometry. Accurate estimation of the epipolar geometry uses computationally expensive iteration schemes based on nonlinear algebraic constraints to deal with the ill-posedness of the problem. Linear techniques are computationally efficient but extremely unstable. Theoretical and practical aspects of linear methods are analyzed and fundamental results are derived from the study. Two main causes of instability are considered. The first one refers to the lack of homogeneity in the input data. To deal with this problem, a highly efficient scaling approach is introduced. The optimality of the technique is proven theoretically and heuristically. It is shown that a second source of instability arises from the linear dependency between rows of the matrix of the linear system. The effect of this problem in the estimation of the essential matrix is analyzed. An additional strategy is introduced to overcome this difficulty. This strategy improves the stability and accuracy of the linear approach even further while reducing the computational cost. Numerical experiments to evaluate the effectiveness of the proposed techniques are reported.
Revisiting guided image filter based stereo matching and scanline optimization for improved disparity estimation

In this paper the scanline optimization used for stereo matching, is revisited. In order to improve the performance of this semi-global technique, a new criterion to check depth discontinuity, is introduced. This criterion is defined according to the mean-shift-based image segmentation result. Additionally, this work proposes the employment of a pixel dissimilarity metric for the computation of the cost term, which is then provided to the guided image filter approach to estimate the initial cost volume. The algorithm is tested on the four images of the online Middlebury stereo evaluation benchmark. Moreover, it is tested on 27 additional Middlebury stereo pairs for assessing thoroughly its performance. The extended comparison verifies the efficiency of this work.
Scene geometric recognition from monocular image

In this paper, we propose an approach to detect scene geometrical structure given only one monocular image. Several typical scene geometries are investigated and corresponding models are built. A scene geometry reasoning system is set up based on image statistical features and scene geometric features. This system is able to find best fitting geometric models for most of the images from the benchmark dataset. Scene categorization could reveal important three-dimensional information contained in an image. We demonstrate how this valuable information could be used to reason the depth profile of a specific scene. Planes co-constructing the scene could be detected and located. Experiments have been done to roughly restore the structure of the scene to verify system performance. By our approach, computer could interpret a single image in terms of its geometry straightforwardly, avoiding usual semantically overlapping and deficiency problems.
Optimised Compression Strategy in Wavelet-Based Video Coding using Improved Context Models

Accurate probability estimation is a key to efficient compression in entropy coding phase of state-of-the-art video coding systems. Probability estimation can be enhanced if contexts in which symbols occur are used during the probability estimation phase. However, these contexts have to be carefully designed in order to avoid negative effects. Methods that use tree structures to model contexts of various syntax elements have been proven efficient in image and video coding. In this paper we use such structure to build optimised contexts for application in scalable wavelet-based video coding. With the proposed approach context are designed separately for intra-coded frames and motion-compensated frames considering varying statistics across different spatio-temporal subbands. Moreover, contexts are separately designed for different bit-planes. Comparison with compression using fixed contexts from embedded ZeroBlock coding (EZBC) has been performed showing improvements when context modelling on tree structures is applied.