nils-herrmann/clean_scirepeval huggingface dataset:
Video Copy Detection Using a Soft Cascade of Multimodal Features

In the video copy detection task, it is widely recognized that none of any single feature can work well for all transformations. Thus more and more approaches adopt a set of complementary features to cope with complex audio-visual transformations. However, most of them utilize individual features separately and the final result is obtained by fusing results of several basic detectors. Often, this will lead to low detection efficiency. Moreover, there are some thresholds or parameters to be elaborately tuned. To address these problems, we propose a soft cascade approach to integrate multiple features for efficient copy detection. In our approach, basic detectors are organized in a cascaded framework, which processes a query video in sequence until one detector asserts it as a copy. To fully exert the complementarity of these detectors, a learning algorithm is proposed to estimate the optimal decision thresholds in the cascade architecture. Excellent performance on the benchmark dataset of TRECVid 2011 CBCD task demonstrates the effectiveness and efficiency of our approach.
Can We Beat DDoS Attacks in Clouds?

Cloud is becoming a dominant computing platform. Naturally, a question that arises is whether we can beat notorious DDoS attacks in a cloud environment. Researchers have demonstrated that the essential issue of DDoS attack and defense is resource competition between defenders and attackers. A cloud usually possesses profound resources and has full control and dynamic allocation capability of its resources. Therefore, cloud offers us the potential to overcome DDoS attacks. However, individual cloud hosted servers are still vulnerable to DDoS attacks if they still run in the traditional way. In this paper, we propose a dynamic resource allocation strategy to counter DDoS attacks against individual cloud customers. When a DDoS attack occurs, we employ the idle resources of the cloud to clone sufficient intrusion prevention servers for the victim in order to quickly filter out attack packets and guarantee the quality of the service for benign users simultaneously. We establish a mathematical model to approximate the needs of our resource investment based on queueing theory. Through careful system analysis and real-world data set experiments, we conclude that we can defeat DDoS attacks in a cloud environment.
An efficient surveillance coding method based on a timely and bit-saving background updating model

Recently, it has been proved to be useful for high-efficiency surveillance video coding. In existing works, the modeling background frame often needs to be high-quality encoded so as to achieve a large bit-rate saving. However, the high-quality background frame requires lots of bits in the code stream, so it is infeasible to update the background frame too frequently. Therefore, a better bit-allocation method is desirable to facilitate in-time background updating and bit-saving background coding. In this paper, we firstly build up a background updating model from a detailed analysis of results on surveillance video. Following this, we propose a bit-saving and quality-maintaining background frame coding method. In our method, the background frame can be updated more timely, consequently leading to the better coding efficiency. Experimental results show that our method can achieve more than 15% bit-rate decrease compared with three state-of-art methods.
Guest Editorial Multimedia: The Biggest Big Data

The goal of this special issue is to provide a premier forum for researchers to present their recent research results on multimedia big data. It follows the recent success event—the First IEEE International Conference on Multimedia Big Data (BigMM 2015) that took place at the Chinese National Convention Center in Beijing, China, from April 20–22, 2015 It also provides an important opportunity for multidisciplinary work connecting big data to multimedia computing.
Spike Camera and Its Coding Methods

This paper introduces a spike camera with a distinct video capture scheme and proposes two methods of decoding the spike stream for texture reconstruction. The spike camera captures light and accumulates the converted luminance intensity at each pixel. A spike is fired when the accumulated intensity exceeds the dispatch threshold. The spike stream generated by the camera indicates the luminance variation. Analyzing the patterns of the spike stream makes it possible to reconstruct the picture of any moment which enables the playback of high speed movement.
An Efficient Background Reconstruction Based Coding Method for Surveillance Videos Captured by Moving Camera

With the proliferation of moving surveillance cameras, how to effectively compress videos captured from them is becoming more and more important. One significant characteristic is that, these cameras always go and return cyclically within a limited area. Thus we propose to dynamically build up a background frame for each input frame from a generated panorama background and employ it for a background frame based motion compensation to improve the coding efficiency. For the background reconstruction procedure, we firstly extract limited number of feature point pairs between the robustly searched area in the decoded panorama and the current frame. Afterwards, the global motion transformation matrix is obtained to rectify the searched area into a projective plane of the current frame, and then the reconstructed background is produced. Experiments on six in-door and out-door surveillance videos show that, the background reconstruction based coding method achieves significant performance gain.
Search video action proposal with recurrent and static YOLO

In this paper, we propose a new approach for searching action proposals in unconstrained videos. Our method first produces snippet action proposals by combining state-of-the-art YOLO detector (Static YOLO) and our regression based RNN detector (Recurrent YOLO). Then, these short action proposals are integrated to form final action proposals by solving two-pass dynamic programming which maximizes actioness score and temporal smoothness concurrently. Our experimental comparison with other state-of-the-arts on challenging UCF101 dataset shows that our method advances state-of-the-art proposal generation performance while maintaining low computational cost.
Video Copy-Detection and Localization with a Scalable Cascading Framework

For video copy detection, no single audio-visual feature, or single detector based on several features, can work well for all transformations. This article proposes a novel video copy-detection and localization approach with scalable cascading of complementary detectors and multiscale sequence matching. In this cascade framework, a soft-threshold learning algorithm is utilized to estimate the optimal decision thresholds for detectors, and a multiscale sequence matching method is employed to precisely locate copies using a 2D Hough transform and multigranularities similarity evaluation. Excellent performance on the TRECVID-CBCD 2011 benchmark dataset shows the effectiveness and efficiency of the proposed approach.
A coding unit classification based AVC-to-HEVC transcoding with background modeling for surveillance videos

To save the storage and transmission cost, it is applicable now to develop fast and efficient methods to transcode the perennial surveillance videos to HEVC ones, since HEVC has doubled the compression ratio. Considering the long-time static background characteristic of surveillance videos, this paper presents a coding unit (CU) classification based AVC-to-HEVC transcoding method with background modeling. In our method, the background frame modeled from originally decoded frames is firstly transcoded into HEVC stream as long-term reference to enhance the prediction efficiency. Afterwards, a CU classification algorithm which employs decoded motion vectors and the modeled background frame as input is proposed to divide the decoded data into background, foreground and hybrid CUs. Following this, different transcoding strategies of CU partition termination, prediction unit candidate selection and motion estimation simplification are adopted for different CU categories to reduce the complexity. Experimental results show our method can achieve 45% bit saving and 50% complexity reduction against traditional AVC-to-HEVC transcoding.
MPLBoost-based mixture model for effective human detection with Deformable Part Model

The Deformable Part Model has shown high accuracy in tackling certain occlusion or deformations of objects such as cars and bikes. However, as for human category characterized by a larger number of articulated parts and more significant appearance variations, its performance gain is not so remarkable. To address this issue, we propose an MPLBoost-based mixture model which splits data into coherent groups and trains one root classifier for each, resulting in automated selection of discriminative root models and better representation of intra-class variations through visual feature clustering. Based on this boosting framework, multiple complementary features are combined to capture shape, texture and color information. Experimental results demonstrate that the proposed model can achieve an impressive performance improvement, especially in handling larger variations of human poses and viewpoints.
YoTube: Searching Action Proposal Via Recurrent and Static Regression Networks

In this paper, we propose YoTube—a novel deep learning framework for generating action proposals in untrimmed videos, where each action proposal corresponds to a spatial-temporal tube that potentially locates one human action. Most of the existing works generate proposals by clustering low-level features or linking image proposals, which ignore the interplay between long-term temporal context and short-term cues. Different from these works, our method considers the interplay by designing a new recurrent YoTube detector and static YoTube detector. The recurrent YoTube detector sequentially regresses candidate bounding boxes using Recurrent Neural Network learned long-term temporal contexts. The static YoTube detector produces bounding boxes using rich appearance cues in every single frame. To fully exploit the complementary appearance, motion, and temporal context, we train the recurrent and static detector using RGB (Color) and flow information. Moreover, we fuse the corresponding outputs of the detectors to produce accurate and robust proposal boxes and obtain the final action proposals by linking the proposal boxes using dynamic programming with a novel path trimming method. Benefiting from the pipeline of our method, the untrimmed video could be effectively and efficiently handled. Extensive experiments on the challenging UCF-101, UCF-Sports, and JHMDB datasets show superior performance of the proposed method compared with the state of the arts.
Robust and discriminative image authentication based on sparse coding

Image authentication is usually approached by checking the preservation of some invariant features, which are expected to be both robust and discriminative so that content-preserving operations are accepted while content-altering manipulations are rejected. However, most of existing features have not obtained convincing performance due to insufficiency of experiments and over biasing of robustness. Motivated by the sparse coding strategy discovered in primary visual cortex, we explore the possibility of using sparse coding coefficients for image authentication. Through extensive experiments, we discover that the proposed feature bears great discrimination as well as robustness, which indicates the effectiveness of sparse coding as a new invariant feature for image authentication.
Keyphrase Extraction Using Semantic Networks Structure Analysis

Keyphrases play a key role in text indexing, summarization and categorization. However, most of the existing keyphrase extraction approaches require human-labeled training sets. In this paper, we propose an automatic keyphrase extraction algorithm, which can be used in both supervised and unsupervised tasks. This algorithm treats each document as a semantic network. Structural dynamics of the network are used to extract keyphrases (key nodes) unsupervised. Experiments demonstrate the proposed algorithm averagely improves 50% in effectiveness and 30% in efficiency in unsupervised tasks and performs comparatively with supervised extractors. Moreover, by applying this algorithm to supervised tasks, we develop a classifier with an overall accuracy up to 80%.
Wavelet based smoke detection method with RGB Contrast-image and shape constrain

Smoke detection in video surveillance is very important for early fire detection. A general viewpoint assumes that smoke is a low frequency signal which may smoothen the background. However, some pure-color objects also have this characteristic, and smoke also produces high frequency signal because the rich edge information of its contour. In order to solve these problems, an improved smoke detection method with RGB Contrast-image and shape constrain is proposed. In this method, wavelet transformation is implemented based on the RGB Contrast-image to distinguish smoke from other low frequency signals, and the existence of smoke is determined by analyzing the combination of the shape and the energy change of the region. Experimental results show our method outperforms the conventional methods remarkably.
Image Saliency Analysis Based on Retina Simulation

In this paper, we provide an insight of visual saliency modeling from the perspective of simulating visual information manipulation process in human vision system. For humans, visual stimuli are converted into spike trains by retina, and the spike signals are then transferred to brain areas for further analysis. Therefore, we propose to estimate image saliency based on a retina neural network, which is built with realistic morphology and electrophysics data. Then, we analyze the correlation between the spike trains generated by retina and image saliency. The experimental results show the effectiveness of retinal spike trains in image saliency analysis.
Deep Relative Distance Learning: Tell the Difference between Similar Vehicles

The growing explosion in the use of surveillance cameras in public security highlights the importance of vehicle search from a large-scale image or video database. However, compared with person re-identification or face recognition, vehicle search problem has long been neglected by researchers in vision community. This paper focuses on an interesting but challenging problem, vehicle re-identification (a.k.a precise vehicle search). We propose a Deep Relative Distance Learning (DRDL) method which exploits a two-branch deep convolutional network to project raw vehicle images into an Euclidean space where distance can be directly used to measure the similarity of arbitrary two vehicles. To further facilitate the future research on this problem, we also present a carefully-organized largescale image database "VehicleID", which includes multiple images of the same vehicle captured by different realworld cameras in a city. We evaluate our DRDL method on our VehicleID dataset and another recently-released vehicle model classification dataset "CompCars" in three sets of experiments: vehicle re-identification, vehicle model verification and vehicle retrieval. Experimental results show that our method can achieve promising results and outperforms several state-of-the-art approaches.
Selective Eigenbackground for Background Modeling and Subtraction in Crowded Scenes

Background subtraction is a fundamental preprocessing step in many surveillance video analysis tasks. In spite of significant efforts, however, background subtraction in crowded scenes remains challenging, especially, when a large number of foreground objects move slowly or just keep still. To address the problem, this paper proposes a selective eigenbackground method for background modeling and subtraction in crowded scenes. The contributions of our method are three-fold: First, instead of training eigenbackgrounds using the original video frames that may contain more or less foregrounds, a virtual frame construction algorithm is utilized to assemble clean background pixels from different original frames so as to construct some virtual frames as the training and update samples. This can significantly improve the purity of the trained eigenbackgrounds. Second, for a crowded scene with diversified environmental conditions (e.g., illuminations), it is difficult to use only one eigenbackground model to deal with all these variations, even using some online update strategies. Thus given several models trained offline, we utilize peak signal-to-noise ratio to adaptively choose the optimal one to initialize the online eigenbackground model. Third, to tackle the problem that not all pixels can obtain the optimal results when the reconstruction is performed at once for the whole frame, our method selects the best eigenbackground for each pixel to obtain an improved quality of the reconstructed background image. Extensive experiments on the TRECVID-SED dataset and the Road video dataset show that our method outperforms several state-of-the-art methods remarkably.
A system based on sequence learning for event detection in surveillance video

Event detection in crowded surveillance videos is a challenging yet important problem. In this paper, we present our eSur (Event detection system on SURveillance video) system, which is derived from TRECVid'12 surveillance tasks. Currently, eSur attempts to detect two categories of events: 1) pair-wise events (e.g., PeopleMeet, PeopleSplitUp and Embrace); 2) action-like events (e.g., ObjectPut, CellToEar, PersonRuns and Pointing). In eSur system, we first employ people detection and tracking algorithms to locate target persons in 3D space-time domain. Then the video sequences in which target persons occur are partitioned into several spatio-temporal cubes. Visual features (i.e. cubic feature and MoSIFT) are computed over these cubes. After that, a sequence learning method, (namely SVM with dynamic time alignment kernel), is employed to infer the existence of an event for the video sequence. According to the TRECVid SED formal evaluation, eSur has yielded fairly encouraging results on TRECVid'12 dataset.
Subsampling-Adaptive Directional Wavelet Transform for Image Coding

In lifting-based directional wavelet transforms, different subsampling patterns may show significant difference for directional signals in image coding. This paper investigates the influence of subsampling in directional wavelet transform. We show that the best subsampling depends on the direction and the directionality strength of the signal. To improve the coding performance, we further propose a subsampling-adaptive directional wavelet transform, which can use different subsampling patterns adaptively and according to the local characteristics of the image. To handle the boundary transition when subsampling changes, a phase completion process is applied to ensure that wavelet transform with various subsampling can be performed without introducing boundary effects and performance loss. Experimental results show that the proposed transform can achieve significant coding gain in image coding compared to other existing directional wavelet transforms.
Overview of Screen Content Video Coding: Technologies, Standards, and Beyond

This paper presents recent advances in screen content video coding, with an emphasis on two state-of-the-art standards: HEVC/H.265 Screen Content Coding Extensions (HEVC-SCC) by ISO/IEC Moving Picture Experts Group and ITU-T Video Coding Experts Group, and Display Stream Compression (DSC) by Video Electronics Standards Association. The HEVC-SCC enhances the capabilities of HEVC in coding screen content, while DSC provides lightweight compression for display links. Although targeting different application domains, they share some design principles and are expected to become the leading formats in the marketplace in the coming years. This paper provides a brief account of their background, key elements, performance, and complexity characteristics, according to their final specifications. As we survey these standards, we also summarize prior arts in the last decade and explore future research opportunities and standards developments in order to give a comprehensive overview of this field.
On the coding gain of intra-predictive transforms

Intra-predictive transforms are a kind of block-based transforms that can exploit both the intra- and inter-block correlations. This paper analyzes the coding gains of intra-predictive transforms for the Gaussian process. The tight upper bound of the coding gain is derived, which is shown to be better than both the discrete cosine transform and the Karhunen Loève transform. The optimal intra-predictive transform that achieves the upper bound is also presented. Actual coding results on images verify the effectiveness of the optimal intra-predictive transform.
Transcoding for progressive fine granularity scalable vide coding

In this paper, we propose a transcoding scheme for progressive fine granularity scalable video coding (PFGS). Unlike transcoding schemes for non-scalable video coding aiming to change the bit-rate of the video, the transcoding scheme proposed here is to improve the coding performance of scalable video signals when bandwidth has large change. We design a simple transcoder that can perform PFGS transcoding efficiently and in a low complexity way. Experimental results show that our transcoding scheme can significantly improve the bitstream's performance while bandwidth changes largely.
Very low bit rate watercolor video

We developed a very low bit rate watercolor-like video form, which is constructed with a bi-level Y luminance component and a number of UV chrominance combinations. The bi-level Y-component outlines the features of an image and the UV-combinations describe the basic color information of the image. The UV combinations are dynamically chosen during encoding. The resultant video looks better than bi-level video and its bit rate is still much lower than that of full-color video. Watercolor video can be used in video broadcast and communication in very low bandwidth networks.
Optimal joint source-channel bit allocation for MPEG-4 fine granularity scalable video over OFDM system

In this paper, we propose an optimal bit allocation method for MPEG-4 fine granularity scalable (FGS) video over OFDM transmission. In the proposed method, we consider the joint optimal bit allocation of source and forward error correction (Reed-Solomon) codes of different rates over OFDM. Our simulation results show that our proposed approach achieves significant PSNR gain over the existing approach without channel error protection when the input and the output data rates are fixed.
Overview of the Emerging HEVC Screen Content Coding Extension

A screen content coding (SCC) extension to High Efficiency Video Coding (HEVC) is currently under development by the Joint Collaborative Team on Video Coding, which is a joint effort from the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. The main goal of the HEVC-SCC standardization effort is to enable significantly improved compression performance for videos containing a substantial amount of still or moving rendered graphics, text, and animation rather than, or in addition to, camera-captured content. This paper provides an overview of the technical features and characteristics of the current HEVC-SCC test model and related coding tools, including intra-block copy, palette mode, adaptive color transform, and adaptive motion vector resolution. The performance of the SCC extension is compared against existing standards in terms of bitrate savings at equal distortion.