nils-herrmann/clean_scirepeval huggingface dataset:
A sequence-based rate control framework for consistent quality real-time video

Most model-based rate control solutions have the generally questionable assumption that video sequence is stationary. In addition, they often suffer from the fundamental problem of model parameter misestimation. In this paper, we propose a sequence-based frame-level bit allocation framework employing a rate-complexity model that has the capability of tracking the nonstationary characteristics in the video source without look-ahead encoding. In addition, a new nonlinear model parameter estimation approach is proposed to overcome the existing problems in previous model parameter estimation schemes where quantization parameter (QP) is determined to achieve the allocated bits for a frame. Furthermore, a general concept of bit allocation guarantee is discussed and its importance is highlighted. The proposed rate control solution can achieve smoother video quality with less quality flicker and motion jerkiness. Both a complete solution where requantization is employed to guarantee the achievement of the allocated bits, and a simplified solution without requantization are studied. Experimental results show that they both provide significantly better performance, in terms of average peak-signal-to-noise ratio and quality smoothness, than the MPEG-4 Annex L frame-level rate control solution.
Mainstream media vs. social media for trending topic prediction - an experimental study

In the recent years, we have witnessed social networks blossom. Social networking reshaped worldwide communication significantly increased the speed of news spread, and connected the world stronger than ever. Although social networking has been such a revolutionary invention for the society, and many researchers have turned towards social media to explore trending topics, mainstream media still remains as the origin of the majority of the news discussed in social networking sites. Social stream mining to make video recommendations based on the trending topics has been an active direction in the research community. Understanding the trending topics and its impact on video sharing sites is very interesting for network traffic engineers. Quality of service can be significantly improved if we can predict what kind of video content will generate large traffic. The focus of this paper is to study which type of media, mainstream or social, can contribute better towards identifying trending topics. We present the experimental study of the story development process in mainstream and social media based on the real-world data. The study helps us properly identify which media source is more appropriate for the video recommendation and network traffic prediction systems. Through our findings, we discovered mainstream media could significantly improve the trend detection.
Estimating side-information for Wyner-Ziv video coding using resolution-progressive decoding and extensive motion exploration

In Wyner-Ziv video coding (WZVC), the quality of side information (SI) has a critical impact on the coding efficiency. Most existing WZVC schemes generate SI using decoder-side motion estimation (ME), where the current frame is unavailable and the ME accuracy is greatly impaired. In this paper, without introducing additional bitrate overhead, we incorporate fractional-pel motion search, reduced block sizes and multiple hypothesis prediction into our previously proposed decoder-side multi-resolution motion refinement framework , where the current frame is progressively decoded, and based on which the decoder iteratively refines the motion to improve its accuracy. Theoretical analysis shows significant gain of the combination of these advanced techniques. A practical SI estimator is implemented and provides prediction performance comparable to H.264/AVC.
Point-wise extended visual masking for JPEG-2000 image compression

One common visual optimization strategy for image compression is to exploit the visual masking effect where artifacts are locally masked by the image acting as a background signal. In this paper, we present a point-wise extended visual masking approach that nonlinearly maps the wavelet coefficients to a perceptually uniform domain prior to quantization by taking advantages of both self-contrast masking and neighborhood masking effects, thus achieving very good visual quality. It is essentially a coefficient-wise adaptive quantization without any overhead. It allows bitstream scalability, as opposed to many previous works. The proposed scheme has been adopted into the working draft of JPEG-2000 Part II.
Towards Cross-Domain Learning for Social Video Popularity Prediction

Previous research on online media popularity prediction concluded that the rise in popularity of online videos maintains a conventional logarithmic distribution. However, recent studies have shown that a significant portion of online videos exhibit bursty/sudden rise in popularity, which cannot be accounted for by video domain features alone. In this paper, we propose a novel transfer learning framework that utilizes knowledge from social streams (e.g., Twitter) to grasp sudden popularity bursts in online content. We develop a transfer learning algorithm that can learn topics from social streams allowing us to model the social prominence of video content and improve popularity predictions in the video domain. Our transfer learning framework has the ability to scale with incoming stream of tweets, harnessing physical world event information in real-time. Using data comprising of 10.2 million tweets and 3.5 million YouTube videos, we show that social prominence of the video topic (context) is responsible for the sudden rise in its popularity where social trends have a ripple effect as they spread from the Twitter domain to the video domain. We envision that our cross-domain popularity prediction model will be substantially useful for various media applications that could not be previously solved by traditional multimedia techniques alone.
Optimum Detection for Spread-Spectrum Watermarking That Employs Self-Masking

Digital watermarking is an efficient and promising approach to protect intellectual property rights of digital media. Spread spectrum (SS) is one of the most widely used image watermarking schemes because of its robustness against attacks and its support for the exploitation of the properties of the human visual system (HVS). To maximize the watermark strength without introducing visual artifacts, in SS watermarking, the watermark signal is usually modulated by the just-noticeable difference (JND) of the host image. In advanced perceptual models, the JND is characterized as a nonlinear function of local image features. The optimum detection scheme for such nonlinearly embedded watermarks, however, has rarely been studied. In this paper, we address this problem and propose a novel approach that transforms the test signal to a perceptually uniform domain and then performs Bayesian hypothesis testing in that domain. Locally optimum detectors for arbitrary host signal distributions and arbitrary JND models that exploit the self-masking property of the HVS are derived in closed forms, in which the test signal is first nonlinearly preprocessed before a linear correlator is applied. The optimality of the proposed detector is justified mathematically according to the Neyman-Pearson criterion. Simulation results demonstrate the superior performances of the proposed detector over the conventional linear correlation detector.
A hybrid approach for tree classification in airborne LIDAR data

In this paper we propose a hybrid approach for tree classification in airborne LIDAR (Light Detection and Ranging) data by integrating the point based supervised classification with region-based unsupervised clustering method. Furthermore we propose a novel 3D robust statistics-based shape feature that can overcome the limitations of existing methods in separating building boundary points from tree points. Experimental results show the new algorithm is very effective and can achieve very high accuracy.
Electrical magnetic hybrid power quality compensation system for V/V traction power supply system

To solve power quality problem in V/V traction power supply system, a novel electrical magnetic hybrid power quality compensation system based on railway power conditioner (RPC) and magnetic static var compensator (MSVC) is proposed in this study. The RPC is only used to transfer active power and suppressing harmonics. The MSVC is used to compensate reactive power and suppress 3rd and 5th harmonic currents. The proposed hybrid compensation system will reduce the capacity as well as the DC-link voltage of RPC. The principle of negative, reactive and harmonic currents compensation is analysed first. And then, a collaborative control strategy is proposed to make the compensation system have a fast respond speed. At last the simulation and experiment results have verified the proposed structure and its control strategy effectively.
Fast self-synchronous content scrambling by spatially shuffling codewords of compressed bitstreams

This paper presents a content access control method by spatially shuffling codewords of the compressed bitstream. This approach is lightweight and incurs no bit overhead. One important advantage of the approach is that the resulting scrambled bitstream can be made compliant to the compression format, thus providing some level of scalability, error resiliency, network friendliness and capability of performing signal processing directly on the encrypted bitstream. In addition, we propose a method for generating or updating the shuffling tables on the fly, based on encrypting some local-content-specific bits using a standard cipher. This local-content-specific bits based table generation process is self-synchronous, which is critical in the presence of packet loss. It also enhances the resistance of this spatial shuffling approach to plain-text attack.
On resolving rightful ownerships of digital images by invisible watermarks

Digital watermarking has been proposed as a means for copyright protection of multimedia data. This paper first discusses some scenarios in which many current watermarking schemes fail to resolve the rightful ownership of an image. We then identify the key problems and discuss some crucial requirements for a valid invisible watermark detection. In particular, we argue that, for the particular application of resolving rightful ownership using invisible watermarks, it might be crucial to require that the original image not be directly involved in the watermark detection process. A general framework for validly detecting the invisible watermarks is then proposed. We show the effectiveness of this technique based on some visual-model-based watermarking schemes.
High Performance Adaptive Video Services Based on Bitstream Switching for IPTV Systems

As the Next Generation Network gains increasing attention, IPTV has been recognized as an efficient way to provide key IP multimedia services. Compared to conventional TV services, IPTV is sensitive to the IP network characteristics such as packet loss, delay and jittering. In addition, the potential long delay in channel switching is another significant challenge for IPTV systems. In this paper, to improve the QoS (Quality of Service) and QoE (Quality of Experience), we propose a framework which includes (1) introducing server-side bit-rate adaptation to deal with bandwidth variation; (2) multicasting low bit-rate videos for preview of multi-channel programs and Picture-in-Picture (PiP); and (3) using a two-step P-frame switching mechanism to reduce the delay in channel switching. Our results from a real-world test-bed show that the proposed framework greatly improves the perceived video quality and user experience at the client side.
Spatio-Temporal Attention-Based LSTM Networks for 3D Action Recognition and Detection

Human action analytics has attracted a lot of attention for decades in computer vision. It is important to extract discriminative spatio-temporal features to model the spatial and temporal evolutions of different actions. In this paper, we propose a spatial and temporal attention model to explore the spatial and temporal discriminative features for human action recognition and detection from skeleton data. We build our networks based on the recurrent neural networks with long short-term memory units. The learned model is capable of selectively focusing on discriminative joints of skeletons within each input frame and paying different levels of attention to the outputs of different frames. To ensure effective training of the network for action recognition, we propose a regularized cross-entropy loss to drive the learning process and develop a joint training strategy accordingly. Moreover, based on temporal attention, we develop a method to generate the action temporal proposals for action detection. We evaluate the proposed method on the SBU Kinect Interaction data set, the NTU RGB + D data set, and the PKU-MMD data set, respectively. Experiment results demonstrate the effectiveness of our proposed model on both action recognition and action detection.
Multi-resolution based hybrid spatiotemporal compression of encrypted videos

Compression of encrypted data can be viewed as a special case of distributed source coding and can be achieved by applying Slepian-Wolf Coding. However, how to compress the encrypted video efficiently remains a challenging problem especially for those videos with irregular high motion. This paper proposes a novel multi-resolution based approach which makes it possible not only to effectively derive the temporal side information from previous frames, but also to generate the spatial side information by having partial access to the current frame. The spatial and temporal side information can then be integrated adaptively to facilitate the compression. Simulation results show that the proposed scheme significantly outperforms other schemes, especially for those video clips with irregular high motion.
Efficient frequency domain selective scrambling of digital video

Multimedia data security is very important for multimedia commerce on the Internet such as video-on-demand and real-time video multicast. Traditional cryptographic algorithms/systems for data security are often not fast enough to process the vast amount of data generated by multimedia applications to meet real-time constraints. This paper presents a joint encryption and compression framework in which video data are scrambled efficiently in the frequency domain by employing selective bit scrambling, block shuffling and block rotation of the transform coefficients and motion vectors. The new approach is very simple to implement, yet provides considerable levels of security and different levels of transparency, and has a very limited adverse impact on compression efficiency and no adverse impact on error resiliency. Furthermore, it allows transcodability/scalability, and other content processing functionalities without having to access the cryptographic key and perform decryption and re-encryption.
A super-fast online face tracking system for video surveillance

In this paper, we propose a novel and practical system for robust online face tracking in surveillance videos. The proposed system has two contributions: 1) sustained high performance for long-term tracking even when faces come in and out of the view frequently, and 2) extremely low complexity which allows for real-time deployment on various platforms. These advantages are achieved by designing a regular update framework based on a state-of-the-art face detector and a new histogram-assisted KLT (HAKLT) tracker. Experimental results demonstrate a superior and super-fast (>100fps) practical face tracking system.
Motion Refinement Based Progressive Side-Information Estimation for Wyner-Ziv Video Coding

During the past ten years, Wyner-Ziv video coding (WZVC) has gained a lot of research interests because of its unique characteristics of "simple encoding, complex de coding." However, the performance gap between WZVC and conventional video coding has never been closed to the point promised by the information theory. In this paper, we illustrate the chicken-and-egg dilemma encountered in WZVC: high efficiency WZVC requires good estimation of side information (SI); however, good SI estimation is not possible for the decoder without access to the decoded current frame. To resolve such a dilemma, we present and advocate a framework that explores an important concept of decoder-side progressive-learning. More specifically, a decoder-side multi-resolution motion refinement (MRMR) scheme is proposed, where the decoder is able to learn from the already-decoded lower-resolution data to refine the motion estimation (ME), which in turn greatly improves the SI quality as well as the coding efficiency for the higher resolution data. Theoretical analysis shows that at high rates, decoder-side MRMR outperforms motion extrapolation by as much as 5 dB, while falling behind conventional encoder-side inter-frame ME by only about 1.5 dB. In addition, since decoder-side ME does not suffer from the bit-rate overhead in transmitting the motion information, further performance gain can be achieved for decoder-side MRMR by incorporating fractional-pel motion search, block matching with smaller block sizes, and multiple hypothesis prediction. We also present a practical WZVC implementation with MRMR, which shows comparable coding performance as H.264 at very high bit rates.
Improving distributed video coding by exploiting context-adaptive modeling

The statistical model of the bits to be encoded is crucial for the coding performance of distributed video coding (DVC). In this paper, a bit-level context-adaptive correlation model is proposed to exploit high-order statistical correlation for better channel coding performance, which consequently improves the video coding efficiency. In the proposed scheme, the wavelet domain DVC is considered and the coefficients are coded in a bit-plane fashion. The context for each bit to be coded is first formed. Then the probability distribution of each bit is estimated by using previously available data with the same context. For magnitude coding, the significant state of the following elements are included in the context, (1) the side information, (2) the local neighborhood, (3) the parent coefficients (if applicable). The condition of side information is considered as well. For sign coding, the context consists of the sign and the quality of the side information. The proposed model is implemented within a recently proposed DVC framework with decoderside multi-resolution motion refinement (MRMR). Experimental results show the effectiveness of the proposed scheme with significant coding gain over the original MRMR based DVC system, especially for videos with high motion intensity and for lower bit rates.
Perceptual watermarking of still images

Content providers on the Internet are faced with the problem of how to secure electronic data. This problem has generated research activity in the area of digital watermarking of electronic content. The challenge is to introduce a digital watermark that is both transparent and highly robust to common signal processing and possible attacks. The two basic requirements for an effective watermarking scheme, robustness and transparency, conflict with each other. We propose a watermarking technique for digital images that is based on utilizing visual models which have been developed in the context of image compression. The visual models give us a direct way to determine the maximum amount of watermark signal that each portion of an image can tolerate without affecting the visual quality of the image. This allows us to provide the maximum strength watermark which in turn, is extremely robust to common image processing and editing such as JPEG compression, rescaling, and cropping. Our watermarking scheme is based on a DCT framework which allows for the possibility of directly watermarking the JPEG bitstream. Our scheme is shown to provide very good results both in terms of image transparency and robustness.
Rate-distortion optimized dynamic bitstream switching for scalable video streaming

Bitstream switching is an effective way to deal with bandwidth variation in transmitting multimedia over time-varying channels such as the Internet or wireless networks. Most previous work has focused on how to switch to reduce the potential error drift, without necessarily taking into account the impact of the switching on the bandwidth and delay requirements. This paper proposes a rate-distortion optimized dynamic bitstream switching algorithm that explicitly takes into account the impact of the switching on both drifting effect and bandwidth requirement. The rate control (or switching decision) of the proposed algorithm explicitly takes into account the delay constraint of multimedia applications, using an integrated end-to-end virtual network buffer management approach. Initial results show that it can achieve good performance with well-controlled quality drift and delay, and provides a promising research direction
Exploiting Overlay Path-Diversity for Scalable Reliable Multicast

IP-multicast is a bandwidth efficient transmission mechanism for multimedia communication. Reliability in IP-multicast, however, remains a significant challenge. This paper addresses the reliability and scalability issues in IP-multicast by exploring a novel, highly distributed overlay peer-to-peer retransmission architecture that exploits path-diversity. A simple unicast-based "tracert" tool is proposed to help to identify peers with disjoint path to the sender as potential retransmission nodes. Probing can help to adapt to the dynamics of the network. We show that a hybrid system with both "tracert" and probing can perform better than probing only or "tracert" only approach. In addition, we identify the proper probing interval that does not introduce significant probing overhead yet can collect sufficiently updated information to help recover most of the lost packets. The proposed hybrid multicast system is practical, scalable, and easy to deploy, requiring no change to the existing network infrastructure.
Structural similarity-based video fingerprinting for video copy detection

The authors propose a video fingerprinting method based on structural similarity and a graph model. Structural similarity-based fingerprint generation and double-layer matching are the two main contributions. The video is mapped to a graph with frames as its vertices, and structural similarity is proposed to compute the weights of the edges. Then, the fingerprint consisting of a match tag (a coarse fingerprint) and a fine fingerprint is generated by this graph. The match tag is generated by an independent set of this graph, and the fine fingerprint is generated by the weight matrix of the graph based on the two-block-dimensional discrete cosine transform. During the matching, the video can be matched at the first-layer using the match tag to obtain a candidate set, whereas the second-layer matching is performed in this candidate set using the fine fingerprint to find a final match. The proposed video fingerprinting method is shown to be resistant to geometric attacks on frames and impairment of transmission channels.