nils-herrmann/clean_scirepeval huggingface dataset:
. Optimum bit allocation and accurate rate control for video coding via ρ-domain source modeling

We present a new framework for rate-distortion (R-D) analysis, where the coding rate R and distortion D are considered as functions of /spl rho/ which is the percentage of zeros among the quantized transform coefficients. Previously (see He, Z. et al.,  2001), we observed that, in transform coding of images and videos, the rate function R(/spl rho/) is approximately linear. Based on this linear rate model, a simple and unified rate control algorithm was proposed for all standard video coding systems, such as MPEG-2, H.263, and MPEG-4. We further develop a distortion model and an optimum bit allocation scheme in the /spl rho/ domain. This bit allocation scheme is applied to MPEG-4 video coding to allocate the available bits among different video objects. The bits target of each object is then achieved by our /spl rho/-domain rate control algorithm. When coupled with a macroblock classification scheme, the above bit allocation and rate control scheme can also be applied to other video coding systems, such as H.263, at the macroblock level. Our extensive experimental results show that the proposed algorithm controls the encoder bit rate very accurately and improves the video quality significantly (by up to 1.5 dB).
Fast human-animal detection from highly cluttered camera-trap images using joint background modeling and deep learning classification

In this paper, we couple effective dynamic background modeling with deep learning classification to develop a fast and accurate scheme for human-animal detection from highly cluttered camera-trap images using joint background modeling and deep learning classification. Specifically, first, we develop an effective background modeling and subtraction scheme to generate region proposals for the foreground objects. We then develop a cross-frame image patch verification to reduce the number of foreground object proposals. Finally, we perform complexity-accuracy analysis of deep convolutional neural networks (DCNN) to develop a fast deep learning classification scheme to classify these region proposals into three categories: human, animals, and background patches. The optimized DCNN is able to maintain high level of accuracy while reducing the computational complexity by 14 times. Our experimental results demonstrate that the proposed method outperforms existing methods on the camera-trap dataset.
Efficient H.264 video coding with a working memory of objects

In this work, we investigate a working memory approach for efficient temporal prediction in H.264 video coding. After video frames are encoded, objects are extracted, analyzed, and indexed in a dynamic database which acts as a working memory for the H.264 video encoder. During the encoding process, objects with similar spatial characteristics are retrieved from the working memory and used for motion prediction of objects in the current video frame. This approach extends the multiple-frame estimation and provides a more generic framework for spatiotemporal prediction of video data. Our experimental results on surveillance video data demonstrate that the proposed approach is able to save the coding bit rate by up to 35% with a small computational overhead.
Reliability Analysis for Global Motion Estimation

Global motion estimation (GME) is the enabling step for many important video exploitation tasks. In this work, we focus on indirect GME methods which have low computational complexity. Typically, an indirect GME method has two major steps. The first step is to find point correspondence between frames through local motion search or feature matching. Then, the second step determines global motion parameters using optimal model fitting, such as least mean-squared error (LMSE) fitting or RANSAC. However, due to image noise and inherent ambiguity in point correspondence, local motion estimation often suffers from relatively large errors, which degrade the performance and reliability of GME. In this work, we propose a method to characterize the reliability of local motion estimation results and use this reliability measure as a weighting factor to determine the importance level of each local motion estimation result during global motion estimation. Our simulation results demonstrate that the proposed scheme is able to significantly improve the accuracy and robustness of global motion estimation with a very small computational overhead.
Performance analysis of wireless video sensors in video surveillance

Wireless video sensor networks (WVSN) have been envisioned for a wide range of important applications, including battlefield intelligence, security monitoring, and environmental tracking. Compared to traditional communication systems, the WVSN operates under a set of unique resource constraints, including limitations with respect to energy supply, on-board computational capability, and transmission bandwidth. The objective of this work is to study the resource utilization behavior of a wireless video sensor and analyze its performance under these resource constraints. More specifically, we develop an analytic power-rate-distortion (P-R-D) model to characterize the inherent relationship between the power consumption of a video encoder and its rate-distortion performance. Based on the P-R-D analysis and a simplified model for wireless transmission power, we study the optimum power allocation between video encoding and wireless transmission. We consider an important scenario in wireless video sensing - wireless video surveillance, and analyze the performance limit of the wireless video sensor. The analysis and results obtained in this paper provide an important guideline for practical wireless video sensor design.
Low-delay rate control for DCT video coding via ?-domain source modeling

By introducing the new concepts of characteristic rate curves and rate curve decomposition, a generic source-modeling framework is developed for transform coding of videos. Based on this framework, the rate-quantization (R-Q) and distortion-quantization (D-Q) functions (collectively called R-D functions in this work) of the video encoder can be accurately estimated with very low computational complexity before quantization and coding. With the accurate estimation of the R-Q function, a frame-level rate control algorithm is proposed for DCT video coding. The proposed algorithm outperforms the TMN8 rate control algorithm by providing more accurate and robust rate regulation and better picture quality. Based on the estimated R-D functions, an encoder-based rate-shape-smoothing algorithm is proposed. With this smoothing algorithm, the output bit stream of the encoder has both a smoothed rate shape and a consistent picture quality, which are highly desirable in practical video coding and transmission.
H.264/AVC rate control with enhanced ratequantisation model and bit allocation

Rate control algorithms aim to achieve perceivable video quality for real-time video communication applications under some real-world constraints, such as bandwidth, delay, buffer sizes etc. This study presents an adaptive and efficient rate control framework for low-delay H.264/AVC video communication. At the basic unit level, to realise quantisation decision improvement, a novel rate–qantisation (R–Q) model is proposed for P frame. This P frame R–Q model is established through the ρ domain rate control theory to compute quantisation according to target bit assignment. In addition, an I frame R–Q model is presented to measure frame level coding activity for quantisation decision without performing computationally intensive intra-prediction, which helps alleviating the frame skipping problem. When compared with Joint Video Team (JVT)-G012, a rate control scheme adopted by JVT H.264/AVC reference software JM12.3, the proposed rate control framework shows good performance in terms of improved average luminance peak signal noise ratio (PSNR), reduced number of skipped frames and smaller mismatch between target and actual bit rates.
Linear Rate Control and Optimum Statistical Multiplexing for H.264 Video Broadcast

The H.264 video coding standard achieves significantly improved video compression efficiency and finds important applications in digital video broadcast. To enable H.264 video encoding for digital TV broadcast and maximize its broadcast efficiency, there are two important issues that need to be adequately addressed. First, we need to understand the complex coding mechanism of an H.264 video encoder and develop a model to analyze and control its rate-distortion (R-D) behavior in an accurate and robust manner. Second, the R-D behaviors of individual channels in the broadcast system should be jointly controlled and optimized under bandwidth and buffer constraints so as to maximize the overall broadcast quality. In this paper, we develop a linear rate model and a linear rate control scheme for H.264 video coding. We develop an optimum statistical multiplexing system to allocate bits across video programs (each being encoded by an H.264 encoder) and video frames so that the overall video broadcast quality is maximized. We study the bandwidth and buffer constraints in video broadcast and formulate the optimum statistical multiplexing into a constrained mathematical optimization problem. Realizing that it is impossible to find a close-form solution for global optima, we propose a simple yet efficient algorithm to find a near-optimum solution for joint rate allocation under buffer constraints. Our extensive simulation results demonstrate that the proposed statistical multiplexing system achieves about 40-50% saving in bandwidth, provides a smooth video quality change across programs and frames, and maintains robust decoder buffer control.
Optimum quantization error feedback filter for wavelet image compression

In wavelet image compression, all information loss occurs during quantization. Based on the property of bi-orthogonal wavelets, optimal 2-D quantization error feedback filters are used to reduce the reconstruction error. With very low complexity with regard to computation and implementation, the error feedback system improves the PSNR of the reconstructed image by about 0.25 dB. In addition, due to its similar structure to the dithered quantizer, it also improves the subjective quality of the reconstructed image by reducing the contouring and ranging effects.
Nighttime in-home action monitoring for eldercare

In this work, we develop a system to automatically monitoring actions of elderly people at home for safety enhancement and health monitoring. We use an Infrared camera embedded in a living environment to capture images. We study the characteristics of different clothing in Infrared images and develop an efficient silhouette extraction method for Infrared (IR) images using spatio-temporal filtering. We recognize human action using supervised learning methods. Our experimental results demonstrate that our proposed method is efficient.
Facial expression recognition based on LLENet

Facial expression recognition plays an important role in lie detection, and computer-aided diagnosis. Many deep learning facial expression feature extraction methods have a great improvement in recognition accuracy and robutness than traditional feature extraction methods. However, most of current deep learning methods need special parameter tuning and ad hoc fine-tuning tricks. This paper proposes a novel feature extraction model called Locally Linear Embedding Network (LLENet) for facial expression recognition. The proposed LLENet first reconstructs image sets for the cropped images. Unlike previous deep convolutional neural networks that initialized convolutional kernels randomly, we learn multi-stage kernels from reconstructed image sets directly in a supervised way. Also, we create an improved LLE to select kernels, from which we can obtain the most representative feature maps. Furthermore, to better measure the contribution of these kernels, a new distance based on kernel Euclidean is proposed. After the procedure of multi-scale feature analysis, feature representations are finally sent into a linear classifier. Experimental results on facial expression datasets (CK+) show that the proposed model can capture most representative features and thus improves previous results.
Adaptive Critic Design for Energy Minimization of Portable Video Communication Devices

Portable video communication devices operate on batteries with limited energy supply. Video compression is computationally intensive and energy-demanding. Therefore, one critical issue in portable video communication system design is to minimize the energy consumption of video encoding so as to prolong the operational lifetime of portable video devices. In this paper, we explore advanced methods in adaptive system control and develop an online complexity control and energy minimization scheme for real-time video encoding. More specifically, we introduce a set of parameters to control the computational complexity and energy consumption of the video encoder. We consider this video encoder as a nonlinear control system. Based on adaptive critic design, an advanced adaptive control method recently developed in control science, we design an online control scheme which is able to select the best configuration of complexity control parameters to minimize the energy consumption under rate-distortion constraints, or equivalently maximize video quality under rate and energy constraints. Our extensive experimental results demonstrate that the proposed scheme approaches the true optimum performance. The proposed online complexity control and energy minimization scheme will provide an important tool for energy minimization of portable video devices.
Low-pass filtering of rate-distortion functions for quality smoothing in real-time video communication

In variable-bit-rate video coding, the video is preprocessed to collect sequence-level statistics, which are used for global bit allocation in the actual encoding stage to obtain a smoothed video presentation quality. However, in real-time video recording and network streaming, this type of two-pass encoding scheme is not allowed because the access to future frames and global statistics is not available. To address this issue, we introduce the concept of low-pass filtering of rate-distortion functions and develop a smoothed rate control (SRC) framework for real-time video recording and streaming. Theoretically, we prove that, using a geometric averaging filter, the SRC algorithm is able to maintain a smoothed video presentation quality while achieving the target bit rate automatically. We also analyze the buffer requirement of the SRC algorithm in real-time video streaming, and propose a scheme to seamlessly integrate robust buffer control into the SRC framework. The proposed SRC algorithm has very low computational complexity and implementation cost. Our extensive experimental results demonstrate that the SRC algorithm significantly reduces the picture quality variation in the encoded video clips.
Video-based activity monitoring for indoor environments

In this work, we study how continuous video monitoring and intelligent video processing can be used in eldercare to assist the independent living of elders and to improve the efficiency of eldercare practice. More specifically, we construct an advanced silhouette extraction and tracking algorithm for indoor environments. An adaptive learning method was developed to estimate the physical location and moving speed of a person from a single camera view without calibration. Then hierarchical decision tree and dimension reduction methods were used for human action recognition. We extract important ADL (activities of daily living) statistics for automated functional assessment. Our extensive tests over these massive video datasets demonstrate that the proposed automated activity analysis system is very efficient.
Technology for Successful Aging

Americans are living longer, and research shows that seniors are embracing independence, and will benefit from living in the same place. These are the rationale for 'Aging in Place' and the development of Tiger Place, an 'Aging in Place' Environment in Columbia Missouri. Our goal is to minimize intrusion, allow the resident complete control over privacy and treatment (if any), and to provide substantive improvement in quality of life. Nevertheless there continues to be significant risks to the elderly which results in reduced functional and cognitive activity. While there has been much technology developed to ameliorate these factors, there is no comprehensive evaluation of the benefit of these devices nor a comprehensive strategy to improve the quality of life of seniors as determined by functional ability and possibly later cognitive ability. With our partners at the University of Virginia we are developing a system of sensors, to monitor the activity of seniors in their residences. We measure motion, footfalls, sleep and restlessness, we have stove sensors and sensing mats, all connected wirelessly to a computer which performs an initial evaluation and data transfer to a secure server for further study. Based upon the monitor data we will implement an intervention to ameliorate functional decline. Focus group studies determine the attitudes, concerns and impressions of the residents and staff. We find that senior's attitude to technology is healthy and they will try helpful approaches. In addition to the statistical comparisons, we model the data using hidden Markov models, integrate or fuse the monitor data with video images, and reason about behavior using fuzzy logic. The results of this work will additionally reduce the workload on caregivers, foster communication between residents and family, and give these seniors independence. We have requested and received IRB approval for this study
HEVC compressed domain moving object detection and classfication

Compressed domain moving object segmentation and classification plays an important role in many real-time applications, such as video indexing and intelligent video surveillance. Compared with the previous international video coding standards, such as H.264/AVC, HEVC introduces a host of new coding features. Therefore, moving object segmentation and classification directly from HEVC compressed videos represents a new challenge. In this paper, we develop a method for segmenting and classifying moving objects, specifically, persons and vehicles, in the HEVC compression domain. We first train a classifier to determine if an image patch belongs to the foreground objects or background using HEVC syntax features. This will generate a bounding box which locates the object in the video frame. We then train a second classification model to classify the moving objects, either persons or vehicles, using bags of spatial-temporal HEVC syntax words. Our extensive experimental results demonstrate that the approach provides the remarkable performance and can classify moving person and vehicles accurately and robustly.
A real-time system for in-home activity monitoring of elders

In this paper, we propose a real-time system for in-home activity monitoring and functional assessment for elder care. We describe the development of the whole system which could be used to assist the independent living of elders and improve the efficiency of eldercare practice. With this system, data is collected, silhouettes extracted, features further analyzed and visualized into graphs from which eldercare professionals are able to understand massive video monitoring data within a short period of time. Our experimental results demonstrate that the proposed system is efficient in indoor elder activities monitoring and easily utilized by eldercare professionals.
Activity Analysis, Summarization, and Visualization for Indoor Human Activity Monitoring

In this work, we study how continuous video monitoring and intelligent video processing can be used in eldercare to assist the independent living of elders and to improve the efficiency of eldercare practice. More specifically, we develop an automated activity analysis and summarization for eldercare video monitoring. At the object level, we construct an advanced silhouette extraction, human detection and tracking algorithm for indoor environments. At the feature level, we develop an adaptive learning method to estimate the physical location and moving speed of a person from a single camera view without calibration. At the action level, we explore hierarchical decision tree and dimension reduction methods for human action recognition. We extract important ADL (activities of daily living) statistics for automated functional assessment. To test and evaluate the proposed algorithms and methods, we deploy the camera system in a real living environment for about a month and have collected more than 200 hours (in excess of 600 G bytes) of activity monitoring videos. Our extensive tests over these massive video datasets demonstrate that the proposed automated activity analysis system is very efficient.
/spl rho/-domain bit allocation and rate control for real time video coding

A novel framework for rate-distortion (RD) analysis is developed. Based on this framework, an optimum bit allocation scheme and an accurate rate control algorithm are proposed for real-time video coding with H.263+ (see Cote, G. et al.,  1998). With the proposed algorithm, the picture quality is significantly improved. The output bit rate of the video encoder is controlled robustly and accurately according to the network condition.
Blockwise zero mapping image coding

A coding algorithm of low addressing and implementation complexity is proposed. It is based on partitioning uniformly quantized wavelet coefficients into multiscale blocks with each block classified as either an all-zero block or a non-zero block. The non-zero blocks are coded by a new method called zero-mapping whose outputs are further compressed by a first-order arithmetic coder. The performance of the proposed coding algorithm compares favorably with that of some well-known coding algorithms, particularly for images with considerable high frequency components.
Delay-Bounded Priority-Driven Resource Allocation for Video Transmission Over Multihop Networks

In this paper we consider the problem of resource allocation for video transmission over mesh networks with delay bound constraints and priority-based packet scheduling. We observe that priority-driven packet scheduling at the intermediate network routers has a direct and significant impact on the queuing behaviors and delay bound violation probabilities of video packets, as well as the overall end-to-end video distortion. Using learning methods, we develop a packet delay bound violation probability model for video transmission over multihop networks with priority-based packet scheduling. With this model, we can successfully predict the probability of packets being dropped due to violation of specified delay bounds. We also observe that the transmission distortion caused by packet drops exhibits a unique exponential behavior with priority-based packet scheduling. With these analysis results, we formulate the resource allocation for multisession video transmission over networks with priority-driven packet scheduling under delay bound constraints as a multiobjective optimization problem. Evolutionary optimization methods based on single- and multiobjective genetic algorithms are proposed to solve the problem and obtain the optimal resource allocation. Extensive experiment results demonstrate the effectiveness of the proposed resource-distortion models and optimization algorithms.