nils-herrmann/clean_scirepeval huggingface dataset:
Peak Transform for Efficient Image Representation and Coding

In this work, we introduce a nonlinear geometric transform, called peak transform (PT), for efficient image representation and coding. The proposed PT is able to convert high-frequency signals into low-frequency ones, making them much easier to be compressed. Coupled with wavelet transform and subband decomposition, the PT is able to significantly reduce signal energy in high-frequency subbands and achieve a significant transform coding gain. This has important applications in efficient data representation and compression. To maximize the transform coding gain, we develop a dynamic programming solution for optimum PT design. Based on PT, we design an image encoder, called the PT encoder, for efficient image compression. Our extensive experimental results demonstrate that, in wavelet-based subband decomposition, the signal energy in high-frequency subbands can be reduced by up to 60% if a PT is applied. The PT image encoder outperforms state-of-the-art JPEG2000 and H.264 (INTRA) encoders by up to 2-3 dB in peak signal-to-noise ratio (PSNR), especially for images with a significant amount of high-frequency components. Our experimental results also show that the proposed PT is able to efficiently capture and preserve high-frequency image features (e.g., edges) and yields significantly improved visual quality. We believe that the concept explored in this work, designing a nonlinear transform to convert hard-to-compress signals into easy ones, is very useful. We hope this work would motivate more research work along this direction.
Object segmentation in the deep neural network feature domain from highly cluttered natural scenes

Deep convolutional neural networks (DCNNs) offer an effective hierarchical representation of images for various vision analysis tasks, including classification and detection. In this paper, we propose to study background modeling and object segmentation from highly cluttered natural scenes in the DCNN feature domain instead of traditional pixel domain. Specifically, we first design and train a DCNN for animal-human-background object classification, which is used to analyze the input image to generate multi-layer feature maps, representing the responses of different image regions to the animal-human-background classifier. From these feature maps, we construct the so-called deep objectness graph for accurate animal-human object segmentation with graph cut. The segmented object regions from each image in the sequence are then verified and fused in the temporal domain using background modeling. Recognizing that the DCNN is very computation-intensive, we explore a fast and efficient design of the DCNN which finds a good trade-off between complexity and the classification-segmentation performance. Our experimental results demonstrate that our proposed method outperforms existing state-of-the-art methods on the camera-trap dataset with highly cluttered natural scenes.
Novel rate-distortion analysis framework for bit rate and picture quality control in DCT visual coding

The rate-distortion (R-D) behaviour of a coding system depends on both the characteristics of the input source data and the coding algorithm. The authors introduce the new concepts of characteristic rate curves and rate curve decomposition to characterise the input source data and model the coding algorithm, respectively. Based on these concepts, a novel framework for R-D analysis is developed, which serves as an alternative to the classical R-D analysis. Based on this framework, a fast algorithm is proposed to predict the R-D curve before quantisation and coding. The proposed algorithm has very low computational complexity. However, in the extensive simulations, its relative prediction error is always less than 5%, which is very small. To the best of the authors' knowledge, this is the first algorithm which is able to accurately estimate the R-D curve before quantisation and coding. With the estimated R-D curve, it is possible to accurately control the output bit rate and picture quality for transform coding of still images and video sequences. In practical visual coding applications, with the proposed R-D estimation algorithm, the coding bit rate can be accurately matched to the available network bandwidth to guarantee the successful transmission of the coded image/video data.
Performance Optimization of Wireless Video Sensor Networks using Swarm Optimization with Convex Mapping

A wireless video sensor network (WVSN) is a system of spatially distributed video sensors which capture, process and transmit video information over a wireless ad hoc network. The performance optimization in WVSN is a nonlinear high-dimension constrained optimization problem. In this work, we consider the unique characteristics of WVSN and develop an evolutionary optimization scheme using a swarm intelligence principle to solve the WVSN performance optimization problem. We transform the solution space defined by flow balance and energy constraints into a convex region in a low-dimensional space. We then merge the convex condition with the swarm intelligence principle to guide the movement of each particle during the evolutionary optimization process. Our theoretical analysis and experimental results demonstrate that the proposed performance optimization scheme is very efficient.
Visual Informatics Tools for Supporting Large-Scale Collaborative Wildlife Monitoring with Citizen Scientists

Collaborative wildlife monitoring and tracking at large scales will help us understand the complex dynamics of wildlife systems, evaluate the impact of human actions and environmental changes on wildlife species, and answer many important ecological and evolutionary research questions. To support collaborative wildlife monitoring and research, we need to develop integrated camera-sensor networking systems, deploy them at large scales, and develop advanced computational and informatics tools to analyze and manage the massive wildlife monitoring data. In this paper, we will cover various aspects of the design of such systems, including (1) long-lived integrated camera-sensor system design, (2) image processing and computer vision algorithms for animal detection, segmentation, tracking, species classification, and biometric feature extraction, (3) cloud-based data management, (4) crowd-sourcing based image annotation with citizen scientists, and (5) applications to wildlife and ecological research.
Local structure learning and prediction for efficient lossless image compression

One major challenge in image compression is to efficiently represent and encode high-frequency structure components in images, such as edges, contours, and texture regions. To address this issue for lossy image compression, in our previous work, we proposed a scheme to learn local image structures and efficiently predict image data based on this structure information. In this work, we applied this structure learning and prediction scheme to lossless image compression and developed a lossless image encoder. Our extensive experimental results demonstrate that the lossless image encoder is competitive and even outperforms the state-of-the-art lossless image compression methods.
Incremental rate control for H.264/AVC video compression

In this study, the authors propose a new rate-complexity-quantisation model and an incremental rate control algorithm for H.264/AVC video coding. One unique property of this algorithm is that, the picture complexity estimation and rate-quantisation modelling are jointly designed with an incremental rate control for P-frames. In addition, the proposed algorithm also introduces a number of efficient rate control techniques, including accurate rate control for intra-frames, enhanced proportional–integral–derivative (PID) buffer controller, and adaptive quantisation parameter determination for B-frames. The proposed algorithm has low computational complexity while providing robust rate control. Our extensive experimental results demonstrate that the proposed algorithm outperforms the current rate control algorithm adopted in the H.264/AVC reference software JM13.2 by achieving more accurate rate control, reducing frame skipping, depressing quality fluctuation and improving the overall coding quality by up to 2.83 dB.
Virtual MIMO-based cross-layer design for wireless sensor networks

In this paper, a novel multihop virtual multiple-input-multiple-output (MIMO) communication protocol is proposed by the cross-layer design to jointly improve the energy efficiency, reliability, and end-to-end (ETE) QoS provisioning in wireless sensor network (WSN). In the protocol, the traditional low-energy adaptive clustering hierarchy protocol is extended by incorporating the cooperative MIMO communication, multihop routing, and hop-by-hop recovery schemes. Based on the protocol, the overall energy consumption per packet transmission is modeled and the optimal set of transmission parameters is found. Then, the issues of ETE QoS provisioning of the protocol are considered. The ETE latency and throughput of the protocol are modeled in terms of the bit-error-rate (BER) performance of each link. Then, a nonlinear constrained programming model is developed to find the optimal BER performance of each link to meet the ETE QoS requirements with a minimum energy consumption. The particle swarm optimization (PSO) algorithm is employed to solve the problem. Simulation results show the effectiveness of the proposed protocol in energy saving and QoS provisioning
A unified rate-distortion analysis framework for transform coding

In our previous work, we have developed a rate-distortion (R-D) modeling framework for H.263 video coding by introducing the new concepts of characteristic rate curves and rate curve decomposition. In this paper, we further show it is a unified R-D analysis framework for all typical image/video transform coding systems, such as embedded zero-tree wavelet (EZW), set partitioning in hierarchical trees (SPIHT) and JPEG image coding; MPEG-2, H.263, and MPEG-4 video coding. Based on this framework, a unified R-D estimation and control algorithm is proposed for all typical transform coding systems. We have also provided a theoretical justification for the unique properties of the characteristic rate curves. A linear rate regulation scheme is designed to further improve the estimation accuracy and robustness, as well as to reduce the computational complexity of the R-D estimation algorithm. Our extensive experimental results show that with the proposed algorithm, we can accurately estimate the R-D functions and robustly control the output bit rate or picture quality of the image/video encoder.
Rate-coverage analysis and optimization for joint audio-video multimedia retrieval

In this work, we consider the problem of automatic content retrieval (ACR) using joint audio-video fingerprints. We focus on how to balance the query accuracy and the size of fingerprint, and how to allocate the fingerprint bits to video and audio frames to maximize the query accuracy. By introducing a novel concept called coverage, which is highly correlated to the query accuracy, we are able to construct a rate-coverage model and formulate the joint audio-video fingerprint bit rate allocation into a dynamic programming optimization problem. Our experimental results demonstrate that, compared to existing approaches, our method improves the retrieval accuracy by up to 25% while using 60% of the original fingerprint bit rate.
Task-Driven Progressive Part Localization for Fine-Grained Object Recognition

The problem of fine-grained object recognition is very challenging due to the subtle visual differences between different object categories. In this paper, we propose a task-driven progressive part localization (TPPL) approach for fine-grained object recognition. Most existing methods follow a two-step approach that first detects salient object parts to suppress the interference from background scenes and then classifies objects based on features extracted from these regions. The part detector and object classifier are often independently designed and trained. In this paper, our major finding is that the part detector should be jointly designed and progressively refined with the object classifier so that the detected regions can provide the most distinctive features for final object recognition. Specifically, we develop a part-based SPP-net (Part-SPP) as our baseline part detector. We then establish a TPPL framework, which takes the predicted boxes of Part-SPP as an initial guess, and then examines new regions in the neighborhood using a particle swarm optimization approach, searching for more discriminative image regions to maximize the objective function and the recognition performance. This procedure is performed in an iterative manner to progressively improve the joint part detection and object classification performance. Experimental results on the Caltech-UCSD-200-2011 dataset demonstrate that our method outperforms state-of-the-art fine-grained categorization methods both in part localization and classification, even without requiring a bounding box during testing.
Building recognition using sketch-based representations and spectral graph matching

In this work, we address the problem of building recognition across two camera views with large changes in scales and viewpoints. The main idea is to construct a semantically rich sketch-based representation for buildings which is invariant under large scale and perspective changes. After multi-scale maximally stable extremal regions (MSER) detection, the proposed approach finds repeated structural components of buildings, such as window, doors, and facades, and extracts semantically rich features, which are organized into a sketch-based representation of buildings. These descriptors are then clustered in association with different planes of the building and matched across video frames using spectral graph analysis. Our experiments demonstrate that the proposed approach outperforms SIFT-based matching schemes, especially for images with large viewpoint changes.
Ensemble Video Object Cut in Highly Dynamic Scenes

We consider video object cut as an ensemble of frame-level background-foreground object classifiers which fuses information across frames and refine their segmentation results in a collaborative and iterative manner. Our approach addresses the challenging issues of modeling of background with dynamic textures and segmentation of foreground objects from cluttered scenes. We construct patch-level bag-of-words background models to effectively capture the background motion and texture dynamics. We propose a foreground salience graph (FSG) to characterize the similarity of an image patch to the bag-of-words background models in the temporal domain and to neighboring image patches in the spatial domain. We incorporate this similarity information into a graph-cut energy minimization framework for foreground object segmentation. The background-foreground classification results at neighboring frames are fused together to construct a foreground probability map to update the graph weights. The resulting object shapes at neighboring frames are also used as constraints to guide the energy minimization process during graph cut. Our extensive experimental results and performance comparisons over a diverse set of challenging videos with dynamic scenes, including the new Change Detection Challenge Dataset, demonstrate that the proposed ensemble video object cut method outperforms various state-of-the-art algorithms.
Energy-aware portable video communication system design for wildlife activity monitoring

In this paper, we introduce our recent research and development effort on energy-efficient portable video communication system design for wildlife activity monitoring. The capability of seeing what an animal sees in the field is very important for wildlife activity monitoring and research. We design an integrated video and sensor system, called DeerCam and mount it on animals so as to collect important video and sensor data about their activities in the field. From the video and sensor data collected by DeerCam, wildlife researchers will be able to extract a wealth of scientific data for studying the behavior patterns of wildlife species and understanding the dynamic of wildlife systems. We present the system architecture of DeerCam, explain our system design goals, and discuss major design issues. One of the central challenges in DeerCam system design is energy minimization. We present a new approach for energy minimization of portable video devices: power-rate-distortion (P-R-D) analysis and optimization. We discuss various approaches to minimizing the energy consumption of DeerCam, which can be also applied to other portable video devices. Results demonstrate that, by incorporating the third dimension of power consumption into conventional rate-distortion (R-D) analysis, P-R-D analysis gives us one extra dimension of flexibility in resource allocation and energy minimization and allows us to significantly reduce energy consumption.
Joint source channel rate-distortion analysis for adaptive mode selection and rate control in wireless video coding

We first develop a rate-distortion (R-D) model for DCT-based video coding incorporating the macroblock (MB) intra refreshing rate. For any given bit rate and intra refreshing rate, this model is capable of estimating the corresponding coding distortion even before a video frame is coded. We then present a theoretical analysis of the picture distortion caused by channel errors and the subsequent inter-frame propagation. Based on this analysis, we develop a statistical model to estimate such channel errors induced distortion for different channel conditions and encoder settings. The proposed analytic model mathematically describes the complex behavior of channel errors in a video coding and transmission system. Unlike other experimental approaches for distortion estimation reported in the literature, this analytic model has very low computational complexity and implementation cost, which are highly desirable in wireless video applications. Simulation results show that this model is able to accurately estimate the channel errors induced distortion with a minimum delay in processing. Based on the proposed source coding R-D model and the analytic channel-distortion estimation, we derive an analytic solution for adaptive intra mode selection and joint source-channel rate control under time-varying wireless channel conditions. Extensive experimental results demonstrate that this scheme significantly improves the end-to-end video quality in wireless video coding and transmission.
Animal Detection From Highly Cluttered Natural Scenes Using Spatiotemporal Object Region Proposals and Patch Verification

In this paper, we consider the animal object detection and segmentation from wildlife monitoring videos captured by motion-triggered cameras, called camera-traps. For these types of videos, existing approaches often suffer from low detection rates due to low contrast between the foreground animals and the cluttered background, as well as high false positive rates due to the dynamic background. To address this issue, we first develop a new approach to generate animal object region proposals using multilevel graph cut in the spatiotemporal domain. We then develop a cross-frame temporal patch verification method to determine if these region proposals are true animals or background patches. We construct an efficient feature description for animal detection using joint deep learning and histogram of oriented gradient features encoded with Fisher vectors. Our extensive experimental results and performance comparisons over a diverse set of challenging camera-trap data demonstrate that the proposed spatiotemporal object proposal and patch verification framework outperforms the state-of-the-art methods, including the recent Faster-RCNN method, on animal object detection accuracy by up to 4.5%.
Analysis and design of power constrained video encoder

In wireless communication networks, an important issue that must be addressed is the limited energy supply of a mobile device, especially in wireless video applications. In this paper, we develop a parametric video encoding architecture, which is fully scalable in power consumption, and establish the power-rate-distortion (P-R-D) model of the video encoding system. Both theoretically and experimentally, we show that by using this P-R-D model, the encoding system is able to automatically adjust its complexity control parameters to match the available energy supply of the mobile device while maximizing the picture quality. The P-R-D model provides a theoretical guideline for system design and performance optimization in wireless video communication under power constraint.
Lossless Image Compression Using Super-Spatial Structure Prediction

We recognize that the key challenge in image compression is to efficiently represent and encode high-frequency image structure components, such as edges, patterns, and textures. In this work, we develop an efficient lossless image compression scheme called super-spatial structure prediction. This super-spatial prediction is motivated by motion prediction in video coding, attempting to find an optimal prediction of structure components within previously encoded image regions. We find that this super-spatial prediction is very efficient for image regions with significant structure components. Our extensive experimental results demonstrate that the proposed scheme is very competitive and even outperforms the state-of-the-art lossless image compression methods.
Real-Time Moving Object Segmentation and Classification From HEVC Compressed Surveillance Video

Moving object segmentation and classification from compressed video plays an important role in intelligent video surveillance. Compared with H.264/AVC, High Efficiency Video Coding (HEVC) introduces a host of new coding features that can be further exploited for moving object segmentation and classification. In this paper, we present a real-time approach to segment and classify moving objects using unique features directly extracted from the HEVC compressed domain for video surveillance. In the proposed method, first, motion vector (MV) interpolation for intra-coded prediction unit (PU) and MV outlier removal are employed for preprocessing. Second, blocks with nonzero MVs are clustered into the connected foreground regions using the four-connectivity component labeling algorithm. Third, object region tracking based on temporal consistency is applied to the connected foreground regions to remove the noise regions. The boundary of moving object region is further refined by the coding unit size and PU size. Finally, a person–vehicle classification model using bag of spatial–temporal HEVC syntax words is trained to classify the moving objects, either persons or vehicles. The experimental results demonstrate that the proposed method provides solid performance and can classify moving persons and vehicles accurately.
Packet-level transmission distortion analysis for video streaming over mesh networks

In video transmission over wired or wireless mesh networks, video packets could be lost due to transmission errors, network congestion or delay bound violation. The compressed video data is highly sensitive to packet loss, and the packet loss will cause decoding failure and more importantly, the error propagation would significantly degrade the reconstructed video quality. In this paper, we extend our previous research and propose a simple yet accurate and robust model to estimate the transmission distortion of each Macro Block (MB) and packet. We then use this transmission distortion estimation results for packet scheduling in video streaming over mesh networks. Extensive simulation results demonstrate the effectiveness and robustness of the proposed model under different experiment settings. More importantly, the estimated transmission distortion, which reveals the unequal importance of each MB and packet, enables important applications in content-aware resource allocation and performance optimization in video communication.