nils-herrmann/clean_scirepeval huggingface dataset:
Three-Dimensional Point-Cloud Plus Patches: Towards Model-Based Image Coding in the Cloud

In the era of cloud computing, there are many correlated images in the cloud, joint compression of these images may provide much higher compression ratio than individual coding. Model-based coding is an appealing approach to image coding in the cloud, as it removes knowledge redundancy among images that share the same model. In this paper, we make an attempt to model-based image coding for landmark images, where our model consists of three-dimensional (3-D) point-cloud plus image patches to describe the geometry and surface color of the landmark respectively. The camera parameters of an input image are estimated based on the 3-D point-cloud and the patches in the model, and then prediction image is generated by selecting, warping, and stitching image patches as well as illuminance compensation, the residue between original and prediction images is compressed by P-frame coding in HEVC encoder. We perform experiments on an Internet photo collection to verify the effectiveness of the proposed scheme. Preliminary results display the superior performance of our scheme that achieves as high as 39.9% bits saving compared to HEVC intra on a single image. The proposed scheme indicates a promising approach to image coding in the cloud and is worthy of in-depth investigation.
Line-based distributed coding scheme for onboard lossless compression of high-resolution stereo images

On board compression coding of remote sensing images requires simple encoder and small storage, etc. However, the existing scheme such as JPEG2000 which is based on DWT has a complex encoder, and many algorithms based on DWT with a lower storage requirement have been researched. In this paper, we propose a novel line-based distributed lossless compression scheme for remote sensing stereo images with low storage costs and light complexity at the encoder. All the lines are encoded independently so that the required storage of the encoder is only one line. In order to achieve low encoding complexity, distributed coding techniques are used to exploit the spatial correlation and inter-view correlation of the stereo images. At the encoder, sub-sampled lines are successively encoded and transmitted. At the decoder, side information is generated with the knowledge of decoded sub-sampled lines and other previously decoded lines for the first view, and the second view will use the previous lines of the first view to remove the inter-view redundancy. In addition, line-based adaptive filter is performed to capture the spatial characteristics and template matching is used to remove the inter-view redundancy. Experimental results on high-resolution remote sensing stereo images demonstrate that the proposed scheme is comparable to JPEG2000 with respect to the compression performance, but with much lower encoding complexity and storage requirement.
Saliency-aware semantic image coding for mobile visual search

Traditional image coding has mainly been concerned about its rate-distortion performance in the sense of what rate can be achieved for a given distortion that is usually measured by either peak signal to noise ratio (PSNR) or subjective quality. However, in many visual analysis scenarios, especially for mobile visual search (MVS), the rate-accuracy performance is crucial. In this paper, we propose a saliency-aware semantic image coding scheme to preserve the discriminative semantic information for visual search and assess the coding performance in the context of MVS, instead of traditional PSNR or subjective quality. We utilize saliency detection to obtain the region of interest (ROI) involving significant semantic information and encode the query image based on the framework of JPEG 2000 with ROI coding. The retrieval accuracy is evaluated on UKBench dataset. Experiment results show that, compared with regular JPEG 2000 without saliency detection, on average 12% and 17% bitrate can be saved by our scheme to achieve similar retrieval accuracy, respectively, for two bag-of-visual words based search methods.
Spatial transcoding from Scalable Video Coding to H.264/AVC

Scalable Video Coding (SVC) is backwards compatible to H.264/AVC in the sense that the base layer sub-bitstream is decodable by an H.264/AVC decoder. However, there are applications wherein it is desirable for an H.264/AVC decoder to obtain a higher resolution video representation than the base layer within SVC. In order to fulfill the needs of such application scenarios, transcoding of SVC enhancement layers to H.264/AVC is required. This paper presents a transcoding scheme that is capable of transcoding a spatial scalable SVC bitstreams to H.264/AVC bitstreams that provide high resolution than the H.264/AVC compliant base layer. To reduce the complexity at the transcoder, a fast mode decision (MD) process is proposed, wherein the original SVC macroblock coding modes and motion information are reused as much as possible. Experimental results show that proposed scheme performs elegantly compared with full-decoding-and-encoding transcoding with low computational complexity.
GRAPH CUT BASED ACTIVE CONTOUR FOR AUTOMATED CELLULAR IMAGE SEGMENTATION IN HIGH THROUGHPUT RNA INTERFACE (RNAi) SCREENING

Recently, image-based, high throughput RNA interference (RNAi) experiments are increasingly carried out to facilitate the understanding of gene functions in intricate biological processes. Effective automated segmentation technique is significant in analysis of RNAi images. However, graph cuts based active contour (GCBAC) method needs interaction during segmentation. Here, we present a novel approach to overcome this shortcoming. The process consists the following steps: First, region-growing algorithm uses extracted nuclei to get the initial contours for segmentation of cytoplasm. Then, constraint factor obtained from binary segmentation of enhanced image is incorporated to improve the performance of cytoplasm segmentation. Finally, morphological thinning algorithm is implemented to solve the touching problem of clustered cells. Our approach can automatically segment clustered cells with polynomial time-consuming. The excellent results verify the effectiveness of the proposed approach
Video frame interpolation using 3-D total variation regularized completion

A new video frame interpolation technique is proposed in this paper.We first use the classical motion compensation interpolation (MCI), but only for motion vectors (MVs) marked as reliable in our classification procedure. To fill in the regions where no such MV is available, we proposed a novel 3-D total variation regularized completion model, which exploits both temporal and spatial smoothness among video frames. Experiments demonstrate its superior performance compared to several classical methods, both in visual quality and PSNR values, while the typical artifacts are significantly reduced.
Efficient hierarchical inter picture coding for H.264/AVC baseline profile

Bi-predictive (B) slices are not supported in the Baseline profile of the Advanced Video Coding (H.264/AVC) standard, which results in a decreased coding efficiency compared with other profiles supporting B slices. However, many application standards, such as the mobile multimedia services specified by the Third Generation Partnership Project (3GPP), use only the Baseline profile for H.264/AVC. Therefore, it is worth investigating H.264/AVC coding when only intra (I) and inter (P) slices are supported. In this paper, a content-adaptive Quantization Parameter (QP) cascading scheme for the hierarchical P coding method compatible with Baseline profile of H.264/AVC is proposed. The proposed method is based on a picture-level QP optimization. The proposed method has a significantly better rate-distortion performance than the traditional IPPP coding structure and outperforms hierarchical P coding methods using fixed delta QP settings between temporal levels noticeably with up to 0.53 dB gain in average luminance Peak Signal-to-Noise Ratio (PSNR).
On the energy-delay tradeoff in lossy network communications

This work investigates the energy-delay tradeoff in real-time network communication systems with ARQ, where energy is measured by power and energy per bit, respectively. We consider the lossy communication case with a finite buffer size, where each block is allowed to be retransmitted under a time constraint. We apply finite blocklength results of channel coding into our system. By adaptively optimizing the blocklength, coding rate, power allocation, and maximum number of retransmissions according to the queue state, we algorithmically achieve the optimal energy-delay tradeoff. We also lower and upper bound the optimal energy-delay function in order to study its asymptotic behavior as the delay and buffer size go to infinity, showing consistency with existing result. That is: the energy per bit approaches the minimum of -1.59 dB as the delay goes to infinity and power approaches zero. We observe that the power increases very rapidly when delay moves away from infinity, meaning that it is very hard to achieve the -1.59 dB limit in practice.
Gradual view refresh in depth-enhanced multiview video

Depth-enhanced multiview video, such as the multiview video plus depth (MVD) format, can be used to provide displaying-time view adjustment capability through depth-image-based rendering (DIBR) and additional compression improvement compared to the Multiview Video Coding standard. In this paper a gradual view refresh (GVR) method is presented to code random access points and provide fast startup in streaming for MVD bitstreams. When decoding is started from a GVR point, a subset of the views can be accurately decoded, while the remaining views can be approximately reconstructed using DIBR. Perfect reconstruction of all views can be reached at a subsequent random access point. The GVR coding was found to be effective with up to 10% bitrate reduction for sequences with static camera arrangement. The use of GVR for fast startup in video streaming was found to be clearly superior to transmitting a MVD bitstream conventionally from rate-distortion point of view. It has been found in earlier studies that there seems to be a delay from stimulus onset until depth is fully perceived, hence giving a reason to believe that accurate reconstruction of all views might not be necessary immediately after starting decoding. Furthermore, it was observed in this paper that the objective picture quality reduction during GVR was only moderate, verifying the applicability of the presented GVR method.
Distortion bounds for source broadcast over degraded channel

This paper investigates the joint source-channel coding problem of sending a memoryless source over a memoryless degraded broadcast channel. An inner bound and an outer bound on the achievable distortion region are derived, which respectively generalize and unify several existing bounds. Moreover, when specialized to Gaussian source broadcast or binary source broadcast, the inner bound and outer bound could recover the best known inner bound and outer bound in the literature. Besides, the inner bound and outer bound are also extended to Wyner-Ziv source broadcast problem, i.e., source broadcast with degraded side information available at decoders. Some new bounds are obtained when specialized to Wyner-Ziv Gaussian case and Wyner-Ziv binary case.
Semantic-Spatial Matching for image classification

Spatial Pyramid Matching (SPM) has been proven a simple but effective extension to bag-of-visual-words image representation for spatial layout information compensation. SPM describes image in coarse-to-fine scale by partitioning the image into blocks over multiple levels and the features extracted from each block are concatenated into a long vector representation. Based on the assumption that images from the same class have similar spatial configurations, SPM matches the blocks from different images according to their spatial layout, by aligning all blocks from an image in a fixed spatial order. However, target objects may appear at any location in the image with various backgrounds. Therefore, the fixed spatial matching in SPM fails to match similar objects located different locations. To solve this problem, we propose an effective and efficient block matching method, Semantic-Spatial Matching (SSM). In this method, not only the spatial layout but also the semantic content is considered for block matching. The experiments on two benchmark image classification datasets demonstrate the effectiveness of SSM.
Robust Temporal-Spatial Decomposition and Its Applications in Video Processing

In this paper, we propose a robust temporal-spatial decomposition (RTSD) model and discuss its applications in video processing. A video sequence usually possesses high correlations among and within its frames. Fully exploiting the temporal and spatial correlations enables efficient processing and better understanding of the video sequence. Considering that the video sequence typically contains slowly changing background and rapidly changing foreground as well as noise, we propose to decompose the video frames into three parts: the temporal-spatially correlated part, the feature compensation part, and the sparse noise part. Accordingly, the decomposition problem can be formulated as the minimization of a convex function, which consists of a nuclear norm, a total variation (TV)-like norm, and an l1 norm. Since the minimization is nontrivial to handle, we develop a two-stage strategy to solve this decomposition problem, and discuss different alternatives to fulfil each stage of decomposition. The RTSD model treats video frames as a unity from both the temporal and spatial point of view, and demonstrates robustness to noise and certain background variations. Experiments on video denoising and scratch detection applications verify the effectiveness of the proposed RTSD model and the developed algorithms.
SOM: Semantic obviousness metric for image quality assessment

Image quality assessment (IQA) tries to estimate human perception based image visual quality in an objective manner. Existing approaches target this problem with or without reference images. For no-reference image quality assessment, there is no given reference image or any knowledge of the distortion type of the image. Previous approaches measure the image quality from signal level rather than semantic analysis. They typically depend on various features to represent local characteristic of an image. In this paper we propose a new no-reference (NR) image quality assessment (IQA) framework based on semantic obviousness. We discover that semantic-level factors affect human perception of image quality. With such observation, we explore semantic obviousness as a metric to perceive objects of an image. We propose to extract two types of features, one to measure the semantic obviousness of the image and the other to discover local characteristic. Then the two kinds of features are combined for image quality estimation. The principles proposed in our approach can also be incorporated with many existing IQA algorithms to boost their performance. We evaluate our approach on the LIVE dataset. Our approach is demonstrated to be superior to the existing NR-IQA algorithms and comparable to the state-of-the-art full-reference IQA (FR-IQA) methods. Cross-dataset experiments show the generalization ability of our approach.
An adaptive down-sampling based video coding with hybrid super-resolution method

It has been proven that performance of video coding at low bit rates can be improved by down-sampling a video before compression and then using super-resolution to up-sample it after decompression. Such techniques are especially important for limited bandwidth communications. In this paper we propose an adaptively down-sampling based coding (DBC) method which performs rate distortion (RD) optimization to determine the coding structure between regular coding and down-sampling coding. In order to restore the original resolution of down-sampling coded video signals, a hybrid super-resolution (SR) algorithm which combines motion compensation (MC) based SR and wiener filter based SR is used. Experimental results show that our method has improvement both in rate-distortion performance and perceived visual quality at low bit rate.
An attention based spatial adaptation scheme for H.264 videos on mobiles

When browsing videos in mobile devices, people often feel that resolution greatly affects their perceptual experience in the limited screen size. In this paper, an attention based spatial video adaptation scheme is proposed to overcome display constraints by producing the region of interest. According to the size of the target display, we automatically detect and crop the informative region in each frame to generate a smooth sequence. To avoid costly fully encoding operations, we employ a set of transcoding techniques based on the H.264 standard. Experimental results show that this approach not only improves the perceptual quality but also saves the bandwidth and computation, especially for the videos which are not well edited.
Hybrid digital-analog scheme for video transmission over fading channel

In this paper, we consider video communication over fading channel, where the perfect instantaneous channel state information (CSI) is available at both sender and receiver. Most of existing coding schemes are inefficient in this communication scenario. The reason is that for digital coding scheme, it has high coding efficiency but unavoidably leads to the cliff effect; while for analog scheme, it has graceful video quality variation with channel varying, but has low coding efficiency. Hence, to integrate the advantages of digital coding and analog coding, we propose a hybrid digital-analog (HDA) scheme. In our scheme, we have adopted adaptive power allocation and adaptive forward error coding (FEC) in digital part to accommodate instantaneous channel quality. The evaluation results show that the proposed HDA scheme outperforms ParCast (a state-of-the-art analog scheme) 0.3~2.2dB under the channel Signal-to-Noise Ratio (SNR) from 3dB to 20dB.
Foveation-Based Wireless Soft Image Delivery

In image compression and transmission, two basic data redundancies can be identified and exploited: statistical redundancy and perceptual redundancy. Statistical redundancy has been studied for a very long time and has been exploited in most image coding schemes. However, perceptual redundancy has not yet been completely exploited. Furthermore, perceptual redundancy is difficult to exploit by using digital coding techniques because power allocation is performed at the bit level for digital coding. However, as one of one-size-fits-all technique, soft transmission is suitable to exploit the perceptual redundancy because its power allocation is directly applied at the pixel level instead of the bit level. In this paper, we propose a novel image transmission scheme, named FoveaCast , for single-input single-output or multi-input multi-output broadcast systems that effectively utilizes the foveation characteristic of human vision and analog coding techniques to achieve higher visual perceptual quality. Hence, our scheme possesses not only high visual perceptual quality but also graceful quality adaptation. Experimental evaluations show that compared with three existing digital schemes and two existing analog schemes, SoftCast and ParCast, FoveaCast achieves better visual perceptual performance. Meanwhile, it achieves a graceful quality variation along with channel variation, behaving just like SoftCast and ParCast.
Adaptive packet encapsulation of Scalable Video Coding bitstream

In recent years, IEEE 802.11 wireless local area networks (WLANs) have been widely deployed. However, robust streaming of video over 802.11 WLANs still faces many challenges, such as bandwidth variation, diversified receivers, transportation over heterogeneous network, etc. Scalable Video Coding (SVC), an extension of the Advanced Video Coding (H.264/AVC), has been designed for heterogeneous network. This paper focuses on SVC video packet transmission from Ethernet to 802.11 and proposes an application layer adaptive video packets encapsulation method, which appends the truncated enhancement layer slice of SVC into the idle space of the 802.11 frame. The proposed method can efficiently utilize the bandwidth of 802.11 and promote the received video quality. The simulation results show that the proposed method can get an average gain about 0.7dB compared with conventional algorithms.
Mixed Gaussian-impulse video noise removal via temporal-spatial decomposition

This paper presents a novel denoising scheme for video sequences corrupted by mixed Gaussian-impulse noise. From a global viewpoint, such a video sequence contains three parts: temporal-spatially correlated video content, uncorrelated dense Gaussian noise, and uncorrelated sparse impulse noise. This fact motivates us to formulate the mixed Gaussian-impulse noise removal task as a temporal-spatial decomposition problem, which amounts to a convex program. A two-stage algorithm is developed to solve this problem efficiently. Effectiveness of the proposed algorithm on mixed Gaussian-impulse noise removal is validated through experiments. The results are satisfactory in both visual quality and PSNR values, while very few prior knowledge of noise statistic is required compared to most state-of-the-art methods.
Rate-distortion optimization with adaptive weighted distortion in high Efficiency Video Coding

This paper presents an adaptive weighted distortion optimization algorithm used in the Rate-Distortion Optimization (RDO) process of the High Efficiency Video Coding (HEVC). RDO is an important tool to improve the coding efficiency. Usually the distortion weights of different color components are equal or predetermined. In this paper, an adaptive weighted distortion optimization algorithm is introduced to improve the coding efficiency. The distortion weight is estimated according to the previous coded pictures belonging to the same temporal level, such that encoding complexity is almost unchanged. With the proposed adaptive weighted distortion optimization method, on average about 3.3% and up to 10.6% bit-saving are obtained based on the latest HEVC reference software, HM-8.0 and the corresponding common test conditions. The proposed algorithm can also be applied to other coding schemes such as H.264/MPEG-4 AVC.
Error Resilient Video Coding Using Redundant Pictures

This paper presents several error resilient video coding methods based on redundant pictures. We combine redundant picture coding with reference picture selection and reference picture list reordering to prevent error propagation in motion compensated video coding. A hierarchical redundant picture allocation method is employed to make a tradeoff between error resilience and coding efficiency. For improved end-to-end rate-distortion performance in packet loss environment, three adaptive redundant picture allocation methods are further developed, utilizing characteristics of the input video content. Simulation results show that the adaptive redundant picture coding methods can achieve average luma peak-signal-to-noise improvements up to 3.5 dB compared to the loss-aware rate distortion optimized intra macroblock refresh algorithm implemented in the H.264/AVC Joint Model (JM). The proposed redundant picture coding methods are standard-compliant and do not introduce any additional end-to-end delay, therefore suit for low-delay applications such as video telephony and video conferencing. Due to the good error resilience performance, some of the proposed redundant picture coding methods have been adopted and integrated into the JM.