nils-herrmann/clean_scirepeval huggingface dataset:
Generalized Interpolation-Based Fractional Sample Motion Compensation

Fractional sample interpolation with finite impulse response (FIR) filters is commonly used for motion-compensated prediction (MCP). The FIR filtering can be viewed as a signal decomposition using restricted basis functions. The concept of generalized interpolation provides a greater degree of freedom for selecting basis functions. We developed a generalized interpolation framework for MCP using fixed-point infinite impulse response and FIR filters. An efficient multiplication-free design of the algorithm that is suited for hardware implementation is shown. A detailed analysis of average and worst case complexities compared to FIR filter-based interpolation techniques is provided. Average bitrate savings of around 2.0% compared to an 8-tap FIR filter are observed over the high-efficiency video coding dataset at a similar worst case complexity.
Fractional-sample motion compensation using generalized interpolation

Typical interpolation methods in video coding perform filtering of reference picture samples using FIR filters for motion-compensated prediction. This process can be viewed as a signal decomposition using basis functions which are restricted by the interpolating constraint. Using the concept of generalized interpolation provides a greater degree of freedom for selecting basis functions. We implemented generalized interpolation using a combination of IIR and FIR filters. The complexity of the proposed scheme is comparable to that of an 8-tap FIR filter. Bit rate savings up to 20% compared to the H.264/AVC 6-tap filter are shown.
Subjective performance evaluation of the SVC extension of H.264/AVC

This contribution presents results of the MPEG verification test that was carried out for the new Scalable Video Coding (SVC) Amendment of H.264/AVC. The test consisted of a series of subjective comparisons of SVC and single layer H.264/AVC coding for different application scenarios including conversational applications, broadcasting over mobile channels, and HD broadcasting. The results show that a reasonable degree of spatial and quality scalability can be supported with a bit rate overhead of less than or about 10% and an indistinguishable visual quality compared to the state of the art single layer coding. This paper describes the coding conditions, the test procedure, and presents the results of the SVC verification test.
Synthesized View Distortion Based 3D Video Coding for Extrapolation and Interpolation of Views

In 3D video coding, the Multi-View Video plus Depth (MVD) representation format enables the interpolation of virtual intermediate views in between the original recorded camera views, as well as the extrapolation of views outside this range using view synthesis algorithms. For this, MVD includes depth maps as additional information to be transmitted. The performance of the associated depth data coding can be improved by considering the distortion of synthesized views in the rate-distortion optimization of the encoder. Therefore, this paper evaluates coding gains achieved with encoder optimization with respect to virtual view positions. The impact of the number and the positions of virtual views used as reference in the encoder side optimization process on the quality of the synthesized views is analyzed. Coding results for view interpolation and view extrapolation are provided. Moreover, the encoder-side complexity with respect to the number of reference views is discussed.
Object-based 3-D wavelet coding using layered object representation

In recent years, object/region-based video coding schemes have attracted considerable attention in the image coding community. These schemes are expected to yield better coding results, especially at low bit-rates. In this paper a new object/region-based video coding scheme is presented. It is based on a layered object representation for image groups along with affine motion fields. Thus, groups of images are decomposed into a number of object sequences. Each object sequence is motion compensated, transformed and coded separately using a 3-D wavelet transform. The presented results prove that object/region-based coding schemes can outperform the block-based standards in objective (PSNR) and subjective (visual) quality.
R-D Optimized Multi-Layer Encoder Control for SVC

The scalable video coding (SVC) extension of H.264/AVC was recently standardized. The encoder control that is described in the joint scalable video model for SVC specifies a bottom-up process in which first the base layer and then the enhancement layer is encoded. The base layer is encoded conforming to H.264/AVC without considering its impact on enhancement layers, which limits the achievable enhancement layer coding efficiency. The losses relative to single-layer H.264/AVC coding are unevenly distributed between base and enhancement layers. In this paper, we present a multi-layer encoder control for SVC, by which we can trade off coding efficiency for base and enhancement layers and generally improve the efficiency of spatial and fidelity scalable coding.
Separable Wiener filter based adaptive in-loop filter for video coding

Recent investigations have shown that a non-separable Wiener filter, that is applied inside the motion-compensation loop, can improve the coding efficiency of hybrid video coding designs. In this paper, we study the application of separable Wiener filters. Our design includes the possibility to adaptively choose between the application of the vertical, horizontal, or combined filter. The simulation results verify that a separable in-loop Wiener filter is capable of providing virtually the same increase in coding efficiency as a non-separable Wiener filter, but at a significantly reduced decoder complexity.
Partial Depth Image Based Re-Rendering for Synthesized View Distortion Computation

3D video systems transmit depth maps in order to render synthesized views (SVs) at a receiver. To anticipate this purpose when processing a depth map, a sender-side depth processing algorithm (DPA), e.g. a depth encoder, can also render the SVs, compute their SV distortion (SVD), and adapt to it. This requires a low-complexity algorithm as computational resources are usually limited. We propose such an algorithm in this paper. First, we discuss a measure that relates a depth change to an SVD change using rendering. Then, we present an optimized process combining basic rendering steps, as warping, occlusion handling, interpolation, hole filling, and blending. Furthermore, we analyze which parts of an SV are affected by a depth change and modify the process to re-render only them. The resulting algorithm is significantly less complex than an unoptimized rendering-based variant and quantifies the SVD more accurately than existing estimation methods. The algorithm is used by the 3D-High Efficiency Video Coding reference software encoder as the main method for distortion computation and can also be used by other DPAs.
Rate-constrained coder control and comparison of video coding standards

A unified approach to the coder control of video coding standards such as MPEG-2, H.263, MPEG-4, and the draft video coding standard H.264/AVC (advanced video coding) is presented. The performance of the various standards is compared by means of PSNR and subjective testing results. The results indicate that H.264/AVC compliant encoders typically achieve essentially the same reproduction quality as encoders that are compliant with the previous standards while typically requiring 60% or less of the bit rate.
A Review of the Minimum Maximum Criterion for Optimal Bit Allocation Among Dependent Quantizers

In this paper, we review a general framework for the optimal bit allocation among dependent quantizers based on the minimum maximum (MINMAX) distortion criterion. The pros and cons of this optimization criterion are discussed and compared to the well-known Lagrange multiplier method for the minimum average (MINAVE) distortion criterion. We argue that, in many applications, the MINMAX criterion is more appropriate than the more popular MINAVE criterion. We discuss the algorithms for solving the optimal bit allocation problem among dependent quantizers for both criteria and highlight the similarities and differences. We point out that any problem which can be solved with the MINAVE criterion can also be solved with the MINMAX criterion, since both approaches are based on the same assumptions. We discuss uniqueness of the MINMAX solution and the way both criteria can be applied simultaneously within the same optimization framework. Furthermore, we show how the discussed MINMAX approach can be directly extended to result in the lexicographically optimal solution. Finally, we apply the discussed MINMAX solution methods to still image compression, intermode frame compression of H.263, and shape coding applications.
Fast millimeter wave threat detection algorithm

Millimeter Wave (MMW) imaging systems are currently being used to detect hidden threats. Unfortunately the current performance of detection algorithms is very poor due to the presence of severe noise, the low resolution of MMW images and, in general, the poor quality of the acquired images. In this paper we present a new real time MMW threat detection algorithm based on a tailored de-noising, body and threat segmentation, and threat detection process that outperforms currently existing detection procedures. A complete comparison with a state of art threat detection algorithm is presented in the experimental section.
A wireless video surveillance system with an active camera

This paper introduced a camera surveillance system in wireless communications. The system contains three major modules, PTU (pan-tilt unit) camera control for surveillance video capture, cross-layer control for data compression and transmission, and error concealment for video quality enhancement. Our contribution is twofold. First, a system design for data collection and transmission over wireless networks is presented and is evaluated with physical surveillance equipments. The camera is capable of following the moving target according to the control information. The end-to-end distortion estimation in the delay constrained video coding process takes into account the dynamic channel condition and physical layer modulation and coding scheme (MCS) to determine optimal coding and transmission parameters. Second, multiple error concealment strategies, including interleaving, boundary match and video up-sampling, are applied utilizing the special property of the PTU camera motion.
A compressed video enhancement algorithm

The problem of the enhancement of a low bit-rate compressed video sequence using the information provided by the encoder is investigated in this paper. The proposed algorithm is spatio-temporally adaptive and enforces different degrees of between-block, within-block, and temporal smoothness of the decompressed frames based on macroblock types. The algorithm uses the projections onto the sets that capture the information conveyed by the transmitted data to constrain the solution space. A partially automatic regularization parameter estimation algorithm is also proposed in this paper. An analysis of PSNR gain per macroblock type is presented to cast insight into the compressed video enhancement problem. The experimental results demonstrate the effectiveness of the proposed approach.
A Logarithmic X-Ray Imaging Model for Baggage Inspection: Simulation and Object Detection

In the last years, many computer vision algorithms have been developed for X-ray testing tasks. Some of them deal with baggage inspection, in which the aim is to detect automatically target objects. The progress in automated baggage inspection, however, is modest and very limited compared to what is needed because X-ray screening systems are still being manipulated by human inspectors. In this work, we present an X-ray imaging model that can separate foreground from background in baggage screening. The model can be used in two main tasks: i) Simulation of new X-ray images, where simulated images can be used in training programs for human inspectors, or can be used to enhance datasets for computer vision algorithms. ii) Detection of (threat) objects, where new algorithms can be employed to perform automated baggage inspection or to aid an user in the inspection task showing potential threats. In our model, rather than a multiplication of foreground and background, that is typically used in X-ray imaging, we propose the addition of logarithmic images. This allows the use of linear strategies to superimpose images of threat objects onto X-ray images and the use of sparse representations in order to segment target objects. In our experiments, we simulate new X-ray images of handguns, shuriken and razor blades, in which it is impossible to distinguish simulated and real X-ray images. In addition, we show in our experiments the effective detection of shuriken, razor blades and handguns using the proposed algorithm outperforming some alternative state-of- the-art techniques.
VAPOR: variance-aware per-pixel optimal resource allocation

Characterizing the video quality seen by an end-user is a critical component of any video transmission system. In packet-based communication systems, such as wireless channels or the Internet, packet delivery is not guaranteed. Therefore, from the point-of-view of the transmitter, the distortion at the receiver is a random variable. Traditional approaches have primarily focused on minimizing the expected value of the end-to-end distortion. This paper explores the benefits of accounting for not only the mean, but also the variance of the end-to-end distortion when allocating limited source and channel resources. By accounting for the variance of the distortion, the proposed approach increases the reliability of the system by making it more likely that what the end-user sees, closely resembles the mean end-to-end distortion calculated at the transmitter. Experimental results demonstrate that variance-aware resource allocation can help limit error propagation and is more robust to channel-mismatch than approaches whose goal is to strictly minimize the expected distortion.
Understanding dynamic scenes by hierarchical motion pattern mining

Our work addresses the problem of analyzing and understanding dynamic video scenes. A two-level motion pattern mining approach is proposed. At the first level, single-agent motion patterns are modeled as distributions over pixel-based features. At the second level, interaction patterns are modeled as distributions over single-agent motion patterns. Both patterns are shared among video clips. Compared to other works, the advantage of our method is that interaction patterns are detected and assigned to every video frame. This enables a finer semantic interpretation and more precise anomaly detection. Specifically, every video frame is labeled by a certain interaction pattern and moving pixels in each frame which do not belong to any singleagent pattern or cannot exist in the corresponding interaction pattern are detected as anomalies. We have tested our approach on a challenging traffic surveillance sequence containing both pedestrian and vehicular motions and obtained promising results.
Compressive Blind Image Deconvolution

We propose a novel blind image deconvolution (BID) regularization framework for compressive sensing (CS) based imaging systems capturing blurred images. The proposed framework relies on a constrained optimization technique, which is solved by a sequence of unconstrained sub-problems, and allows the incorporation of existing CS reconstruction algorithms in compressive BID problems. As an example, a non-convex lp quasi-norm with 0 <; p <; 1 is employed as a regularization term for the image, while a simultaneous auto-regressive regularization term is selected for the blur. Nevertheless, the proposed approach is very general and it can be easily adapted to other state-of-the-art BID schemes that utilize different, application specific, image/blur regularization terms. Experimental results, obtained with simulations using blurred synthetic images and real passive millimeter-wave images, show the feasibility of the proposed method and its advantages over existing approaches.
Challenges and Opportunities in Video Transmission

Supporting video communication over lossy channels such as wireless networks and the Internet is a challenging task due to the stringent quality of service (QoS) required by video applications and the many channel impairments. Two important QoS characteristics for video are the degree of signal distortion and the transmission delay. Another important consideration is the cost associated with transmission, for example, the energy consumption in the wireless channel case and the cost for differentiated services in the Internet (with DiffServ) case. In this presentation we consider the joint adaptation of the source coding parameters, such as the quantization step-size and prediction mode, along with the physical layer resources, such as the transmission rate and power. Our goal is to provide acceptable QoS while taking into account system constraints such as the energy utilization. We discuss a general framework that allows a number of "resource/distortion" optimal formulations for balancing the requirements of different applications. We conclude the presentation with some of the grand opportunities and challenges in designing and developing video communication systems.
Local feature extraction for video copy detection in a database

In this paper a new content-based copy identification method for video sequences is presented. It is robust to a number of image transformations and particulary robust to compression artifacts. A scale and rotation invariant local image descriptor for corner points in detected key frames is proposed based on a generalized Radon transform. In addition, a distance similarity metric is used that fuses intensity and geometry information to compare key frames extracted using a scene detection algorithm. Furthermore, to achieve low querying computational complexity a DP approach is employed. Experimental results demonstrate the effectiveness of our approach.
Optimized Bit Extraction Using Distortion Estimation in the Scalable Extension of H.264/AVC

The newly adopted scalable extension of H.264/AVC video coding standard (SVC), demonstrates significant improvements in coding efficiency in addition to an increased degree of supported scalability relative to the scalable profiles of prior video coding standards. For efficient adaptation of SVC bit streams to intermediate bit rates, the concept of quality layers has been introduced in the design of the SVC. The concept of quality layers allow a rate distortion (RD) optimal bit extraction; However, existing Quality Layer assignment methods do not consider all network abstraction layer (NAL) units from different layers for the optimization. In this paper, we first propose a technique to accurately and efficiently estimate the quality degradation resulting from discarding an arbitrary number of NAL units from multiple layers of a bitstream. Then, we utilize this distortion estimation technique to assign quality layers to NAL units for a more efficient extraction. Experimental results show that a significant gain can be achieved by the proposed schemes.
Quantization optimized H.264 encoding for traffic video tracking applications

The compression of video can reduce the accuracy of post-compression tracking algorithms. This is problematic for centralized applications such as traffic surveillance systems, where remotely captured and compressed video is transmitted to a central location for tracking. We propose a low complexity optimization framework that automatically identifies video features critical to tracking and concentrates bitrate on these features via quantization tables. Using the H.264 video coding standard and two commonly used state-of-the-art trackers we show that our algorithm allows for over 60% bitrate savings while maintaining comparable tracking accuracy.
So Simple evan a child can do it

eople had warned us this would happen. After a combined total of 28 columns, we were indeed wondering I what to write about. Maybe Novem-ber just isn't a particularly inspiring month. Maybe DSP is exciting only in the warmer months. Maybe the artifice of writing about November when it's really September finally became too stressful, as we sat emptp-screencd in OUT shared despair. Actually this column has a longer history, dating back to last March's ICASSP99 in Phoenix, AZ, when we met in a hallway and commiserated about the relentlessness of tlie bimonthly column. We recalled occasional discussions about publishing the D E E S&nal Processing Magazine as a monthly periodical and secretly prayed " Not during my term. " Then, through some random wandering of the conversation, we mentioned how our children enjoyed being mentioned in the columns; how even children came in touch witli signal processing these days; and hence (eureka!), how our children could even write one of our columns! Ed Delp joined in the discussion and it eventually progressed to one of those " I'll do it, if you'll do it " conversations that can so often lead to foolhardy actions. In this case, though, the outcome is, we believe, morc enlightening than foolhardy, and does indeed say some significant things about the field of signal processing. What follows are three short pieces written by our childrcn: Ercni Icatsaggelos (Aggelos' 10-year-old daughter), Caitlin Adam (Leah's 10-year-old daughter), and Ed Delp IV (Ed Delp's 16-year-old son). We'U come back at tlie end to give our perspectives , but first, listen to the luds. What a digital signal means to me is a voice, a so~md, or an image, that is either on the computer or the Internct. What Digital Signal Processing (DSP) means to me is changing such a voice, sound, or image to help male it sound or look better, dearer, or sharper. I have learned about DSP at the conferences I have been to in Santorini, Greece, and Antalya, Turkey. Whcn I went to Santorini, I was seven years old. Two years later, this past summer, I went to Antalya. I think both coderences were interesting and fun. During dicse conferences, I learned about different kinds ofDSP. To be good in DSP you nccd to be good in math. I especially enjoyed leariling about how they use DSP in music. Alimed (the professor that tallred …
Virtual touring: A Content Based Image Retrieval application

This paper proposes a Content Based Image Retrieval (CBIR) application for searching landmarks and buildings in a city using a smartphone. A user can snap a picture of the building using his smartphone. The application is able to quickly and accurately find the name of the building along with many other interesting information, such as the history of the building and its Wi-Fi availability. We present a novel client-server CBIR application that combines Laplacian-SIFT for feature descriptor, multiple kd-trees for indexing and two levels of geometric verification. We present back-end and front-end Application Programming Interfaces (API) for client-server CBIR applications and we propose a distributed system architecture to support multiple client requests. The application consists of two user interfaces, a web interface and a mobile interface. Image retrieval results demonstrate the accuracy of the system in recognizing buildings.
Regularized edge detection

It has been observed that the edge detection problem in image processing is ill-posed. The ill-posed aspect of this problem is the differentiation step, which is an explicit operation in many schemes for finding edges in images. Differentiation is very sensitive to noise and its discrete version is an ill-conditioned operation. The authors concentrate on regularizing the differentiation operation and incorporating it in an edge detection scheme. They consider both a direct implementation of Miller regularization and one involving projections onto convex sets.<<ETX>>