nils-herrmann/clean_scirepeval huggingface dataset:
Extending SVC by Content-adaptive Spatial Scalability

This paper provides details on a complete integration of Content-adaptive Spatial Scalability (CASS) into the scalable video coding extension of H.264/AVC (SVC). CASS enables the efficient encoding of a high-quality bit stream that contains several versions of an original image sequence. Thereby, each such image sequence has been created by content-adaptive and art directed retargeting to different display aspect-ratios and/or resolutions. Non-linear dependencies between spatial layers, which have been introduced through content-adaptive retargeting, are exploited by a generalization of the three inter-layer prediction tools of SVC, i.e. by content-adaptive inter-layer texture, motion and residual prediction. The CASS extended SVC enables the transmission of video content which has been specifically adapted in an art-directed way to multiple display configurations (e.g. to SD and HD displays with 4:3 and 16:9 aspect-ratios, respectively) using a single compressed bit stream. With our extension, video content of higher semantic quality can be transmitted in a scalable way by introducing an average overhead in bit rate of 9.3%.
Computational sports broadcasting: Automated director assistance for live sports

Live sports broadcast is seeing a large increase in the number of cameras used for filming. More cameras can provide better coverage of the field and a wider range of experiences for viewers. However, choosing optimal cameras for broadcast demands a high level of concentration, awareness and experience from sports broadcast directors. We present an automatic assistant to help select likely candidates from a large array of possible cameras. Sports directors can then choose the final broadcast camera from the reduced suggestion set. Our assistant uses both widely acknowledged cinematography guidelines for sports directing, as well as a data-driven approach that learns specific styles from directors.
Efficient Compression of Multi-View Depth Data Based on MVC

This paper presents a method for efficient compression of multi-view depth data based on our multi-view video coding approach for color data. The idea is to exploit statistical dependencies from both temporal and inter-view reference pictures for prediction. For this purpose a multi-view video data set including color and depth information is analyzed in terms of coding efficiency. Coding experiments using prediction structures with and without inter-view reference pictures are performed with multi-view depth data and compared to multi-view video coding. The results show that additionally applying inter-view prediction to temporal prediction with hierarchical B pictures improves coding efficiency for depth as well as color, reporting average gains in PSNR-Y of 0.5 dB for depth and 0.3 dB for color.
Robust global motion estimation using a simplified M-estimator approach

Global motion estimation is an important task in a variety of video processing applications, such as coding, segmentation, classification/indexing or mosaicing. Due to the possible presence of differently moving foreground objects and other sources of distortions, robust methods such as M-estimators have to be applied. We present a simplified implementation of a robust M-estimator for global motion estimation that does not increase the computational complexity significantly compared to a non-robust estimator, while providing excellent results in terms of estimation accuracy. Additionally, unstructured image regions are detected and rejected for the estimation. This avoids aperture problems, that can have an bad impact especially on robust estimators that rely on a certain error measure.
A computational model for perception of stereoscopic window violations

Creating a computational model for stereoscopic 3D perception is a highly complex undertaking. As one step towards this goal, this paper investigates stereoscopic window violation artifacts, which often interfere with artistic freedom and constrain the comfortable depth volume. Window violations need to be compensated for in most 3D feature movies. Currently this is done in an ad-hoc manner due to a limited understanding of the problem. In this work, we present a model predicting problematic window violations that are visually disturbing. The model parameters were defined through psychophysical experiments on simple stimuli. Then the model was calibrated and validated on real, complex stereoscopic images. Finally, we present a system to provide visualization of problematic stereoscopic window violations as well as details for how to correct them.
Free viewpoint video extraction, representation, coding, and rendering

Free viewpoint video provides the possibility to freely navigate within dynamic real world video scenes by choosing arbitrary viewpoints and view directions. So far, related work only considered free viewpoint video extraction, representation, and rendering methods. Compression and transmission has not yet been studied in detail and combined with the other components into one complete system. In this paper, we present such a complete system for efficient free viewpoint video extraction, representation, coding, and interactive rendering. Data representation is based on 3D mesh models and view-dependent texture mapping using video textures. The geometry extraction is based on a shape-from-silhouette algorithm. The resulting voxel models are converted into 3D meshes that are coded using MPEG-4 SNHC tools. The corresponding video textures are coded using an H.264/AVC codec. Our algorithms for view-dependent texture mapping have been adopted as an extension of MPEG-4 AFX. The presented results illustrate that based on the proposed methods a complete transmission system for efficient free viewpoint video can be built.
DRAM or no-DRAM? Exploring linear solver architectures for image domain warping in 28 nm CMOS

Solving large optimization problems within the energy and cost budget of mobile SoCs in real-time is a challenging task and motivates the development of specialized hardware accelerators. We present an evaluation of different linear solvers suitable for least-squares problems emanating from image processing applications such as image domain warping. In particular, we estimate implementation costs in 28 nm CMOS technology, with focus on trading on-chip memory vs. off-chip (DRAM) bandwidth. Our assessment shows large differences in circuit area, throughput and energy consumption and aims at providing a recommendation for selecting a suitable architecture. Our results emphasize that DRAM-free accelerators are an attractive choice in terms of power consumption and overall system complexity, even though they require more logic silicon area when compared to accelerators that make use of external DRAM.
Increasing the Accuracy of the Space-Sweeping Approach to Stereo Reconstruction, using Spherical Backprojection Surfaces

In this paper, interest is focused on the accurate and time-efficient stereo reconstruction, for the purpose of generating 3D animated scenes from multiple synchronized videos. The plane-sweeping approach is reviewed as relevant to the goal of time-efficiency, since its execution can be optimized on a GPU. A method compatible for optimization on the GPU is proposed as a more accurate alternative to plane sweeping and to the derived visibility computation. The method is compared to plane sweeping as to its accuracy, by evaluating the backprojected 3D model against independent views and using n-fold cross validation to estimate the peak signal to noise ratio (PSNR). Finally, the method's output is casted integratable with multicamera stereo reconstruction frameworks.
Real-time temporally coherent local HDR tone mapping

Subjective studies showed that most HDR video tone mapping operators either produce disturbing temporal artifacts, or are limited in their local contrast reproduction capability. Recently, both these issues have been addressed by a novel temporally coherent local HDR tone mapping method, which has been shown, both qualitatively and through a subjective study, to be advantageous compared to previous methods. However, this method's high-quality results came at the cost of a computationally expensive workflow that could only be executed offline. In this paper, we present a modified algorithm which builds upon the previous work by redesigning key components to achieve real-time performance. We accomplish this by replacing the optical flow based per-pixel temporal coherency with a tone-curve-space alternative. This way we eliminate the main computational burden of the original method with little sacrifice in visual quality.
Reliability-based generation and view synthesis in layered depth video

In this paper, a system for video rendering on multiscopic 3D displays is considered where the data is represented as layered depth video (LDV). This representation consists of one full or central video with associated per-pixel depth and additional residual layers. Thus, only one full view with additional residual data needs to be transmitted. The LDV data is used at the receiver to generate all intermediate views for the display. The paper presents the LDV layer extraction as well as the view synthesis, using a scene reliability-driven approach. Here, unreliable image regions are detected and in contrast to previous approaches the residual data is enlarged to reduce artifacts in unreliable areas during rendering. To provide maximum data coverage, the residual data remains at its original positions and will not be projected towards the central view. The view synthesis process also uses this reliability analysis to provide higher quality intermediate views than previous approaches. As a final result, high quality intermediate views for an existing 9-view auto-stereoscopic display are presented, which prove the suitability of the LDV approach for advanced 3D video (3DV) systems.
Content creation for stereoscopic 3D and beyond

2010 was the year of breakthrough for 3D video. Stereoscopic 3D is established in cinema, on Blu-ray, TV, PCs, laptops, and mobile devices. Since technology for stereo 3D is mature and content creation is understood well enough, these developments are expected to be sustainable this time. Most current systems rely on classical approaches to 3D video, i.e., representation as stereo or multiview video, coding and transmission using simulcast, frame-compatible composition or MVC. More advanced “next generation” approaches exploit some kind of understanding of the 3D scene geometry such as depth or disparity, in order to extend functionality and increase efficiency. This includes for instance flexible adjustment of depth impression to viewing conditions and user preferences or support of autostereoscopic multiview displays. Also content creation for classical stereo 3D can greatly benefit from such 3D geometry aware processing. Naturally such advanced 3D video representation formats require advanced processing algorithms, e.g., to extract 3D geometry and to render virtual views. Such advanced 3D video representation and processing will be the focus of this talk.
The Effect of Depth Compression on Multiview Rendering Quality

This paper presents a comparative study on different techniques for depth-image compression and its implications on the quality of multiview video plus depth virtual view rendering. A novel coding algorithm for depth images that concentrates on their special characteristics, namely smooth regions delineated by sharp edges, is compared to H.264 intra-coding with depth- images. These two coding techniques are evaluated in the context of multiview video plus depth representations, where depth information is used to render virtual intermediate camera views of the scene. Therefore it is important to evaluate the influence of depth-image coding artifacts on the quality of rendered virtual views. The results of this evaluation show, that the coding algorithm specialized on the characteristics of depth images outperforms H.264 intra-coding, although its RD-performance is worse.
Video content and structure description based on keyframes, clusters and storyboards

In this paper we present a novel system to extract keyframes, shot clusters and structural storyboards for video content description, which can be used for a variety of summarization, visualization, classification, indexing and retrieval applications. The system automatically selects an appealing set of keyframes and creates meaningful clusters of shots. It further identifies sections that appear recurrently, which are called anchors, and typically divide television shows into different parts. This information about anchors can then be used to browse video content in a new fashion. Finally, our system creates a new type of interactive storyboard suitable to visualize and analyze the structure of the video in a novel way.
Coding Algorithms for 3DTV—A Survey

Research efforts on 3DTV technology have been strengthened worldwide recently, covering the whole media processing chain from capture to display. Different 3DTV systems rely on different 3D scene representations that integrate various types of data. Efficient coding of these data is crucial for the success of 3DTV. Compression of pixel-type data including stereo video, multiview video, and associated depth or disparity maps extends available principles of classical video coding. Powerful algorithms and open international standards for multiview video coding and coding of video plus depth data are available and under development, which will provide the basis for introduction of various 3DTV systems and services in the near future. Compression of 3D mesh models has also reached a high level of maturity. For static geometry, a variety of powerful algorithms are available to efficiently compress vertices and connectivity. Compression of dynamic 3D geometry is currently a more active field of research. Temporal prediction is an important mechanism to remove redundancy from animated 3D mesh sequences. Error resilience is important for transmission of data over error prone channels, and multiple description coding (MDC) is a suitable way to protect data. MDC of still images and 2D video has already been widely studied, whereas multiview video and 3D meshes have been addressed only recently. Intellectual property protection of 3D data by watermarking is a pioneering research area as well. The 3D watermarking methods in the literature are classified into three groups, considering the dimensions of the main components of scene representations and the resulting components after applying the algorithm. In general, 3DTV coding technology is maturating. Systems and services may enter the market in the near future. However, the research area is relatively young compared to coding of other types of media. Therefore, there is still a lot of room for improvement and new development of algorithms.
Three-Dimensional Video Postproduction and Processing

This paper gives an overview of the state-of-the-art in 3-D video postproduction and processing as well as an outlook to remaining challenges and opportunities. First, fundamentals of stereography are outlined that set the rules for proper 3-D content creation. Manipulation of the depth composition of a given stereo pair via view synthesis is identified as the key functionality in this context. Basic algorithms are described to adapt and correct fundamental stereo properties such as geometric distortions, color alignment, and stereo geometry. Then, depth image-based rendering is explained as the widely applied solution for view synthesis in 3-D content creation today. Recent improvements of depth estimation already provide very good results. However, in most cases, still interactive workflows dominate. Warping-based methods may become an alternative for some applications in the future, which do not rely on dense and accurate depth estimation. Finally, 2-D to 3-D conversion is covered, which is an important special area for reuse of existing legacy 2-D content in 3-D. Here various advanced algorithms are combined in interactive workflows.
Automatic content creation for multiview autostereoscopic displays using image domain warping

Content creation for autostereoscopic display is a widely unresolved task. Typical methods rely on view synthesis based on depth image based rendering. Our method applies purely image domain warping instead. Input video is analyzed and information about sparse disparity, vertical edges and saliency is extracted. A constrained energy minimization problem is formulated and efficiently solved. The resulting image warping functions are used to synthesize novel views. Our approach is fully automatic, accurate, and reliable. Disocclusions and related artifacts are avoided due to smooth, saliency-driven warping functions. Our method also works well for extrapolation of views in a limited range, thus supporting multiview creation from stereo input, which is the most relevant use case scenario
Semi-automatic video object segmentation by advanced manipulation of segmentation hierarchies

For applications that require very accurate video object segmentations, semi-automatic algorithms are typically used, which help operators to minimize the annotation time, as off-the-shelf automatic segmentation techniques are still far from being precise enough in this context. This paper presents a novel interface based on a click-and-drag interaction that allows to rapidly select regions from state-of-the-art segmentation hierarchies. The interface is very responsive, allows to obtain very accurate segmentations, and is designed to minimize the human interaction. To evaluate the results, we provide a new set of object video ground truth data.
Automatic View Synthesis by Image-Domain-Warping

Today, stereoscopic 3D (S3D) cinema is already mainstream, and almost all new display devices for the home support S3D content. S3D distribution infrastructure to the home is already established partly in the form of 3D Blu-ray discs, video on demand services, or television channels. The necessity to wear glasses is, however, often considered as an obstacle, which hinders broader acceptance of this technology in the home. Multiviewautostereoscopic displays enable a glasses free perception of S3D content for several observers simultaneously, and support head motion parallax in a limited range. To support multiviewautostereoscopic displays in an already established S3D distribution infrastructure, a synthesis of new views from S3D video is needed. In this paper, a view synthesis method based on image-domain-warping (IDW) is presented that automatically synthesizes new views directly from S3D video and functions completely. IDW relies on an automatic and robust estimation of sparse disparities and image saliency information, and enforces target disparities in synthesized images using an image warping framework. Two configurations of the view synthesizer in the scope of a transmission and view synthesis framework are analyzed and evaluated. A transmission and view synthesis system that uses IDW is recently submitted to MPEG's call for proposals on 3D video technology, where it is ranked among the four best performing proposals.
A multi-feature description scheme for image and video database retrieval

This paper reports about a description scheme for visual information content, which has been developed in the context of the forthcoming MPEG-7 standard. The system supports similarity-based retrieval of visual (image and video) data along feature axes like color, texture, shape/geometry and motion. The descriptors for these features have been developed in a way such that invariance against common transformations of visual material, e.g. filtering, contrast/color manipulation, resizing etc. is achieved, and that they are fitted to human perception properties. Furthermore, descriptors have been designed that allow a fast, hierarchical search procedure. A search engine has been developed on the basis of this description scheme, which allows similarity-based retrieval from an image or video database. The results show that efficient search and retrieval in visual database systems is possible based on a normative feature description such as MPEG-7.
A Vario-power ME architecture using content-based subsample algorithm

The motion estimator is a key element in many video compression systems and it tends to dominate the power consumption in them. With increasing demand of portable, power-aware multimedia devices, an architecture that can be flexible in both power consumption and compression quality is essential. To meet this requirement, this paper presents a novel power-aware architecture, called the Vario-power architecture, for the motion estimation. Based on a semi-systolic array with the content-based subsample algorithm, the architecture real-time disables some processing elements to reduce power consumption. By performing the edge extraction first, a threshold is then set as the criterion of whether to enable or disable processing elements and thus the switch activities of the system can be reduced. As the simulation shows, the architecture may operate at different power consumption modes according to the remaining capacity of the battery pack giving little quality degradation and the power overhead under 0.36%.
Fourth-Order Cascaded $\Sigma \Delta$  Modulator Using Tri-Level Quantization and Bandpass Noise Shaping for Broadband Telecommunication Applications

This paper presents a 14-bit cascaded sigma-delta modulator for broadband telecommunication applications. The modulator is a 2-1-1 cascaded architecture that employs a resonator-based topology in the first stage, three tri-level quantizers, and two different pairs of reference voltages. As shown in the experimental result, for a 2.5-MHz signal bandwidth, the modulator achieves a dynamic range of 86 dB and a peak signal-to-noise and distortion ratio of 78.5 dB with an oversampling ratio of 16. The proposed modulator including reference voltage buffers and bandgap circuitry dissipates 62.5 mW from a 2.5-V supply. The active area is 1.2-mm2 in a 0.25-mum CMOS technology.
Offline State-of-Health Estimation for High-Power Lithium-Ion Batteries Using Three-Point Impedance Extraction Method

This paper presents an offline state-of-health (SoH) estimation based on charge transfer resistance for high-power lithium-ion (Li-ion) batteries, such as lithium iron phosphate (LFP) batteries. As shown in the experimental results, the charge transfer resistance has a great aging change with battery degradation and good abilities against state-of-charge (SoC) drift and external resistance variation in the impedance parameter set of a single-time-constant equivalent circuit model (ECM), including ohmic resistance, charge transfer resistance, double-layer capacitance, and time constant, for SoH estimation. A fast and efficient three-point (TP) impedance extraction method is also proposed in this paper for accurately extracting the charge transfer resistance in offline SoH estimation. The results of long-term cycling test demonstrate that the TP impedance extraction method can successfully indicate the SoH of LFP batteries with low estimation error of 6.1%.