nils-herrmann/clean_scirepeval huggingface dataset:
A new 6D motion gesture database and the benchmark results of feature-based statistical recognition

A motion gesture can be represented by a 3D spatial trajectory and maybe augmented by additional 3 dimensions of orientation. Depending on the tracking technology in use, the 6D motion gesture can be tracked explicitly with the position and orientation or implicitly with the acceleration and angular speed. In this work, we first present a motion gesture database which contains both explicit and implicit 6D motion information. This database allows us to compare the recognition performance over different tracking signals on a common ground. Our main contribution is to investigate the relative effectiveness of various feature dimensions in motion gesture recognition. Using a simple and primitive recognizer, we evaluate the recognition results of both explicit and implicit motion data. In our experiments, both user dependent and user independent cases are addressed. We also propose two general techniques to improve the recognition accuracy: smoothing and the temporal extension. Our pilot study produces benchmark results that give an insight into the attainable recognition accuracy with different tracking devices.
Three-dimensional position and amplitude VLC coding in H.264/AVC

Hybrid variable length coding (HVLC) was recently proposed as a novel entropy coding scheme for block-based image and video compression, which divides each transform block into low frequency (LF) region and high frequency (HF) region and codes them differently. To take advantage of the clustered nature of the nonzero coefficients in the LF region, a two-dimensional position and one- dimensional amplitude coding scheme (2DP1DA) was also proposed, which codes the 2D position information, i.e., run of consecutive zero-valued coefficients and run of consecutive nonzero coefficients, jointly by a 2D code table. To further improve the coding efficiency, joint position and amplitude coding is desirable. In this paper, we propose a three-dimensional position and amplitude coding scheme (3DPA) to jointly code the position and amplitude information for each nonzero cluster by a 3D code table. The experimental results show that HVLC with 3DPA for coding LF region achieves about 3.2%~3.7% bit rate reduction for a wide range of quantization parameters (QP) compared with CAVLC in H.264. Moreover, the 3DPA scheme is a compact way to realize the joint position and amplitude coding and it is very suitable for context-adaptive HVLC design.
The Mosaic Camera: Streaming, Coding and Compositing Experiments

The HP Fan Camera is a panoramic mosaic king camera that is a composite of 24-imager array system. Streaming the captured video is a challenging problem due to several factors such as the large bandwidth requirements, the limited capabilities of the client's machines, and our desire to provide independent viewing controls for users. In the process of developing an optimal rate controller for the HP Fan Camera we developed a client-server framework for multi-camera streaming and performed a set of experiments using various bandwidth allocation schemes. From our preliminary research, we found that sending individual streams of the cameras over the network provides more interactivity to the end users and requires less bandwidth in case the behavior of the end users is aggressive in scene selection. In this paper we present this framework and share the results of our conducted experiments.
Unsupervised Uncertainty Estimation Using Spatiotemporal Cues in Video Saliency Detection

In this paper, we address the problem of quantifying the reliability of computational saliency for videos, which can be used to improve saliency-based video processing algorithms and enable more reliable performance and objective risk assessment of saliency-based video processing applications. Our approach to quantify such reliability is twofold. First, we explore spatial correlations in both the saliency map and the eye-fixation map. Then, we learn the spatiotemporal correlations that define a reliable saliency map. We first study spatiotemporal eye-fixation data from the public CRCNS data set and investigate a common feature in human visual attention, which dictates a correlation in saliency between a pixel and its direct neighbors. Based on the study, we then develop an algorithm that estimates a pixel-wise uncertainty map that reflects our supposed confidence in the associated computational saliency map by relating a pixel’s saliency to the saliency of its direct neighbors. To estimate such uncertainties, we measure the divergence of a pixel, in a saliency map, from its local neighborhood. In addition, we propose a systematic procedure to evaluate uncertainty estimation performance by explicitly computing uncertainty ground truth as a function of a given saliency map and eye fixations of human subjects. In our experiments, we explore multiple definitions of locality and neighborhoods in spatiotemporal video signals. In addition, we examine the relationship between the parameters of our proposed algorithm and the content of the videos. The proposed algorithm is unsupervised, making it more suitable for generalization to most natural videos. Also, it is computationally efficient and flexible for customization to specific video content. Experiments using three publicly available video data sets show that the proposed algorithm outperforms state-of-the-art uncertainty estimation methods with improvement in accuracy up to 63% and offers efficiency and flexibility that make it more useful in practical situations.
Scale selective extended local binary pattern for texture classification

In this paper, we propose a new texture descriptor, scale selective extended local binary pattern (SSELBP), to characterize texture images with scale variations. We first utilize multi-scale extended local binary patterns (ELBP) with rotation-invariant and uniform mappings to capture robust local microand macro-features. Then, we build a scale space using Gaussian filters and calculate the histogram of multi-scale ELBPs for the image at each scale. Finally, we select the maximum values from the corresponding bins of multi-scale ELBP histograms at different scales as scale-invariant features. A comprehensive evaluation on public texture databases (KTH-TIPS and UMD) shows that the proposed SSELBP has high accuracy comparable to state-of-the-art texture descriptors on gray-scale-, rotation-, and scale-invariant texture classification but uses only one-third of the feature dimension.
Adaptive Multi-Resolution Coding for 3D Scenes using Vector Quantization

This paper proposes a multi-resolution coding system for 3D graphic scenes using vector quantization (VQ). For improved compression efficiency, the proposed system generates the VQ codebook for a given 3D scene using mesh objects contained in the scene database instead of separate training models. Furthermore, objects are weighted according to their unequal display importance resulting from interactions, which is incorporated into both codebook training and rate-distortion optimization. Empirical results are presented to verify the efficiency of the proposed coding system.
Rate-Constrained Distributed Estimation in Wireless Sensor Networks

In this paper, we consider the distributed parameter estimation in wireless sensor networks where a total bit rate constraint is imposed. We study the optimal tradeoff between the number of active sensors and the quantization bit rate for each active sensor to minimize the estimation mean-square error (MSE). To facilitate the solution, we first introduce a concept of equivalent 1-bit MSE function. Next, we present an optimal distributed estimation algorithm for homogeneous sensor networks based on minimizing the equivalent 1-bit MSE function. Then, we present a quasi-optimal distributed estimation algorithm for heterogeneous sensor networks, which is also based on the equivalent 1-bit MSE function, and the upper bound of the estimation MSE of the proposed algorithm is addressed. Furthermore, a theoretical nonachievable lower bound of the estimation MSE under the total bit rate constraint is stated and it is shown that our proposed algorithm is quasi-optimal within a factor 2.2872 of the theoretical lower bound. Simulation results also show that significant reduction in estimation MSE is achieved by our proposed algorithm when compared with other uniform methods
Multistreaming of 3-D Scenes With Optimized Transmission and Rendering Scalability

Three-dimensional (3D) graphic scenes require considerable network bandwidth to be transmitted and computing power to be rendered on a user's terminal. Toward high-quality display in real time, we propose a sender-driven mechanism for streaming 3D scenes in a resource-constrained environment. In doing so, objects are encoded into multiresolutions to provide transmission and rendering scalability, and a weighted distortion metric is developed to measure the quality of a scene rendered with multiresolution objects, modeling objects' unequal importance regarding display. To preserve the manipulation independency of multiple objects in data delivery while provide preferential treatment for different objects as well as different layers of each object, transmission of the objects is performed over multiple streams in a partially sequenced and partially reliable fashion. A rate-distortion optimization framework is developed, which determines an optimal level of reliability for every chunk of data in each stream, taking into account the rendering importance of the object, the distortion-rate performance of the data chunks, and the statistics of the network link. Compared with heuristical methods, simulation results show that the proposed framework maximizes the display quality of the scene while minimizing the amount of data that needs to be processed by the client's rendering engine
The role of perceptual texture dissimilarity in automating seismic data interpretation

Automating the interpretation of post-migrated seismic data would considerably reduce the time and cost of the interpretation process. In this paper, we study the role of texture perception in automating the delineation of an important sub-surface structure: salt dome. First, we propose a new seismic attribute, the gradient of texture, based on which we develop a framework for detecting and delineating salt domes. Since the new attribute can be defined based on a variety of dissimilarity measures, we define a perceptual dissimilarity measure and compare its performance against several other perceptual and non-perceptual texture dissimilarity measures in delineating salt domes in a real seismic data set. Our experimental evaluation reveals that results based perceptual measures are more consistent with human-interpreted results. Therefore, an important contribution of this paper is confirming that the human interpretation process does not only rely on geological and geophysical knowledge, but it also relies on the visual perception of 2D seismic data. Further, the proposed perceptual dissimilarity measure yields the best result and computational efficiency among all studied measures.
Hierarchical Hole-Filling For Depth-Based View Synthesis in FTV and 3D Video

Three-dimensional television (3DTV) is believed to be the future of television broadcasting that would replace current 2D HDTV technology. Future 3DTV would bring a more life-like and visually immersive home entertainment experience, in which users will have the freedom to navigate through the scene to choose a different viewpoint. A desired view can be synthesized at the receiver side using depth image-based rendering (DIBR). While this approach has many advantages, one of the key challenges in DIBR is how to fill the holes caused by disocclusion regions and wrong depth values. In this paper, we propose two new approaches for disocclusion removal in DIBR. Both approaches namely hierarchical hole-filling (HHF) and depth adaptive hierarchical hole-filling eliminate the need for any smoothing or filtering of the depth map. Both techniques use a pyramid-like approach to estimate the hole pixels from lower resolution estimates of the 3D wrapped image. The lower resolution estimates involve a pseudo zero canceling plus Gaussian filtering of the wrapped image. The depth adaptive HHF incorporates the depth information to produce a higher resolution rendering around previously occluded areas. Experimental results show that HHF and depth adaptive HHF yield virtual images and stereoscopic videos that are free of geometric distortions and a better rendering quality both subjectively and objectively than traditional hole-filling approaches.
Traffic Signs in the Wild: Highlights from the IEEE Video and Image Processing Cup 2017 Student Competition [SP Competitions]

Robust and reliable traffic sign detection is necessary to bring autonomous vehicles onto our roads. State-of-the-art algorithms successfully perform traffic sign detection over existing databases that mostly lack severe challenging conditions. VIP Cup 2017 competition focused on detecting such traffic signs under challenging conditions. To facilitate such task and competition, we introduced a video dataset denoted as CURE-TSD that includes a variety of challenging conditions. The goal of this challenge was to implement traffic sign detection algorithms that can robustly perform under such challenging conditions. In this article, we share an overview of the VIP Cup 2017 experience including competition setup, teams, technical approaches, participation statistics, and competition experience through finalist teams members' and organizers' eyes.
Trajectory triangulation: 3D motion reconstruction with ℓ1 optimization

In this paper, we first explain the formulation of the trajectory triangulation: 3D reconstruction of a moving point from a series of 2D projections. The system has to be overconstrained to be solved by least squares techniques. We take advantage of the sparseness of real-world motions in the transformed domain, and borrow the concept of compressive sampling to reformulate the problem with ℓ1 optimization so that it is possible to reconstruct the trajectory even in an underconstrained system. Thus, fewer measurements are needed to reconstruct a 3D trajectory of even larger bandwidth coverage. We conduct experiments on both synthetic and real-world motion data to verify our proposed method, and compare the reconstruction results based on ℓ1 and ℓ2 optimization.
BLeSS: Bio-inspired low-level spatiochromatic similarity assisted image quality assessment

This paper proposes a biologically-inspired low-level spatiochromatic-model-based similarity method (BLeSS) to assist full-reference image-quality estimators that originally oversimplify color perception processes. More specifically, the spatiochromatic model is based on spatial frequency, spatial orientation, and surround contrast effects. The assistant similarity method is used to complement image-quality estimators based on phase congruency, gradient magnitude, and spectral residual. The effectiveness of BLeSS is validated using FSIM, FSIMc and SR-SIM methods on LIVE, Multiply Distorted LIVE, and TID 2013 databases. In terms of Spearman correlation, BLeSS enhances the performance of all quality estimators in color-based degradations and the enhancement is at 100% for both feature- and spectral residualbased similarity methods. Moreover, BleSS significantly enhances the performance of SR-SIM and FSIM in the full TID 2013 database.
MIQM: A novel Multi-view Images Quality Measure

Although several subjective and objective quality assessment methods have been proposed in the literature for images and videos from single cameras, no comparable effort has been devoted to the quality assessment for multi-camera images. With the increasing popularity of multi-view applications, quality assessment for multi-camera images and videos is becoming fundamental to the development of these applications. The quality of images, which are captured by a multi-view system, are affected by multiple factors such as camera configuration, number of cameras, and the calibration process. In the process of developing an objective metric specifically designed for multi-camera systems, we identified two types of visual distortions in multi-view images: photometric distortions and geometric distortions. In this paper, we show that in the presence of well defined reference these distortions can be translated into luminance, contrast, spatial motion and edge-based structure components. Then, we propose different index values that can quantify these components. We provide several examples to demonstrate the correlation between each of these components and the corresponding index metric. Then, we combine these indexes into one Multi-view Image Quality Measure (MIQM). The results and examples show that not only MIQM can capture the perceptual quality of multi-view images it also outperforms the Structural SIMilarity(SSIM) measure for single-view images quality assessment.
UNIQUE: Unsupervised Image Quality Estimation

In this letter, we estimate perceived image quality using sparse representations obtained from generic image databases through an unsupervised learning approach. A color space transformation, a mean subtraction, and a whitening operation are used to enhance descriptiveness of images by reducing spatial redundancy; a linear decoder is used to obtain sparse representations; and a thresholding stage is used to formulate suppression mechanisms in a visual system. A linear decoder is trained with 7 GB worth of data, which corresponds to 100 000 8 × 8 image patches randomly obtained from nearly 1000 images in the ImageNet 2013 database. A patch-wise training approach is preferred to maintain local information. The proposed quality estimator UNIQUE is tested on the LIVE, the Multiply Distorted LIVE, and the TID 2013 databases and compared with 13 quality estimators. Experimental results show that UNIQUE is generally a top performing quality estimator in terms of accuracy, consistency, linearity, and monotonic behavior.
Multimedia Quality Assessment [DSP Forum]

This IEEE Signal Processing Magazine forum discusses the latest advances and challenges in multimedia quality assessment. The forum members bring their expert insights into issues such as perceptual models and quality measures for future applications such as three-dimensional (3-D) videos and interactivity media. The invited forum members are Al Bovik (University of Texas), Chris Plack (University of Manchester), Ghassan AlRegib (Georgia Institute of Technology), Joyce Farrell (Stanford University), Patrick Le Callet (University de Nantes), Quan Huynh-Thu (Tech-nicolor), Sebastian Möller (Deutsche Telekom Labs, TU Berlin), and Stefan Winkler (Advanced Digital Sciences Center). The moderator of this forum is Dr. Fatih Porikli (MERL, Cambridge).
Modified weak fusion model for depthless streaming of 3D videos

We propose a nonlinear fusion model to optimally reconstruct depth maps from monocular cues. The authors in  showed the effectiveness of estimating depth from individual cues, which saves bandwidth without compromising the quality. In this paper, we combine such depth map estimates as a Sensor Fusion problem. These depth map estimates are extracted from color, motion and texture structure of the scene. We address the combination of monocular cues as a nonlinear optimization problem with linear constraints. 2D and 3D objective quality metrics are used as reliability metrics. At first, we perform Particle Swarm Optimization based on PSNR to obtain an initial estimate for the linear weights. Then, we use these weights to perform Active-Sets based on 3VQM. We tested our approach on various video sequences with different monocular characteristics. Experimental results show that the quality of the rendered views based on combined depth maps is comparable to the quality of rendered views based on ground truth depth maps while the savings in bandwidth is up to 38:8%.
Air-Writing Recognition—Part I: Modeling and Recognition of Characters, Words, and Connecting Motions

Air-writing refers to writing of linguistic characters or words in a free space by hand or finger movements. Air-writing differs from conventional handwriting; the latter contains the pen-up-pen-down motion, while the former lacks such a delimited sequence of writing events. We address air-writing recognition problems in a pair of companion papers. In Part I, recognition of characters or words is accomplished based on six-degree-of-freedom hand motion data. We address air-writing on two levels: motion characters and motion words. Isolated air-writing characters can be recognized similar to motion gestures although with increased sophistication and variability. For motion word recognition in which letters are connected and superimposed in the same virtual box in space, we build statistical models for words by concatenating clustered ligature models and individual letter models. A hidden Markov model is used for air-writing modeling and recognition. We show that motion data along dimensions beyond a 2-D trajectory can be beneficially discriminative for air-writing recognition. We investigate the relative effectiveness of various feature dimensions of optical and inertial tracking signals and report the attainable recognition performance correspondingly. The proposed system achieves a word error rate of 0.8% for word-based recognition and 1.9% for letter-based recognition. We also subjectively and objectively evaluate the effectiveness of air-writing and compare it with text input using a virtual keyboard. The words per minute of air-writing and virtual keyboard are 5.43 and 8.42, respectively.
Tensor-based subspace learning for tracking salt-dome boundaries constrained by seismic attributes

We propose a method to delineate salt-dome structures by tracking manually labeled boundaries through seismic volumes. We first extract texture features from boundary regions using the tensor-based subspace learning method. Then, we utilize one seismic attribute, the gradient of texture (GoT), as a constraint on the tracking process. Using texture features and GoT maps, we can identify tracked points and optimally connect them to synthesize the boundaries. The proposed method is evaluated using real-world seismic data and experimental results show that it outperforms the state of the art in accuracy, robustness, and computational efficiency.
A comparative study of computational aesthetics

Objective metrics model image quality by quantifying image degradations or estimating perceived image quality. However, image quality metrics do not model what makes an image more appealing or beautiful. In order to quantify the aesthetics of an image, we need to take it one step further and model the perception of aesthetics. In this paper, we examine computational aesthetics models that use hand-crafted, generic and hybrid descriptors. We show that generic descriptors can perform as well as state of the art hand-crafted aesthetics models that use global features. However, neither generic nor hand-crafted features is sufficient to model aesthetics when we only use global features without considering spatial composition or distribution. We also follow a visual dictionary approach similar to state of the art methods and show that it performs poorly without the spatial pyramid step.
A curvelet-based distance measure for seismic images

We introduce a new curvelet-based distance measure for post-migration seismic data. The measure exploits the highly directional content of seismic images. It calculates the sum of the squared chord distances between the histograms of the curvelet coefficients of two images, over all orientations and scales. In a retrieval task consisting of 918 retrieval instances, the proposed method successfully retrieves all images. This wasn't achieved by other methods in the literature. Furthermore, in comparison to the state-of-the-art, the proposed measure required 86% less computation time.