nils-herrmann/clean_scirepeval huggingface dataset:
Adaptive smoothness constraints for efficient stereo matching using texture and edge information

An efficient stereo matching algorithm, which applies adaptive smoothness constraints using texture and edge information, is proposed in this work. First, we determine non-textured regions, on which an input image yields flat pixel values. In the non-textured regions, we penalize depth discontinuity and complement the primary CNN-based matching cost with a color-based cost. Second, by combining two edge maps from the input image and a pre-estimated disparity map, we extract denoised edges that correspond to depth discontinuity with high probabilities. Thus, near the denoised edges, we penalize small differences of neighboring disparities. Based on these adaptive smoothness constraints, the proposed algorithm outperforms the conventional methods significantly and achieves the state-of-the-art performance on the Middlebury stereo benchmark.
Efficient depth video coding based on view synthesis distortion estimation

An efficient coding algorithm for depth map images and videos, based on view synthesis distortion estimation, is proposed in this work. We first analyze how a depth error is related to a disparity error and how the disparity vector error affects the energy spectral density of a synthesized color video in the frequency domain. Based on the analysis, we propose an estimation technique to predict the view synthesis distortion without requiring the actual synthesis of intermediate view frames. To encode the depth information efficiently, we employ a Lagrangian cost function to minimize the view synthesis distortion subject to the constraint on a transmission bit rate. In addition, we develop a quantization scheme for residual depth data, which adaptively assigns bits according to block complexities. Simulation results demonstrate that the proposed depth video coding algorithm provides significantly better R-D performance than conventional algorithms.
Unsupervised Primary Object Discovery in Videos Based on Evolutionary Primary Object Modeling With Reliable Object Proposals.

A novel primary object discovery (POD) algorithm, which uses reliable object proposals while exploiting the recurrence property of a primary object in a video sequence, is proposed in this paper. First, we generate both color-based and motion-based object proposals in each frame, and extract the feature of each proposal using the random walk with restart simulation. Next, we estimate the foreground confidence for each proposal to remove unreliable proposals. By superposing the features of the remaining reliable proposals, we construct the primary object models. To this end, we develop the evolutionary primary object modeling technique, which exploits the recurrence property of the primary object. Then, using the primary object models, we choose the main proposal in each frame and find the location of the primary object by merging the main proposal with candidate proposals selectively. Finally, we refine the discovered bounding boxes by exploiting temporal correlations of the recurring primary object. Extensive experimental results demonstrate that the proposed POD algorithm significantly outperforms conventional algorithms.
SEQM: Edge quality assessment based on structural pixel matching

A novel quality metric for binary edge maps, called the structural edge quality metric (SEQM), is proposed in this work. First, we define the matching cost between an edge pixel in a detected edge map and its candidate matching pixel in the ground-truth edge map. The matching cost includes a structural term, as well as a positional term, to measure the discrepancy between the local structures around the two pixels. Then, we determine the optimal matching pairs of pixels using the graph-cut optimization, in which a smoothness term is employed to take into account global edge structures in the matching. Finally, we sum up the matching costs of all edge pixels to determine the quality index of the detected edge map. Simulation results demonstrate that the proposed SEQM provides more faithful and reliable quality indices than conventional metrics.
Reflection Removal Under Fast Forward Camera Motion

The image quality of an in-vehicle black box camera is often degraded by the reflections of internal objects, dirt, and dust on the windshield. In this paper, we propose a novel algorithm that simultaneously removes the reflections and small dirt artifacts from in-vehicle black box videos under fast forward camera motion. The algorithm exploits the spatiotemporal coherence of the reflection and dirt, which remain stationary relative to the fast-moving background. Unlike previous algorithms, the algorithm first separates stationary reflection and then restores the background scene. To this end, we propose an average image prior, thereby imposing spatiotemporal coherence. The separation model is a two-layer model composed of stationary and background layers, where different gradient sparsity distributions are utilized in a region-based manner. Motion compensation in postprocessing is proposed to alleviate layer jitter due to vehicle vibrations. In evaluation experiments, the proposed algorithm successfully extracts the stationary layer from several real and synthetic black box videos.
Colorization-based inter-plane prediction algorithm for high-fidelity color image compression

We propose a colorization-based prediction algorithm, which reduces the correlation between luminance and chrominance planes, to achieve compact compression of color images. We first develop a priority-based colorization scheme, which adds chrominance values to gray pixels using a set of colored pixels, assuming that neighboring pixels with similar luminance values tend to have similar chrominance values as well. The colorization scheme assigns a high priority to a pixel that has a large number of neighbors with similar luminance values. Then, we employ the colorization scheme to predict the chrominance planes from the luminance plane. Since the priority-based colorization scheme predicts color data accurately even in complicated color regions, the prediction residuals can be efficiently compressed. Simulation results on rich color images show that the proposed algorithm yields better compression performance than H.264/AVC.
Visual Tracking Using Pertinent Patch Selection and Masking

A novel visual tracking algorithm using patch-based appearance models is proposed in this paper. We first divide the bounding box of a target object into multiple patches and then select only pertinent patches, which occur repeatedly near the center of the bounding box, to construct the foreground appearance model. We also divide the input image into non-overlapping blocks, construct a background model at each block location, and integrate these background models for tracking. Using the appearance models, we obtain an accurate foreground probability map. Finally, we estimate the optimal object position by maximizing the likelihood, which is obtained by convolving the foreground probability map with the pertinence mask. Experimental results demonstrate that the proposed algorithm outperforms state-of-the-art tracking algorithms significantly in terms of center position errors and success rates.
Comparison of objective functions in CNN-based prostate magnetic resonance image segmentation

We investigate the impacts of objective functions on the performance of deep-learning-based prostate magnetic resonance image segmentation. To this end, we first develop a baseline convolutional neural network (BCNN) for the prostate image segmentation, which consists of encoding, bridge, decoding, and classification modules. In the BCNN, we use 3D convolutional layers to consider volumetric information. Also, we adopt the residual feature forwarding and intermediate feature propagation techniques to make the BCNN reliably trainable for various objective functions. We compare six objective functions: Hamming distance, Euclidean distance, Jaccard index, dice coefficient, cosine similarity, and cross entropy. Experimental results on the PROMISE12 dataset demonstrate that the cosine similarity provides the best segmentation performance, whereas the cross entropy performs the worst.
Exemplar-based frame rate up-conversion with congruent segmentation

A novel motion-compensated frame interpolation algorithm to increase video frame rates is proposed in this work, which employs texture optimization techniques to refine inaccurate motion vector fields. We enforce the congruent segment constraint in the motion refinement so that matching objects have similar local image structures. More specifically, we use the congruence energy as well as the appearance energy in the motion optimization to estimate high quality motion vectors. Simulation results show that, by preserving complex object shapes and texture, the proposed algorithm provides more faithful intermediate frames than conventional algorithms.
GEQM: A quality metric for gray-level edge maps based on structural matching

An accurate quality metric, called GEQM, for gray-level edge maps based on the structural matching of edge pixels is proposed in this work. We design the positional matching cost, which reflects the distance between two edge pixels, and the structural matching cost, which measures the structural shapes of edges as well as the differences of edge strength levels. Based on the cost functions, we perform the graph-cut optimization to obtain the optimal pixel-based matching between source and target edge maps bidirectionally. Finally, we compute the GEQM score by summing up the optimal matching costs of all edge pixels. Experimental results show that the proposed GEQM performs the edge map quality assessment more accurately and more reliably than conventional metrics. Especially, GEQM is suitable for assessing the qualities of synthesized intermediate views in multi-view image processing.
Robust contrast enhancement of noisy low-light images: Denoising-enhancement-completion

A robust contrast enhancement algorithm for noisy low-light images, called the denoising-enhancement-completion (DEC), is proposed in this work. We observe that noise components in low-light images degrade the performance of the contrast enhancement. Therefore, we first reduce noise components in an input image. Then, we compute the reliability weight for each pixel, by measuring the difference between the input image and the denoised image, and categorize each pixel into one of two classes: noise-free or noisy. We perform the selective histogram equalization to enhance the contrast of the noise-free pixels only. Finally, we restore missing values of the noisy pixels using the enhanced noise-free pixel values, by employing a low-rank matrix completion scheme. Experimental results show that the proposed DEC algorithm removes noise and enhances the contrast of low-light images more effectively than conventional algorithms.
Frame-level matching of near duplicate videos based on ternary frame descriptor and iterative refinement

A frame-level video matching algorithm, which achieves dense frame matching between near-duplicate videos, is proposed in this work. First, we propose a ternary frame descriptor for the near-duplicate video matching. The ternary descriptor partitions a frame into patches and uses ternary digits to represent relations between pairs of patches. Second, we formulate the frame-level matching problem as the minimization of a cost function, which consists of matching costs and adaptive unmatching costs. We develop an iterative refinement scheme that converges to a local minimum of the cost function. The iterative scheme performs competitively with the global optimization techniques while demands a significantly lower computational complexity. Experimental results show that the proposed algorithm achieves effective frame description and efficient frame matching of near duplicate videos.
Semi-regular representation and progressive compression of 3-D dynamic mesh sequences

We propose an algorithm that represents three-dimensional dynamic objects with a semi-regular mesh sequence and compresses the sequence using the spatiotemporal wavelet transform. Given an irregular mesh sequence, we construct a semi-regular mesh structure for the first frame and then map it to subsequent frames based on the hierarchical motion estimation. The regular structure of the resulting mesh sequence facilitates the application of advanced coding schemes and other signal processing techniques. To encode the mesh sequence compactly, we develop an embedded coding scheme, which supports signal-to-noise ratio and temporal scalability modes. Simulation results demonstrate that the proposed algorithm provides significantly better compression performance than the static mesh coder, which encodes each frame independently
Automatic Video Genre Classification Using Multiple SVM Votes

A video genre classification algorithm based on the voting from multiple SVMs is proposed in this work. While conventional genre classifiers use generic baseline features, we employ more specialized features to describe five video genres: animation, commercial, entertainment, drama, and sports. We also present a robust classification algorithm using multiple SVMs, which consider all possible binary grouping of the five genres. Given a query video, each SVM casts a probabilistic vote for each genre. Then, the optimal genre with the maximum votes is selected. Experimental results show that the proposed algorithm provides more accurate classification performance than conventional algorithms.
Probabilistic depth-guided multi-view image denoising

A novel probabilistic depth-guided multi-view denoising (PDMD) algorithm is proposed in this work. We formulate the multi-view image denoising problem by considering the uncertainties in depth estimates in noisy environments. Specifically, we employ the geometric distributions of nonlocal neighbors, as well as the block similarities, to approximate the probabilities of depth estimates. We then use those probabilities to average all nonlocal neighbors and perform the minimum mean square error (MMSE) denoising. Simulation results show that the proposed PDMD algorithm provides better denoising performance than conventional algorithms.
Content-aware image and video resizing based on frequency domain analysis

An adaptive image and video resizing algorithm based on the frequency domain analysis is proposed in this work. Given an image, we first construct an importance map by combining the gradient and the saliency information. We partition the image into several strips so that each strip contains pixels of similar importance levels. We model the distortion, which is caused by scaling a strip, in the frequency domain. Then, we scale each strip adaptively to minimize the overall distortion of the whole image. Moreover, we extend the proposed algorithm for video resizing. We add the motion term to construct the importance map, and suppress excessive parameter variations to achieve jitter-free video resizing. Simulation results show that the proposed algorithm provides higher quality resizing results than conventional algorithms, although it requires lower computational complexity.
Gradient Domain Tone Mapping of High Dynamic Range Videos

A gradient domain tone mapping algorithm is proposed to display high dynamic range (HDR) video sequences in low dynamic range (LDR) devices in this work. The proposed algorithm obtains a pixelwise motion vector field and incorporates the motion information into the Poisson equation. Then, by attenuating large spatial gradients, the proposed algorithm can yield a high-quality tone-mapped result without flickering artifacts. Simulation results show that the proposed algorithm provides a better performance than the frame-based method, which processes each frame independently.
Compression of 2-D wide multi-view video sequences using view interpolation

In this work, we propose an efficient coding algorithm for multi-view video sequences, captured with a 2-D camera array. We develop an efficient prediction structure, which can exploit inter-view redundancies as well as temporal redundancies in 2-D multi-view video sequences. Moreover, we propose a view interpolation technique based on the bilateral criterion and the skip mode to improve the coding performance. Simulation results show that the proposed algorithm provides better PSNR performance than the conventional multi-view video coding algorithm.
Stereo video deraining and desnowing based on spatiotemporal frame warping

A novel rain (or snow) streak removal algorithm for stereo video sequences is proposed in this work. We observe that rain streaks appear at different locations in spatiotemporally adjacent frames. Thus, to derain a left-view frame, we synthesize it by warping the spatially adjacent right-view frame and the temporally previous and next frames, respectively. We subtract each warped frame from the original frame, and apply the median filter to the three difference images to obtain a reliable rain mask. Then, we remove rain streaks by replacing each rainy pixel value with a weighted average of non-locally neighboring pixel values. Experimental results demonstrate that the proposed algorithm removes rain streaks reliably and recovers original scene contents faithfully.
Video saliency detection based on random walk with restart

A graph-based video saliency detection algorithm is proposed in this work. We model eye movements on an image plane as random walks on a graph. To detect the saliency of the first frame in a video sequence, we construct a fully connected graph, in which each node represents an image block. We assign an edge weight to be proportional to the dissimilarity between the incident nodes and inversely proportional to their geometrical distance. We extract the saliency level of each node from the stationary distribution of the random walker on the graph. Next, to detect the saliency of each subsequent frame, we add the criterion that an edge, connecting a slow motion node to a fast motion node, should have a large weight. We then compute the stationary distribution of the random walk with restart (RWR) simulation, in which the saliency of the previous frame is used as the restarting distribution. Experimental results show that the proposed algorithm provides more reliable and accurate saliency detection performance than conventional algorithms.
Efficient macroblock ordering for chrominance planes in rich color image compression

A novel approach to adapt the macroblock (MB) coding order of chrominance planes is proposed to improve the compression performance of the H.264/AVC intra coder. The proposed algorithm first encodes the luminance plane of an image and then uses the luminance data to determine the MB coding order of the chrominance planes adaptively. The proposed algorithm consists of three steps. First, we estimate the similarities between neighboring chrominance MBs using the luminance gradients. Second, we assign a high priority to a chrominance MB, if it is similar to many neighboring MBs. Finally, we encode the chrominance MBs in the decreasing order of the priorities. Moreover, we develop efficient submodes, representing prediction directions, to exploit the correlations of the MBs along the adaptive scan order. Experimental results show that the proposed algorithm yields significantly better compression performance than H.264/AVC on both YUV 4:4:4 and RGB 4:4:4 images.
Near-duplicate video clustering using multiple complementary video signatures

A near-duplicate video clustering algorithm based on multiple complementary video signatures is proposed in this work. We use three kinds of frame descriptors: RGB histogram, color name histogram, and ternary pattern. Then, we convert each kind of frame descriptors for a video into a video signature based on the bag-of-visual-words scheme. Consequently, we have three signatures to represent the video. These signatures are complementary to one another, since they are robust to different near-duplication types. Also, we develop a clustering technique to refine pairwise matching results and categorize near-duplicate videos. Experimental results on an extensive video dataset show that the proposed algorithm detects near-duplicate videos more effectively than conventional algorithms.
Compact encoding of 3-D voxel surfaces based on pattern code representation

In this paper, we propose a lossless compression algorithm for three-dimensional (3-D) binary voxel surfaces, based on the pattern code representation (PCR). In PCR, a voxel surface is represented by a series of pattern codes. The pattern of a voxel v is defined as the 3 x 3 x 3 array of voxels, centered on v. Therefore, the pattern code for informs of the local shape of the voxel surface around . The proposed algorithm can achieve the coding gain, since the patterns of adjacent voxels are highly correlated to each other. The performance of the proposed algorithm is evaluated using various voxel surfaces, which are scan-converted from triangular mesh models. It is shown that the proposed algorithm requires only 0.5 approximately 1 bits per black voxel (bpbv) to store or transmit the voxel surfaces.
Temporal Feature Modulation for Video Watermarking

We propose two temporal feature modulation algorithms that extract a feature from each video frame and modulate the features for a series of frames to embed a watermark codeword. In the first algorithm, the existence of a frame is used as the frame feature, and a watermark codeword is embedded into the original video by skipping selected frames. In the second algorithm, the centers of gravity of blocks in a frame are used as the frame feature. By modifying the centers of gravity, we embed 1-bit information into the frame. Simulation results demonstrate that the proposed algorithms are robust against compression and temporal attacks.
Efficient side information generation using assistant pixels for distributed video coding

A side information generation method using assistant pixels is proposed for distributed video coding (DVC) in this work. To generate higher quality side information frames and improve the rate-distortion (R-D) performance of a DVC system, we propose transmitting assistant pixels, which help to improve the motion estimation accuracy of the decoder at the cost of an overhead bit-rate. Also, we develop a mode decision scheme, which determines the interpolation mode for each block by combining forward and backward motion vectors. Simulation results demonstrate that the proposed algorithm provides better R-D performance than the state-of-the-art DVC algorithm.
Progressive Transmission Of Pointtexture 3-D Images

A progressive compression and transmission algorithm for PointTexture 3-D images is proposed in this work. The proposed algorithm represents a PointTexture image hierarchically using an octree. The geometry information in octree nodes is encoded by the predictive partial matching (PPM) method, while the color information is encoded using the discrete cosine transform (DCT). The encoder achieves the progressive transmission of the 3-D image by transmitting the octree nodes in a top-down manner. We develop a transmission scheme, based on the rate-distortion (R-D) optimization in order to maximize the image quality subject to a given bit budget. Extensive simulation results demonstrate that the proposed algorithm is an efficient method for progressive transmission of 3-D data