nils-herrmann/clean_scirepeval huggingface dataset:
Guided co-training for multi-view spectral clustering

We address the problem of how to design a more effective co-training scheme to tackle the multi-view spectral clustering. The conventional co-training procedure treats information from all views equally and often converges to a compromised consensus view that does not fully utilize the multiview information. We instead propose to learn an augmented view and construct its corresponding affinity matrix from a spectral decomposition of an information-rich matrix formed by the eigenvectors of the Laplacian matrices from all views. As the augmented view is expected to be more favorable for carrying out spectral clustering, we design a new pairwise co-training procedure to guide the improvements of the given multiple views separately and iteratively. Our experimental results on three popular benchmark datasets support that the convergent augmented view by the guided co-training process is useful to multi-view spectral clustering, and can yield state-of-the-art performance.
Fast video super-resolution via approximate nearest neighbor search

Image super-resolution has gained much attention in these years, while video super-resolution remains almost unchanged. In this paper, we propose a fast super-resolution method for video. We exploit recent development of learning-based technique that achieves state-of-the-art in accuracy and efficiency for image super-resolution. We leverage the temporal coherency of video contents to approximate the nearest neighbor search in learning-based SR. Experimental results show that our method is able to produce visually similar or better results while being 20 times faster than baseline frame-by-frame fast image SR and being orders of magnitude faster than complex optimization-based video SR.
On the performance of 3D just noticeable difference models

The just noticeable difference (JND) notion reflects the maximum tolerable distortion. It has been extensively used for the optimization of 2D applications. For stereoscopic 3D (S3D) content, this notion is different since it relies on different mechanisms linked to our binocular vision. Unlike 2D, 3D-JND models appeared recently and the related literature is rather limited. These models can be used for the sake of compression and quality assessment improvement for S3D content. In this paper, we propose a deep and comparative study of the existing 3D-JND models. Additionally, in order to analyze their performance, the 3D-JND models have been integrated in recent metric dedicated to stereoscopic image quality assessment (SIQA). The results are reported on two widely used S3D image databases.
Dense-Hog-based 3D face tracking for infant pain monitoring

This paper presents a new algorithm for 3D face tracking intended for clinical infant pain monitoring under challenging conditions. The algorithm uses a cylinder head model and head pose recovery by alignment of dynamically extracted templates based on dense-HOG features. The algorithm is motivated from the application context and compared with a variant based on intensities. The paper reports experimental results on videos of moving infants in hospital who are relaxed or in pain. Results show good short-term tracking behavior for poses up to 50 degrees from upright-frontal, with significantly higher accuracy resulting from the use of dense-HOG features.
Accurate 3D face modeling and recognition from RGB-D stream in the presence of large pose changes

We propose a 3D face modeling and recognition system using an RGB-D stream in the presence of large pose changes. In the previous work, all facial data points are registered with a reference to improve the accuracy of 3D face model from a low-resolution depth sequence. This registration often fails when applied to non-frontal faces. It causes inaccurate 3D face models and poor performance of matching. We address this problem by pre-aligning each input face (`frontalization') before the registration, which avoids registration failures. For each frame, our method estimates the 3D face pose, assesses the quality of data, segments the facial region, frontalizes it, and performs an accurate registration with the previous 3D model. The 3D-3D recognition system using accurate 3D models from our method outperforms other face recognition systems and shows 100% rank 1 recognition accuracy on a dataset with 30 subjects.
Dehazing of color image using stochastic enhancement

Images captured in presence of fog, haze or snow usually suffer from poor contrast and visibility. In this paper we propose a novel dehazing method to increase visibility from a single view without using any prior knowledge about the outdoor scene. The proposed method estimates a visibility map of the scene from the input image and uses stochastic iterative algorithm to remove fog and haze. The method can be applied to color and grayscale images. Experimental results show that the proposed algorithm outperforms most of the state-of-the-art algorithms in terms of contrast, colorfulness and visibility.
Detection of the midsagittal plane in MR images using a sheetness measure from eigenanalysis of local 3D phase congruency responses

The midsagittal plane (MSP) separates the cerebrum into left and right hemispheres and its detection has a number of useful applications in brain image processing. We propose an automatic technique for the detection of the MSP in magnetic resonance (MR) images that uses a sheetness measure obtained from eigenanalysis of local matrix of second-order moments of 3D phase congruency responses to determine those voxels most likely to belong to the MSP. A weighted least-squares fitting algorithm is used in a coarse-to-fine iterative manner to find the best fitting plane (the MSP) through the selected voxels. Unlike most of the proposed approaches, which are mainly based on symmetric measures of the brain, our technique uses a direct measure to find the MSP. Our technique was applied to 202 MR images (40 clinical and 162 synthetic) and it has shown to be very effective for both symmetrical and asymmetrical brain images. Quantitative assessment, using the angle between unit normals of the detected and reference MSPs, yielded to a mean absolute angle value bellow 0.5°.
A block parallel majorize-minimize memory gradient algorithm

In the field of 3D image recovery, huge amounts of data need to be processed. Parallel optimization methods are then of main interest since they allow to overcome memory limitation issues, while benefiting from the intrinsic acceleration provided by recent multicore computing architectures. In this context, we propose a Block Parallel Majorize-Minimize Memory Gradient (BP3MG) algorithm for solving large scale optimization problems. This algorithm combines a block coordinate strategy with an efficient parallel update. The proposed method is applied to a 3D microscopy image restoration problem involving a depth-variant blur, where it is shown to lead to significant computational time savings with respect to a sequential approach.
Semantic highlight retrieval

Finding highlights relevant to a text query in unedited videos has become increasingly important due to their unprecedented growth. We refer this task as semantic highlight retrieval and propose a query-dependent video representation for retrieving a variety of highlights. Our method consist of two parts: (1) “viralets”, a mid-level representation bridging between visual and semantic spaces; (2) a novel Semantically-Modulation (SM) procedure to make viralets query-dependent (referred to as SM viralets). Given SM viralets, we train a single highlight ranker to predict the highlightness of clips with respect to a variety of queries, whereas existing approaches can be applied only in a few predefined domains. We collect a viral video dataset1 including users' comments, highlights, and/or original videos. Among a database with 1189 (13% highlights) clips, our highlight ranker achieves 41.2% recall at top-10 retrieved clips. It is significantly higher than a state-of-the-art domain-specific highlight ranker and its extension. Similarly, our method also outperforms all baseline methods on the video highlight dataset.
Mobile periocular matching with pre-post cataract surgery

Ocular recognition algorithms, including iris matching, have been used in several applications including large scale national ID projects such as India's Aadaar. Deployment of large-scale biometric systems is expected to rely on using multiple devices including mobile devices to ensure widespread adoption of biometric recognition systems. Ocular images captured using mobile devices may have challenges such as uncontrolled illumination, complex background, and geometric distortions. Further, among many enrollees of large scale biometrics program, some may have ocular diseases. One of the most common ocular disease in elderly is cataract. While it is established that iris recognition may be challenging due to ocular diseases, this paper investigates periocular recognition with pre and post cataract surgery images. In this research, we present a mobile periocular database of 145 subjects1. Baseline results also include a framework that achieves over 69% rank-10 accuracy and around 24% genuine accept rate at 1% false accept rate in inter-session experiments.
Multi-view semantic temporal video segmentation

In this work, we propose a multi-view temporal video segmentation approach that employs a Gaussian scoring process for determining the best segmentation positions. By exploiting the semantic action information that the dense trajectories video description offers, this method can detect intra-shot actions as well, unlike shot boundary detection approaches. We compare the temporal segmentation results of the proposed method to both single-view and multi-view methods, and also compare the action recognition results obtained on ground truth video segments to the ones obtained on the proposed multi-view segments, on the IMPART multi-view action data set.
SSSC-AM: A unified framework for video co-segmentation by structured sparse subspace clustering with appearance and motion features

Video co-segmentation typically refers to the task to jointly segment common objects existing in a given group of videos. In practice, high-dimensional data such as videos are often conceptually thought of being drawn from a union of subspaces corresponding to multiple categories. Therefore, segmenting data into respective subspaces, known as subspace clustering, has widespread applications in computer vision, including co-segmentation. State-of-the-art methods via subspace clustering seek to solve the problem in two steps: learning an affinity matrix, followed by applying spectral clustering to the affinity matrix. However, it is insufficient to obtain an optimal solution since it does not take into account the interdependence of the affinity matrix and the segmentation. In this paper, we present a new unified video co-segmentation framework inspired by Structured Sparse Subspace Clustering (S3C), which yields more consistent segmentation results. In order to improve the detectability of motion features with missing trajectories, we add an extra signature to motion trajectories. Moreover, we reformulate the S3C algorithm by adding the affine subspace constraint in order to make it more suitable to segment rigid motions lying in affine subspaces of dimension at most 3. Experiments on MOViCS dataset demonstrate the effectiveness of our approaches and robustness with heavy noise.
Dense prediction for text line segmentation in handwritten document images

We present a novel approach to segment text lines from handwritten document images. In contrast to existing approaches which mainly use hand-designed features or heuristic rules to estimate the location of text lines, we train a fully convolutional network (FCN) to predict text line structure in document images. By using the FCN, a line map which is a rough estimation of text line is obtained. From this line map, text strings that pass through characters in each text line are constructed. To deal with touching text lines, line adjacency graph (LAG) is used to separate the touching characters into different text strings. The testing result on ICDAR2013 Handwritten Segmentation Contest dataset shows high performance together with the robustness of our system with different types of languages and multi-skewed text lines.
Robust calibration of broadcast cameras based on ellipse and line contours

Professional TV studio footage often poses specific challenges to camera calibration due to lack of features and complex camera operation. As available algorithms often fail, we propose a novel approach based on robust tracking of ellipse and line features of a predefined logo. We further devise a predictive and iterative estimation algorithm, which incorporates confidence measures and filtering. Our results validate accuracy and reliability of our approach, demonstrated with challenging professional footage.
Image reconstruction and threshold design for Quanta Image Sensors

Quanta Image Sensor (QIS) has been envisioned as a candidate solution for next generation image sensors. We provide two new contributions to the signal processing aspects of QIS. First, we develop an image reconstruction algorithm to recover the underlying images from the QIS data, which is a massive array of binarized Poisson random variables. The new algorithm supersedes existing methods by enabling arbitrary threshold level. Second, we present a threshold design scheme to adaptively update the threshold level for optimal image reconstruction. We discuss the existence of a phase transition in determining the optimal threshold. Experimental results on tone-mapped high dynamic range images validates the effectiveness of the threshold scheme and the image reconstruction algorithm.
Scale-adaptive EigenEye for fast eye detection in wild web images

Detecting eyes in images is fundamental for many computer vision applications including face detection, face recognition, and human-computer interaction. Most existing methods are designed and tested on datasets acquired under controlled lab settings (e.g., fixed scale, known poses, clean background, etc.), leaving their performance to be further examined on real-world, uncontrolled images, such as on-line images. This paper presents an effort on developing a fast and accurate eye detector for on-line images for which the acquisition condition is unknown and varies from one image to another, resulting in unpredictable background and variable scales for the eyes/faces. The key idea is to develop a scale-adaptive EigenEye approach, which employs an approximate scale estimated from face detection to modulate the pre-trained EigenEye basis in searching for the best match in a test image. The effort also includes building a 2845-image dataset with accurately-annotated eye locations and size, which will be made public to the community for future comparative study. Evaluation using this dataset, with comparison with a few leading state-of-the-art approaches, demonstrates the advantages of the proposed method.
A new robust active shape model formulation for cardiac MRI segmentation

The 3D segmentation of the left ventricle (LV) in cardiac MRI is a challenging problem, due to the presence of other anatomical structures and artifacts (outliers) around the LV. In this paper, a new formulation of a Robust Active Shape Model (RASM) is presented that is able to deal with those outliers. Instead of using the traditional one-to-one mapping of edge points and model points to compute the shape model parameters, the proposed approach uses a one-to-many mapping strategy and groups these edge points into edge segments (strokes). Then, a probabilistic framework provides a robust estimation of the model parameters, in which the influence in the segmentation of the unreliable outliers is reduced. The proposed method was tested on a public dataset comprising 660 volumes. The results indicate that this methodology provides accurate segmentations that are competitive with other state-of-the art methods.
An optical flow-based no-reference video quality assessment algorithm

We present an optical flow-based no-reference video quality assessment (NR-VQA) algorithm for assessing the perceptual quality of natural videos. Our algorithm is based on the hypothesis that distortions affect flow statistics both locally and globally. To capture the effects of distortion on optical flow, we measure irregularities at the patch level and at the frame level. At the patch level, we measure intra- and inter-patch level irregularities in the flow magnitude's variance and mean. We also measure the correlation in the patch level flow randomness between successive frames. At the frame level, we measure the normalized mean flow magnitude difference between successive frames. We rely on the robust NIQE algorithm for no-reference spatial quality assessment of the frames. These temporal and spatial features are averaged over all the frames to arrive at a video level feature vector. The video level features and the corresponding DMOS scores are used to train a support vector machine for regression (SVR). This machine is used to estimate the quality score of a test video. The competence of the proposed method is clearly demonstrated on SD and HD video databases that include common distortion types such as compression artifacts, packet loss artifacts, additive noise, and blur.
A fast adaptive spatio-temporal 3D feature for video-based person re-identification

Video-based person re-identification has become a hot topic in the field of research on computer vision and intelligent surveillance, which is more robust to the variations in a person's appearance than single-shot based methods and involves space-time information. However, the most existing spatiotemporal features have been proposed for action recognition that they mainly focus on the exact spatial changes over time. Unlike action recognition, pedestrians captured in person re-identification problem show similar and cyclic walking activities. The essential spatio-temporal information for person re-identification is the statistical information over time. In this paper, we propose a novel spatio-temporal feature, namely Fast Adaptive Spatio-Temporal 3D feature (FAST3D), for video-based person re-identification. The feature is able to extract the statistical motion information based on densely computed multi-direction gradients and an adaptive fusion process. We evaluate our method on two challenging datasets and the experimental results show the effectiveness and efficiency of the proposed feature.
Adaptive cascaded regression

The two predominant families of deformable models for the task of face alignment are: (i) discriminative cascaded regression models, and (ii) generative models optimised with Gauss-Newton. Although these approaches have been found to work well in practise, they each suffer from convergence issues. Cascaded regression has no theoretical guarantee of convergence to a local minimum and thus may fail to recover the fine details of the object. Gauss-Newton optimisation is not robust to initialisations that are far from the optimal solution. In this paper, we propose the first, to the best of our knowledge, attempt to combine the best of these two worlds under a unified model and report state-of-the-art performance on the most recent facial benchmark challenge.
Low-power analog smart camera sensor for edge detection

This work presents an intelligent analog image sensor system for smart camera applications with the need of edge or marker detection. The system consists of a 3×3 read-out CMOS image sensor, an analog Sobel stage and additional circuitry like operational amplifiers and comparators to compute a 1 bit image with the edges present in the taken photo. This information can then be further processed digitally to detect specific shapes in order to control robot routines, for example. The architecture of the proposed system is highly desirable as dedicated analog hardware has significant advantages in terms of power and speed compared to digital implementations. The overall system is simulated with the help of a 3×3 CMOS image sensor IC as well as Cadence Virtuoso for analog circuit simulation and MATLAB to convert the sequential information back to an image, and compared to other state of the art CMOS image sensors with edge detection capability. The analog Sobel circuit runs with a clock of 10 MHz and consumes less than 0.79 mW average power for the computation of the example image, and the whole 200×200 pixel image sensor consumes only 5.5 mW at a frame rate of 75 fps.
Robust building roof segmentation using airborne point cloud data

Approximation of the geometric features is an essential step in point cloud segmentation and surface reconstruction. Often, the planar surfaces are estimated using principal component analysis (PCA), which is sensitive to noise and smooths the sharp features. Hence, the segmentation results into unreliable reconstructed surfaces. This article presents a point cloud segmentation method for building detection and roof plane extraction. It uses PCA for saliency feature estimation including surface curvature and point normal. However, the point normals around the anisotropic surfaces are approximated using a consistent isotropic sub-neighbourhood by Low-Rank Subspace with prior Knowledge (LRSCPK). The developed segmentation technique is tested using two real-world samples and two benchmark datasets. Per-object and per-area completeness and correctness results indicate the robustness of the approach and the quality of the reconstructed surfaces and extracted buildings.
Auto-context modeling using multiple Kernel learning

In complex visual recognition systems, feature fusion has become crucial to discriminate between a large number of classes. In particular, fusing high-level context information with image appearance models can be effective in object/scene recognition. To this end, we develop an auto-context modeling approach under the RKHS (Reproducing Kernel Hilbert Space) setting, wherein a series of supervised learners are used to approximate the context model. By posing the problem of fusing the context and appearance models using multiple kernel learning, we develop a computationally tractable solution to this challenging problem. Furthermore, we propose to use the marginal probabilities from a kernel SVM classifier to construct the auto-context kernel. In addition to providing better regularization to the learning problem, our approach leads to improved recognition performance in comparison to using only the image features.