nils-herrmann/clean_scirepeval huggingface dataset:
Annotating retrieval database with active learning

In this paper, we describe a retrieval system that uses hidden annotation to improve the performance. The contribution of this paper is a novel active learning framework that can improve the annotation efficiency. For each object in the database, we maintain a list of probabilities, each indicating the probability of this object having one of the attributes. This list of probabilities serves as the basis of our active learning algorithm, as well as semantic features to determine the similarity between objects in the database. We show active learning has better performance than random sampling in all our experiments.
Analyzing the Optimality of Predictive Transform Coding Using Graph-Based Models

In this letter, we provide a theoretical analysis of optimal predictive transform coding based on the Gaussian Markov random field (GMRF) model. It is shown that the eigen-analysis of the precision matrix of the GMRF model is optimal in decorrelating the signal. The resulting graph transform degenerates to the well-known 2-D discrete cosine transform (DCT) for a particular 2-D first order GMRF, although it is not a unique optimal solution. Furthermore, we present an optimal scheme to perform predictive transform coding based on conditional probabilities of a GMRF model. Such an analysis can be applied to both motion prediction and intra-frame predictive coding, and may lead to improvements in coding efficiency in the future.
Geometrically Constrained Room Modeling With Compact Microphone Arrays

The geometry of an acoustic environment can be an important information in many audio signal processing applications. To estimate such a geometry, previous work has relied on large microphone arrays, multiple test sources, moving sources or the assumption of a 2-D room. In this paper, we lift these requirements and present a novel method that uses a compact microphone array to estimate a 3-D room geometry, delivering effective estimates with low-cost hardware. Our approach first probes the environment with a known test signal emitted by a loudspeaker co-located with the array, from which the room impulse responses (RIRs) are estimated. It then uses an ℓ1-regularized least-squares minimization to fit synthetically generated reflections to the RIRs, producing a sparse set of reflections. By enforcing structural constraints derived from the image model, these are classified into first-, second-, and third-order reflections, thereby deriving the room geometry. Using this method, we detect walls using off-the-shelf teleconferencing hardware with a typical range resolution of about 1 cm. We present results using simulations and data from real environments.
Light Field Sampling

Light field is one of the most representative image-based rendering techniques that generate novel virtual views from images instead of 3D models. The light field capture and rendering process can be considered as a procedure of sampling the light rays in the space and interpolating those in novel views. As a result, light field can be studied as a high-dimensional signal sampling problem, which has attracted a lot of research interest and become a convergence point between computer graphics and signal processing, and even computer vision. This lecture focuses on answering two questions regarding light field sampling, namely how many images are needed for a light field, and if such number is limited, where we should capture them. The book can be divided into three parts. First, we give a complete analysis on uniform sampling of IBR data. By introducing the surface plenoptic function, we are able to analyze the Fourier spectrum of non-Lambertian and occluded scenes. Given the spectrum, we also app...
Robust part-based face matching with multiple templates

Part-based matching can handle significant variations in facial expression and partial occlusion. However, under large pose variations, it is sometimes difficult to find corresponding local parts in a pair of images. In this paper, we propose a novel part-based face matching scheme that relies on multiple face templates. These templates serve as bridges to help finding corresponding parts between test images with different face poses, thus making the overall algorithm more robust compared with traditional schemes. Experimental results on Multi-PIE public benchmark data set show that our method is very effective in improving face matching performance with significant pose variations.
Multi-View Imaging: Capturing and Rendering Interactive Environments

This paper presents a self-reconfigurable camera array system that captures and renders 3D virtual scenes interactively. It is composed of an array of 48 cameras mounted on mobile platforms. We propose an efficient algorithm that is capable of rendering high-quality novel views from the captured images. The algorithm reconstructs a view-dependent multiresolution 2D mesh model of the scene geometry on the fly and uses it for rendering. The algorithm combines region of interest (ROI) identification, JPEG image decompression, lens distortion correction, scene geometry reconstruction and novel view synthesis seamlessly on a single Intel Xeon 2.4 GHz processor, which is capable of generating novel views at 4 10 frames per second (fps). In addition, we present a view-dependent adaptive capturing scheme that moves the cameras in order to show even better rendering results. Such camera reconfiguration naturally leads to a nonuniform arrangement of the cameras on the camera plane, which is both view-dependent and scene-dependent.
Facial expression tracking from head-mounted, partially observing cameras

Head-mounted displays (HMDs) have gained more and more interest recently. They can enable people to communicate with each other from anywhere, at anytime. However, since most HMDs today are only equipped with cameras pointing outwards, the remote party would not be able to see the user wearing the HMD. In this paper, we present a system for facial expression tracking based on head-mounted, inward looking cameras, such that the user can be represented with animated avatars at the remote party. The main challenge is that the cameras can only observe partial faces since they are very close to the face. We experiment with multiple machine learning algorithms to estimate facial expression parameters based on training data collected with the assistance of a Kinect depth sensor. Our results show that we can reliably track people's facial expression even from very limited view angles of the cameras.
See-through Image Enhancement through Sensor Fusion

Many hardware designs have been developed to allow a camera to be placed optically directly behind the screen. The purpose of such setups is to enable two-way video teleconferencing that maintains eye-contact. However, the image from the see-through camera usually exhibits a number of imaging artifacts such as low signal to noise ratio, incorrect color balance, and lost of details. We develop a novel image enhancement framework that utilizes an auxiliary color+depth camera that is mounted on the side of the screen. By fusing the information from both cameras, we are able to significantly improve the quality of the see-through image. Experimental results have demonstrated that our fusion method compares favorably against traditional image enhancement/warping methods that uses only a single image.
Semantic propagation from relevance feedbacks

Relevance feedback has been a very useful tool to enhance the performance of content-based information retrieval (CBIR) systems. To fully make use of the precious user feedback provided to a system, we propose an approach named semantic propagation, which reveals the deep semantic relationships among objects in the database, given a set of relevance feedbacks between object pairs. In particular, we present two semantic propagation algorithms that are applicable to CIBR systems with a feature vector space model and a general metric space model, respectively. Experiments on a 3D model retrieval system and a logo image retrieval system are performed to show the effectiveness of the proposed methods.
Personal 3D audio system with loudspeakers

Traditional 3D audio systems often have a limited sweet spot for the user to perceive 3D effects successfully. In this paper, we present a personal 3D audio system with loudspeakers that has unlimited sweet spots. The idea is to have a camera track the user's head movement, and recompute the crosstalk canceller filters accordingly. As far as the authors are aware of, our system is the first non-intrusive 3D audio system that adapts to both the head position and orientation with six degrees of freedom. The effectiveness of the proposed system is demonstrated with subjective listening tests comparing our system against traditional non-adaptive systems.
Robust Visual Tracking via Pixel Classification and Integration

We propose a novel framework for tracking non-rigid objects via pixel classification and integration (PCI). Given a new input frame, the tracker first performs object classification on each pixel and then finds the region that has the highest integral of scores. There are several key advantages of the proposed approach: it is computationally very efficient; it finds a global, instead of local (e.g., mean-shift), optimal solution within a search range; and it is inherently robust to different object scales with minimum extra computation. Within this framework, a mixture of long-term and short-term appearance model is further introduced to perform PCI. As a result, the tracker is able to adapt to both slow and rapid appearance changes without drifting. Challenging video sequences are presented to illustrate how the proposed tracker handles large motion, dramatic shape changes, scale variations, illumination variations and partial occlusions
Surface plenoptic function: a tool for the sampling analysis of image-based rendering

In this paper, we introduce the surface plenoptic function (SPF) as a tool for the sampling analysis of image-based rendering (IBR). SPF is a plenoptic function defined on the object surface. It can be mapped onto the normal IBR representation via a coordinate transform. By assuming some properties of the SPF, we can provide insightful analysis on the sampling of the IBR representation, for both uniform and non-uniform methods.
Video face beautification

This paper presents a novel system framework of face beautification. Unlike prior works that deal with single images, the proposed beautification framework is designed for an input video and it is able to improve both the appearance and the shape of a face. Our system adopts a state-of-the-art algorithm to synthesize and track 3D face models using blendshapes. The personalized 3D model can be edited to satisfy personal preference. This interactive process is needed only once per subject. Based on the tracking result and the modified face model, we present an algorithm to beautify the face video efficiently and consistently. Furthermore we develop a variant of content preserving warping to reduce warping distortions along the face boundary. Finally we adopt real time bilateral filtering to remove wrinkles, freckles, and unwanted blemishes. This framework is evaluated on a set of videos. The experiments demonstrate that our framework can generate consistent and pleasant results over video frames while the original expressions and features are persevered naturally.
Hybrid speaker tracking in an automated lecture room

We present a hybrid speaker tracking scheme based on a single pan/tilt/zoom (PTZ) camera in an automated lecture capturing system. Given that the camera's video resolution is higher than the required output resolution, we frame the output video as a sub-region of the camera's input video. This allows us to track the speaker both digitally and mechanically. Digital tracking has the advantage of being smooth, and mechanical tracking can cover a wide area. The hybrid tracking achieves the benefits of both worlds. In addition to hybrid tracking, we present an intelligent pan/zoom selection scheme to improve the aestheticity of the lecture scene.
Light Weight Background Blurring for Video Conferencing Applications

blurring is an effective way to both preserving privacy and keeping communication effective during video conferencing. This paper proposes a light weight real-time algorithm to perform background blurring using a fast background modeling algorithm combined with a face detector/tracker. A soft decision is made at each pixel whether it belongs to the foreground or the background based on multiple vision features. The classification results are mapped to a per-pixel blurring radius image to blur the background. The algorithm produces satisfactory results under a wide range of conditions, and occupies less than 30% of the CPU cycles on a 3 GHz Pentium 4 machine without further optimization.
Automatic speech emotion recognition using recurrent neural networks with local attention

Automatic emotion recognition from speech is a challenging task which relies heavily on the effectiveness of the speech features used for classification. In this work, we study the use of deep learning to automatically discover emotionally relevant features from speech. It is shown that using a deep recurrent neural network, we can learn both the short-time frame-level acoustic features that are emotionally relevant, as well as an appropriate temporal aggregation of those features into a compact utterance-level representation. Moreover, we propose a novel strategy for feature pooling over time which uses local attention in order to focus on specific regions of a speech signal that are more emotionally salient. The proposed solution is evaluated on the IEMOCAP corpus, and is shown to provide more accurate predictions compared to existing emotion recognition algorithms.
Security analysis for key generation systems using face images

In this paper, we analyze the security problem of user information associated key generation (UIAKG) systems. We consider three kinds of attacks from the hacker: the exhaustive search attack, the authentic key statistics attack and the device key statistics attack. Under each attack, we give the estimate of the number of guesses the hacker has to make in order to access the system. Such analysis provides useful guidelines in designing a UIAKG system. The analysis also suggests that a user-dependent UIAKG is more secure than a user-independent one. Two UIAKG systems using face images as inputs are designed and compared to support the above theoretical analysis.
Boosting-Based Face Detection and Adaptation

Face detection, because of its vast array of applications, is one of the most active research areas in computer vision. In this book, we review various approaches to face detection developed in the past decade, with more emphasis on boosting-based learning algorithms. We then present a series of algorithms that are empowered by the statistical view of boosting and the concept of multiple instance learning. We start by describing a boosting learning framework that is capable to handle billions of training examples. It differs from traditional bootstrapping schemes in that no intermediate thresholds need to be set during training, yet the total number of negative examples used for feature selection remains constant and focused (on the poor performing ones). A multiple instance pruning scheme is then adopted to set the intermediate thresholds after boosting learning. This algorithm generates detectors that are both fast and accurate. Table of Contents: A Brief Survey of the Face Detection Literature / Cascade-based Real-Time Face Detection / Multiple Instance Learning for Face Detection / Detector Adaptation / Other Applications / Conclusions and Future Work
Calibration between depth and color sensors for commodity depth cameras

Commodity depth cameras have created many interesting new applications in the research community recently. These applications often require the calibration information between the color and the depth cameras. Traditional checkerboard based calibration schemes fail to work well for the depth camera, since its corner features cannot be reliably detected in the depth image. In this paper, we present a maximum likelihood solution for the joint depth and color calibration based on two principles. First, in the depth image, points on the checker-board shall be co-planar, and the plane is known from color camera calibration. Second, additional point correspondences between the depth and color images may be manually specified or automatically established to help improve calibration accuracy. Uncertainty in depth values has been taken into account systematically. The proposed algorithm is reliable and accurate, as demonstrated by extensive experimental results on simulated and real-world examples.
A fast & accurate non-iterative algorithm for regularized non-Cartesian MRI

We introduce a novel algorithm for regularized reconstruction of non-Cartesian MRI data. The proposed non-iterative scheme closely approximates the Tikhonov regularized least squares method, but provides a significant speed up over standard implementation based on iterative conjugate gradient algorithm. This computational complexity of the proposed scheme is comparable to that of gridding. However, the proposed scheme is significantly more robust to undersampling and measurement noise. Numerical simulations clearly demonstrate the advantages of the proposed algorithm over traditional schemes. The proposed algorithm may be very useful in dynamic and functional MRI applications, where the fast reconstruction of several undersampled images is required.
BSLIM: Spectral Localization by Imaging With Explicit $B_{0}$ Field Inhomogeneity Compensation

Magnetic resonance spectroscopy imaging (MRSI) is an attractive tool for medical imaging. However, its practical use is often limited by the intrinsic low spatial resolution and long acquisition time. Spectral localization by imaging (SLIM) has been proposed as a non-Fourier reconstruction algorithm that incorporates spatial a priori information about spectroscopically uniform compartments. Unfortunately, the influence of the magnetic field inhomogeneity - in particular, the susceptibility effects at tissues' boundaries - undermines the validity of the compartmental model. Therefore, we propose BSLIM as an extension of SLIM with field inhomogeneity compensation. A B0-field inhomogeneity map, which can be acquired rapidly and at high resolution, is used by the new algorithm as additional a priori information. We show that the proposed method is distinct from the generalized SLIM (GSLIM) framework. Experimental results of a two-compartment phantom demonstrate the feasibility of the method and the importance of inhomogeneity compensation.
Accelerated MRI using iterative non-local shrinkage

We introduce a fast iterative non-local shrinkage algorithm to recover MRI data from undersampled Fourier measurements. This approach is enabled by the reformulation of current non-local schemes as an alternating algorithm to minimize a global criterion. The proposed algorithm alternates between a non-local shrinkage step and a quadratic subproblem. The resulting algorithm is observed to be considerably faster than current alternating non-local algorithms. We use efficient continuation strategies to minimize local minima issues. The comparisons of the proposed scheme with state-of-the-art regularization schemes show a considerable reduction in alias artifacts and preservation of edges.
Dynamic MRI Using SmooThness Regularization on Manifolds (SToRM)

We introduce a novel algorithm to recover real time dynamic MR images from highly under-sampled k- t space measurements. The proposed scheme models the images in the dynamic dataset as points on a smooth, low dimensional manifold in high dimensional space. We propose to exploit the non-linear and non-local redundancies in the dataset by posing its recovery as a manifold smoothness regularized optimization problem. A navigator acquisition scheme is used to determine the structure of the manifold, or equivalently the associated graph Laplacian matrix. The estimated Laplacian matrix is used to recover the dataset from undersampled measurements. The utility of the proposed scheme is demonstrated by comparisons with state of the art methods in multi-slice real-time cardiac and speech imaging applications.
An Exact Method for Computing the Area Moments of Wavelet and Spline Curves

We present a method for the exact computation of the moments of a region bounded by a curve represented by a scaling function or wavelet basis. Using Green's theorem, we show that the computation of the area moments is equivalent to applying a suitable multidimensional filter on the coefficients of the curve and thereafter computing a scalar product. The multidimensional filter coefficients are precomputed exactly as the solution of a two-scale relation. To demonstrate the performance improvement of the new method, we compare it with existing methods such as pixel-based approaches and approximation of the region by a polygon. We also propose an alternate scheme when the scaling function is sinc(x).
Efficient energies and algorithms for parametric snakes

Parametric active contour models are one of the preferred approaches for image segmentation because of their computational efficiency and simplicity. However, they have a few drawbacks which limit their performance. In this paper, we identify some of these problems and propose efficient solutions to get around them. The widely-used gradient magnitude-based energy is parameter dependent; its use will negatively affect the parametrization of the curve and, consequently, its stiffness. Hence, we introduce a new edge-based energy that is independent of the parameterization. It is also more robust since it takes into account the gradient direction as well. We express this energy term as a surface integral, thus unifying it naturally with the region-based schemes. The unified framework enables the user to tune the image energy to the application at hand. We show that parametric snakes can guarantee low curvature curves, but only if they are described in the curvilinear abscissa. Since normal curve evolution do not ensure constant arc-length, we propose a new internal energy term that will force this configuration. The curve evolution can sometimes give rise to closed loops in the contour, which will adversely interfere with the optimization algorithm. We propose a curve evolution scheme that prevents this condition.
A Fast Algorithm for Convolutional Structured Low-Rank Matrix Recovery

Fourier-domain structured low-rank matrix priors are emerging as powerful alternatives to traditional image recovery methods such as total variation and wavelet regularization. These priors specify that a convolutional structured matrix, i.e., Toeplitz, Hankel, or their multilevel generalizations, built from Fourier data of the image should be low-rank. The main challenge in applying these schemes to large-scale problems is the computational complexity and memory demand resulting from lifting the image data to a large-scale matrix. We introduce a fast and memory-efficient approach called the generic iterative reweighted annihilation filter algorithm that exploits the convolutional structure of the lifted matrix to work in the original unlifted domain, thus considerably reducing the complexity. Our experiments on the recovery of images from undersampled Fourier measurements show that the resulting algorithm is considerably faster than previously proposed algorithms and can accommodate much larger problem sizes than previously studied.