nils-herrmann/clean_scirepeval huggingface dataset:
Tensor-Based Human Body Modeling

In this paper, we present a novel approach to model 3D human body with variations on both human shape and pose, by exploring a tensor decomposition technique. 3D human body modeling is important for 3D reconstruction and animation of realistic human body, which can be widely used in Tele-presence and video game applications. It is challenging due to a wide range of shape variations over different people and poses. The existing SCAPE model is popular in computer vision for modeling 3D human body. However, it considers shape and pose deformations separately, which is not accurate since pose deformation is person-dependent. Our tensor-based model addresses this issue by jointly modeling shape and pose deformations. Experimental results demonstrate that our tensor-based model outperforms the SCAPE model quite significantly. We also apply our model to capture human body using Microsoft Kinect sensors with excellent results.
A terrain referenced underwater positioning using sonar bathymetric profiles and multiscale analysis

Terrain-based underwater navigation using sonar bathymetric profiles is presented. It deals with matching high resolution local depth maps against a large, onboard, low resolution reference map. The matching algorithm locates the local depth map within the a priori larger map to determine absolute position and heading of the vehicle. It uses cliff maps which are steep gradient contours extracted from both local and reference maps. This segmentation provides us with a means to extract critical points, which are defined as high curvature values. The problem is reduced to a singular point-based matching algorithm: given two point sets, find correspondences and estimate transformation between the two sets. In order to register maps at different scales, a part of this study will focus on partial differential equations based scale-space techniques. This approach is tested using real terrain data of the Var underwater canyon (France).
A compact multi-sensor headset for hands-free communication

The need for hands-free communication has led to an increased popularity in the use of headsets with mobile phones. Comfort and portability concerns have led to the desire for headsets with a small form factor. Unfortunately, this size constraint typically requires that the microphone be placed farther from the user's mouth, making it highly susceptible to environmental noise. One long term goal of our work is to develop a headset that can achieve the sound capture performance of a close-talking microphone located at the user's mouth, while maintaining the desired compact size. Toward this end, we have designed a headset consisting of three air microphones and a bone-conductive sensor. The speech enhancement is performed in two stages, a fixed beamformer followed by a single-channel adaptive post-filter. Unlike other techniques, the beamformer is calibrated in a purely data-driven manner. The bone sensor provides a robust speech activity detector for use in the post-filtering stage. We present preliminary experimental results using real data collected in multiple environments. The proposed approach results in significant improvements in both speech recognition accuracy and SNR.
Estimating motion and structure from correspondences of line segments between two perspective images

Presents an algorithm for determining 3D motion and structure from correspondences of line segments between two perspective images. To our knowledge, the paper is the first investigation of use of line segments in motion and structure from motion. Classical methods use their geometric abstraction, namely straight lines, but then three images are necessary for the motion and structure determination process. We show that two views are in general sufficient when we use line segments. The assumption we use is that two matched line segments contain the projection of a common part of the corresponding line segment in space. Indeed this is what we use to match line segments between different views. Both synthetic and real data have been used to test the proposed algorithm, and excellent results have been obtained with real data containing a relatively large set of line segments. The results are comparable with those obtained using stereo calibration.<<ETX>>
Real-Time Whiteboard Capture and Processing Using a Video Camera for Remote Collaboration

This paper describes our recently developed system which captures pen strokes on physical whiteboards in real time using an off-the-shelf video camera. Unlike many existing tools, our system does not instrument the pens or the whiteboard. It analyzes the sequence of captured video images in real time, classifies the pixels into whiteboard background, pen strokes and foreground objects (e.g., people in front of the whiteboard), extracts newly written pen strokes, and corrects the color to make the whiteboard completely white. This allows us to transmit whiteboard contents using very low bandwidth to remote meeting participants. Combined with other teleconferencing tools such as voice conference and application sharing, our system becomes a powerful tool to share ideas during online meetings
Euclidean structure from uncalibrated images using fuzzy domain knowledge: application to facial images synthesis

Use of uncalibrated images has found many applications such as image synthesis. However, it is not easy to specify the desired position of the new image in projective or affine space. This paper proposes to recover Euclidean structure from uncalibrated images using domain knowledge such as distances and angles. The knowledge we have is usually about an object category, but not very precise for the particular object being considered. The variation (fuzziness) is modeled as a Gaussian variable. Six types of common knowledge are formulated. Once we have an Euclidean description, the task to specify the desired position in Euclidean space becomes trivial. The proposed technique is then applied to synthesis of new facial images. A number of difficulties existing in image synthesis are identified and solved. For example, we propose to use edge points to deal with occlusion.
Note-taking with a camera: whiteboard scanning and image enhancement

The paper describes a system for scanning the content on a whiteboard into a computer by a digital camera and also for enhancing the visual quality of whiteboard images. Because digital cameras are becoming accessible to average users, more and more people use digital cameras to take pictures of whiteboards instead of copying manually, thus increasing productivity significantly. However, the images are usually taken from an angle, resulting in undesired perspective distortion. They also contain other distracting regions such as walls. We have developed a system that automatically locates the boundary of a whiteboard, crops out the whiteboard region, rectifies it into a rectangle, and corrects the color to make the whiteboard completely white. In the case where a single image is not enough (e.g., large whiteboard and low-res camera), we have developed a robust feature-based technique to stitch multiple overlapping images automatically. We therefore reproduce the whiteboard content as a faithful electronic document which can be archived or shared with others. The system has been tested extensively, and very good results have been obtained.
Segment-based structure from an imprecisely located moving camera

A probabilistic geometric model for 2D image line segments is first presented. We then propose a method using 2D segments, to accumulate evidence along an image sequence of a polyhedral scene. The proposed method degrades gracefully with camera location noise. The matched 2D segments are fused with an extended Kalman filter to reconstruct the scene structure. The correspondences are established using sequential data association. The matching function encodes the consistency between the 2D segments and the 3D segment computed from their fusion. The computation of a 3D segment from two 2D segments is overconstrained, and so using our model, the after-fusion consistency test can reject false matches even from two images, allowing the pruning of false matching hypotheses at early stages. Two examples are provided. The first determines a scene structure when the camera location is known precisely; the structure is then compared with that obtained by trinocular stereo. The proposed method improves 3D segment orientation computation and reduces the number of spurious segments. A second experiment demonstrates the graceful degradation of the performance, especially in orientation, with respect to the camera location noise.
A Generative-Discriminative Framework using Ensemble Methods for Text-Dependent Speaker Verification

Speaker verification can be treated as a statistical hypothesis testing problem. The most commonly used approach is the likelihood ratio test (LRT), which can be shown to be optimal using the Neymann-Pearson lemma. However, in most practical situations the Neymann-Pearson lemma does not apply. In this paper, we present a more robust approach that makes use of a hybrid generative-discriminative framework for text-dependent speaker verification. Our algorithm makes use of a generative models to learn the characteristics of a speaker and then discriminative models to discriminate between a speaker and an impostor. One of the advantages of the proposed algorithm is that it does not require us to retrain the generative model. The proposed model, on an average, yields 36.41% relative improvement in EER over a LRT.
Face Re-Lighting from a Single Image under Harsh Lighting Conditions

In this paper, we present a new method to change the illumination condition of a face image, with unknown face geometry and albedo information. This problem is particularly difficult when there is only one single image of the subject available and it was taken under a harsh lighting condition. Recent research demonstrates that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace using spherical harmonic representation. However, the approximation error can be large under harsh lighting conditions thus making it difficult to recover albedo information. In order to address this problem, we propose a subregion based framework that uses a Markov Random Field to model the statistical distribution and spatial coherence of face texture, which makes our approach not only robust to harsh lighting conditions, but insensitive to partial occlusions as well. The performance of our framework is demonstrated through various experimental results, including the improvement to the face recognition rate under harsh lighting conditions.
VTouch: Vision-enhanced interaction for large touch displays

We propose a system that augments touch input with visual understanding of the user to improve interaction with a large touch-sensitive display. A commodity color plus depth sensor such as Microsoft Kinect adds the visual modality and enables new interactions beyond touch. Through visual analysis, the system understands where the user is, who the user is, and what the user is doing even before the user touches the display. Such information is used to enhance interaction in multiple ways. For example, a user can use simple gestures to bring up menu items such as color palette and soft keyboard; menu items can be shown where the user is and can follow the user; hovering can show information to the user before the user commits to touch; the user can perform different functions (for example writing and erasing) with different hands; and the user's preference profile can be maintained, distinct from other users. User studies are conducted and the users very much appreciate the value of these and other enhanced interactions.
Robust and Accurate Visual Echo Cancelation in a Full-Duplex Projector-Camera System

We developed a comprehensive set of techniques to address the visual echo problem in a full-duplex projectorcamera system. A calibration procedure records the geometric and photometric transfer between the projector and camera in a look-up table. With the calibration information, the predicted camera view of the projected image is compared against the captured camera image to find echo pixels. Only non-echo pixels are used for display, therefore achieving the goal of suppressing visual echo. Compared to previous techniques, our approachs main advantages are two-fold. First, it accurately handles full color images with no assumption about the surface reflectance or the photometric response of the projector or camera. Secondly, it is robust to geometric registration errors and quantization effect. It is particularly effective for high-frequency contents such as texts and hand drawings. We demonstrate the effectiveness of our approach with a variety of real images in a full-duplex projector-camera system.
Camera Calibration With Three Noncollinear Points Under Special Motions

Plane-based (2D) camera calibration is becoming a hot research topic in recent years because of its flexibility. However, at least four image points are needed in every view to denote the coplanar feature in the 2D camera calibration. Can we do the camera calibration by using the calibration object that only has three points? Some 1D camera calibration techniques use the setup of three collinear points with known distances, but it is a kind of special conditions of calibration object setup. How about the general setup-three noncollinear points? We propose a new camera calibration algorithm based on the calibration objects with three noncollinear points. Experiments with simulated data and real images are carried out to verify the theoretical correctness and numerical robustness of our results. Because the objects with three noncollinear points have special properties in camera calibration, they are midway between 1D and 2D calibration objects. Our method is actually a new kind of camera calibration algorithm.
What can be determined from a full and a weak perspective image?

This paper presents a first investigation on the structure from motion problem from the combination of full and weak perspective images. This problem arises in multiresolution object modeling where multiple zoomed-in or close-up views are combined with wider or distant reference views. The narrow field-of-view (FOV) images from the zoomed-in or closeup views can be approximated as weak perspective projection. Using a full perspective projection model for the narrow FOV images, although more accurate, actually leads to instabilities during the estimation process due to the non-linearities in the imaging model. The weak perspective approximation leads to more stable estimation algorithms, although at the cost of a small amount of modeling inaccuracy. Previous work in structure from motion focused either on two (or more) perspective images or on a set of weak perspective (more generally, affine) images. The main contribution of this paper is the study of the SFM problem for the much neglected case of one perspective and one (or more) weak perspective image. We show that in contrast to the case of a pair of weak perspective images, there is adequate information to recover Euclidean structure from a single perspective and a single weak perspective image. The epipolar geometry is simpler than with two perspective images leading to simpler and more stable estimation algorithms. Computer simulation shows that more stable results can be obtained with the technique presented in this paper than if two images are both considered to be full perspective.
Multimodal collaboration and human-computer interaction

The research effort at Microsoft Research on multimodal collaboration and human-computer interaction aims at developing tools that allow people across geographically distributed sites to interact collaboratively with immersive experience. Our prototype systems consist of cameras, displays, speakers, microphones, computer controllable lights, and/or input devices such as touch sensitive surface, stylus, keyboard, and mouse. They require real-time processing a huge amount of data, such as foreground-background substraction, region-ofinterest extraction, color estimation and correction, speaker detection, stereo matching, 3D reconstruction and rendering, without mentioning audio and video encoding and decoding possibly involving multiple microphones and cameras. Some of the processing can be easily parallelizable through generalpurpose computation on graphics processing units (GPGPU) or on a multi-core processor machine, while others are not so trivial. In this extended summary, I describe two projects: Visual Echo Cancellation in Shared Tele-collaborative Space, and Distributed Meeting Capture and Broadcasting System. During the talk, I will also present two recent projects: Personal Telepresence Station and Situated Interaction.
Head pose determination from one image using a generic model

We present a new method for determining the pose of a human head from its 2D image. It does not use any artificial markers put on a face. The basic idea is to use a generic model of a human head, which accounts for variation in shape and facial expression. Particularly, a set of 3D curves are used to model the contours of eyes, lips and eyebrows. A technique called iterative closest curve matching (ICC) is proposed, which aims at recovering the pose by iteratively minimizing the distances between the projected model curves and their closest image curves. Because curves contain richer information (such as curvature and length) than points, ICC is both more robust and more efficient than the well-known iterative closest point matching techniques (ICP). Furthermore, the image can be taken by a camera with unknown internal parameters, which can be recovered by our technique thanks to the 3D model. Preliminary experiments show that the proposed technique is promising and that an accurate pose estimate can be obtained from just one image with a generic head model.
Computing rectifying homographies for stereo vision

Image rectification is the process of applying a pair of 2D projective transforms, or homographies, to a pair of images whose epipolar geometry is known so that epipolar lines in the original images map to horizontally aligned lines in the transformed images. We propose a novel technique for image rectification based on geometrically well defined criteria such that image distortion due to rectification is minimized. This is achieved by decomposing each homography into a specialized projective transform, a similarity transform, followed by a shearing transform. The effect of image distortion at each stage is carefully considered.
An effecive night video enhancement algorithm

Night video enhancement is important for video surveillance since many objects or activities of interest occur in a dark environment which cannot be seen easily without enhancement. In this paper, we discuss several problems of existing techniques for illumination-fusion based night video enhancement, which fuses video frames from day-time backgrounds and night-time video. We then present a simple enhancement algorithm without these problems. The algorithm uses an additive enhancement term with foreground object extraction and constrained low-passed object illumination to avoid light-inversion and sensitivity problems and to reduce ghost patterns. Experimental results show the effectiveness and robustness of the proposed algorithm.
Dimension Reduction With Extreme Learning Machine.

Data may often contain noise or irrelevant information, which negatively affect the generalization capability of machine learning algorithms. The objective of dimension reduction algorithms, such as principal component analysis (PCA), non-negative matrix factorization (NMF), random projection (RP), and auto-encoder (AE), is to reduce the noise or irrelevant information of the data. The features of PCA (eigenvectors) and linear AE are not able to represent data as parts (e.g. nose in a face image). On the other hand, NMF and non-linear AE are maimed by slow learning speed and RP only represents a subspace of original data. This paper introduces a dimension reduction framework which to some extend represents data as parts, has fast learning speed, and learns the between-class scatter subspace. To this end, this paper investigates a linear and non-linear dimension reduction framework referred to as extreme learning machine AE (ELM-AE) and sparse ELM-AE (SELM-AE). In contrast to tied weight AE, the hidden neurons in ELM-AE and SELM-AE need not be tuned, and their parameters (e.g, input weights in additive neurons) are initialized using orthogonal and sparse random weights, respectively. Experimental results on USPS handwritten digit recognition data set, CIFAR-10 object recognition, and NORB object recognition data set show the efficacy of linear and non-linear ELM-AE and SELM-AE in terms of discriminative capability, sparsity, training time, and normalized mean square error.
Real-time whiteboard capture and processing using a video camera for teleconferencing

This paper describes our recently developed system which captures pen strokes on whiteboards in real time using an off-the-shelf video camera. Unlike many existing tools, our system does not instrument the pens or the whiteboard. It analyzes the sequence of captured video images in real time, classifies the pixels into whiteboard background, pen strokes and foreground objects (e.g., people in front of the whiteboard), and extracts newly written pen strokes. This allows us to transmit whiteboard contents using very low bandwidth to remote meeting participants. Combined with other teleconferencing tools such as voice conference and application sharing, our system becomes a powerful tool to share ideas during online meetings.
Exploring Discriminative Learning for Text-Independent Speaker Recognition

Speaker verification is a technology of verifying the claimed identity of a speaker based on the speech signal from the speaker (voice print). To learn the score of similarity between each pair of target and trial utterances, we investigated two different discriminative learning frameworks: Fisher mapping followed by SVM learning and utterance transform followed by iterative cohort modeling (ICM). In both methods, a mapping is applied to map speech utterance from a variable-length acoustic feature sequence into a fixed dimensional vector. SVM learning constructs a classifier in the mapped vector space for speaker verification. ICM learns a metric in this vector space by incorporating discriminative learning methods. The obtained metric is then used by a nearest neighbor classifier for speaker verification. The experiments conducted on NIST02 corpus show that both discriminative learning methods outperform the baseline GMM-UBM system. Furthermore, we observe that the ICM-based method is more effective than the SVM-based method, indicating that the metric learning scheme is more powerful in constructing a better metric in the mapped vector space.
Motion and structure of four points from one motion of a stereo rig with unknown extrinsic parameters

An analytical method for recovering 3-D motion and structure of four or more points from one motion of a stereo rig is described. The extrinsic parameters are unknown. Because of the exploitation of information redundancy, the approach gains over the traditional motion and structure from motion approach in that less features and less motions are required. Thus, more robust estimation of motion and structure can be obtained. Since the constraint on the rotation matrix is not fully exploited in the analytical method, nonlinear minimization can be used to improve the result. Both computer simulated data and real data are used to validate the proposed algorithm. Very promising results are obtained.<<ETX>>