nils-herrmann/clean_scirepeval huggingface dataset:
A unified framework for atlas-based segmentation with forward deformation and label refinement

In this paper, a novel unified framework for atlas-based segmentation is proposed, consisting of two main components: forward deformation and label refinement. A newly designed distance constraint on mesh edges is enforced with contrast sensitivity in forward deformation based on Markov random field. With the edge distance constraint, the object shapes in the atlas and the target images can remain similar during deformation. Considering the shape variations caused by individual difference, we then develop a label refinement process embracing patch registration and label fusion to compensate the small variations around the structural surfaces. As the anatomical correspondences determined in forward deformation can differ from that in label refinement, the conventional one-to-one correspondence constraint can be relaxed in our framework. Experiments on two publicly available databases IBSR and LPBA40 demonstrate that our method can obtain better performance as compared with other state-of-the-art methods.
Measuring the engagement level of TV viewers

This work studies the feasibility of using visual information to automatically measure the engagement level of TV viewers. Previous studies usually utilize expensive and invasive devices (e.g., eye trackers or physiological sensors) in controlled settings. Our work differs by only using an RGB video camera in a naturalistic setting, where viewers move freely and respond naturally and spontaneously. In particular, we recorded 47 people while watching a TV program and manually coded the engagement levels of each viewer. From each video, we extracted several features characterizing facial and head gestures, and used several aggregation methods over a short time window to capture the temporal dynamics of engagement. We report on classification results using the proposed features, and show improved performance over baseline methods that mostly rely on head-pose orientation.
Active Lighting for Video Conferencing

In consumer video conferencing, lighting conditions are usually not ideal thus the image qualities are poor. Lighting affects image quality on two aspects: brightness and skin tone. While there has been much research on improving the brightness of the captured images including contrast enhancement and noise removal (which can be thought of as components for brightness improvement), little attention has been paid to the skin tone aspect. In contrast, it is a common knowledge for professional stage lighting designers that lighting affects not only the brightness but also the color tone which plays a critical role in the perceived look of the host and the mood of the stage scene. Inspired by stage lighting design, we propose an active lighting system which automatically adjusts the lighting so that the image looks visually appealing. The system consists of computer controllable light emitting diode light sources of different colors so that it improves not only the brightness but also the skin tone of the face. Given that there is no quantitative formula on what makes a good skin tone, we use a data driven approach to learn a good skin tone model from a collection of photographs taken by professional photographers. We have developed a working system and conducted user studies to validate our approach.
Graphical modeling and decoding of human actions

This paper presents a graphical model for learning and recognizing human actions. Specifically, we propose to encode actions in a weighted directed graph, referred to as action graph, where nodes of the graph represent salient postures that are used to characterize the actions and shared by all actions. The weight between two nodes measures the transitional probability between the two postures. An action is encoded as one or multiple paths in the action graph. The salient postures are modeled using Gaussian mixture models (GMM). Both the salient postures and action graph are automatically learned from training samples through unsupervised clustering and expectation and maximization (EM) algorithm. Experimental results have verified the performance of the proposed model, its tolerance to noise and viewpoints and its robustness across different subjects and datasets.
Meta-tag propagation by co-training an ensemble classifier for improving image search relevance

The ever-increasing gigantic amount of images over the Web necessitates automatic schemes for meta-tagging content descriptions such as object categories. These meta-tags are essential to text-based image search engines to improve their search relevance. Traditional supervised scheme is not suitable for this task because it needs too much manual labelling efforts and yet is hard to scale to a large number of object categories. Notice that in the search scenarios, the meta-tagging does not need to be perfect to help improve relevance because the available text tags and user click-through logs can partially rectify the inaccurate information. A weakly supervised scheme would be ideal when only sporadic labelled examples are exploited in spite of the expected loss in tagging accuracy. In this paper, we develop a novel weakly semi-supervised ensemble classifier trained based on a co-training framework for this tagging task. In essence the meta-tags are recursively propagated from the sparsely tagged examples to the un-tagged ones. Preliminary experiments on benchmark database such as Graz02 demonstrate the efficacy of the proposed approach.
Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron

The authors investigate the use of two types of features extracted from face images for recognizing facial expressions. The first type is the geometric positions of a set of fiducial points on a face. The second type is a set of multi-scale and multi-orientation Gabor wavelet coefficients extracted from the face image at the fiducial points. They can be used either independently or jointly. The architecture developed is based on a two-layer perceptron. The recognition performance with different types of features has been compared, which shows that Gabor wavelet coefficients are much more powerful than geometric positions. Furthermore, since the first layer of the perceptron actually performs a nonlinear reduction of the dimensionality of the feature space, they have also studied the desired number of hidden units, i.e., the appropriate dimension to represent a facial expression in order to achieve a good recognition rate. It turns out that five to seven hidden units are probably enough to represent the space of feature expressions.
On the epipolar geometry between two images with lens distortion

In order to achieve a 3D, either Euclidean or projective, reconstruction with high precision, one has to consider lens distortion. In almost all work on multiple-views problems in computer vision, a camera is modeled as a pinhole. Lens distortion has usually been corrected off-line. This paper intends to consider lens distortion as an integral part of a camera. We first describe the epipolar geometry between two images with lens distortion. For a point in one image, its corresponding point in the other image should lie on a so-called epipolar curve. We then investigate the possibility of estimating the distortion parameters and the fundamental matrix based on the generalized epipolar constraint. Experimental results with computer simulation show that the distortion parameters can be estimated correctly if the noise in image points is low and the lens distortion is severe. Otherwise, it is better to treat the cameras as being distortion-free.
Speech Modelingwith Magnitude-Normalized Complex Spectra and Its Application to Multisensory Speech Enhancement

A good speech model is essential for speech enhancement, but it is very difficult to build because of huge intra- and extra-speaker variation. We present a new speech model for speech enhancement, which is based on statistical models of magnitude-normalized complex spectra of speech signals. Most popular speech enhancement techniques work in the spectrum space, but the large variation of speech strength, even from the same speaker, makes accurate speech modeling very difficult because the magnitude is correlated across all frequency bins. By performing magnitude normalization for each speech frame, we are able to get rid of the magnitude variation and to build a much better speech model with only a small number of Gaussian components. This new speech model is applied to speech enhancement for our previously developed microphone headsets that combine a conventional air microphone with a bone sensor. Much improved results have been obtained
CIS Publication Spotlight [Publication Spotlight]

Digital Object Identifier: 10.1109/ TNNLS.2013.2295813 “This paper proposes an offline algorithm for incrementally constructing and training radial basis function (RBF) networks. In each iteration of the error correction (ErrCor) algorithm, one RBF unit is added to fit and then eliminate the highest peak (or lowest valley) in the error surface. This process is repeated until a desired error level is reached. Experimental results on real world data sets show that the ErrCor algor ithm designs very compact RBF networks compared with the other exiting algorithms. Several benchmark tests such as the duplicate patterns test and the two spiral problem were applied to show the robustness of the ErrCor algorithm. The proposed ErrCor algorithm generates very compact networks. This compactness leads to greatly reduced computation times of trained networks.” A New Learning Algorithm for a Fully Connected Neuro-Fuzzy Inference System, by C.L.P. Chen, J. Wang, C.H. Wang, and L. Chen,  2014,
Enhancing stereophonic teleconferencing with microphone arrays through sound field warping

It has been proven that spatial audio enhances the realism of sound for teleconferencing. Previously, solutions have been proposed for multiparty conferencing where each remote participant is assumed to have his/her own microphone, and for conferencing between two rooms where the microphones in one room are connected to the equal number of loudspeakers in the other room. Either approach has its limitations. Hence, we propose a new scheme to improve stereophonic conferencing experience through an innovative use of microphone arrays. Instead of operating in the default mode where a single channel is produced using spatial filtering, we propose to transmit all channels forming a collection of spatial samples of the sound field. Those samples are warped appropriately at the remote site, and are spatialized together with audio streams from other remote sites if any, to produce the perception of a virtual sound field. Real-world audio samples are provided to showcase the proposed technique. The informal listening test shows that majority of the users prefer the new experience.
Motion and Structure of Four Points from One Motion of a Stereo Rig with Unknown Extrinsic Parameters

We describe an analytical method for recovering 3D motion and structure of four or more points from one motion of a stereo rig. The extrinsic parameters are unknown. The motion of the stereo rig is also unknown. Because of the exploitation of information redundancy, the approach gains over the traditional "motion and structure from motion" approach in that less features and less motions are required, and thus more robust estimation of motion and structure can be obtained. Since the constraint on the rotation matrix is not fully exploited in the analytical method, nonlinear minimization can be used to improve the result. We propose to estimate directly the motion and structure by minimizing the difference between the measured positions and the predicted ones in the image plane. Both computer simulated data and real data are used to validate the proposed algorithm, and very promising results are obtained.
CIS Publication Spotlight [Publication Spotlight]

Digital Object Identifier: 10.1109/ TNNLS.2012.2235082 nonconvex framework is proposed for learning the essential low-rank structure from corrupted data. Different from traditional approaches, which directly utilizes convex norms to measure the sparseness, this method introduces more reasonable nonconvex measurements to enhance the sparsity in both the intrinsic low-rank structure and the sparse corruptions. It includes how to combine the widely used Lp norm (0<p<1) and log-sum term into the framework of low-rank structure learning. Although the proposed optimization is no longer convex, it still can be effectively solved by a major ization–minimization (MM)type algorithm, with which the nonconvex objective function is iteratively replaced by its convex surrogate and the nonconvex problem finally falls in to the genera l f r amework of reweighed approaches. It is proved that the MM-type algorithm can converge to a stationary point after successive iterations. The proposed model is applied to solve two typical problems: robust pr incipa l component analysis and low-rank representation. Exper imental results on low-rank structure learning demonstrate that our nonconvex heuristic methods, especially the log-sum heuristic recovery algorithm, generally perform much better than the convex-norm-based method (0<p<1) for both data with higher rank and with denser corruptions.”
Model-based head pose tracking with stereovision

We present a robust model-based stereo head tracking algorithm that operates in real time on a commodity PC. The use of an individualized three-dimensional head model, coupled with the epipolar constraint from the stereo image pair greatly improves the robustness of the tracking. Experimental results have shown that our method is able to track all the six degrees of freedom of the rigid part of head motions, over an extended period of time, in the presence of large angular and translational head motions, partial occlusions, and/or dramatic facial expression changes. Applications include human-computer interaction and eye-gaze correction for videoconferencing.
CIS Publication Spotlight [Publication Spotlight]

Digital Object Identifi er: 10.1109/ TNN.2011.2109396 “Online blind source separation (BSS) can overcome the high computational cost problem, which limits the practical applications of traditional batch BSS algorithms. However, existing online BSS methods are mainly used to separate independent or uncorrelated sources. In this paper, an incremental nonnegative matrix factorization (NMF) with volume constraint is derived and utilized for solving online BSS, which can be used to separate correlative sources. The volume constraint to the mixing matrix enhances the identifi ability of the sources, while the incremental learning mode reduces the computational cost. This new method takes advantage of the natural gradient based multiplication updating rule, and it performs especially well in the recovery of dependent sources.”
Calibration of a mobile robot with application to visual navigation

The authors address the calibration problem between the stereo coordinate system and the odometric system of the INRIA mobile robot. A well-known equation is used to recover the rotation axis of the mobil robot in the stereo system, using information of motion from stereo. The calibration can be done with some simple maneuvers (pure rotation and pure translation). A motion-estimation algorithm from stereo is used to determine the displacement in the stereo coordinate system. With the calibration results, it is possible to transform the displacement that the robot should perform in the stereo reference to achieve its goal into the robot's internal commands. As a sample application, a complete visual navigation loop is demonstrated.<<ETX>>
Computer Vision Technologies for Remote Collaboration Using Physical Whiteboards, Projectors and Cameras

A whiteboard provides a large shared space for the participants to focus their attention and express their ideas, and is therefore a great collaboration tool for information workers. However, it has several limitations; notably, the contents on the whiteboard are hard to archive or share with people who are not present in the discussions. This paper presents our work in developing tools to facilitate collaboration on physical whiteboards by using a camera and a microphone. In particular, we have developed two systems: a whiteboard-camera system and a projector-whiteboardcamera system. The whiteboard-camera system allows a user to take notes of a whiteboard meeting when he/she wants or in an automatic way, to transmit the whiteboard content to remote participants in real time and in an efficient way, and to archive the whole whiteboard session for efficient post-viewing. The projector-whitebord-camera system incorporates a whiteboard into a projectorcamera system. The whiteboard serves as the writing surface (input) as well as the projecting surface (output). Many applications of such a system inevitably require extracting handwritings from video images that contain both handwritings and the projected content. By analogy with echo cancellation in audio conferencing, we call this problem visual echo cancellation, and we describe one approach to accomplish the task. Our systems can be retrofit to any existing whiteboard. With the help of a camera and a microphone and optionally a projector, we are effectively bridging the physical and digital worlds.
Action recognition based on a bag of 3D points

This paper presents a method to recognize human actions from sequences of depth maps. Specifically, we employ an action graph to model explicitly the dynamics of the actions and a bag of 3D points to characterize a set of salient postures that correspond to the nodes in the action graph. In addition, we propose a simple, but effective projection based sampling scheme to sample the bag of 3D points from the depth maps. Experimental results have shown that over 90% recognition accuracy were achieved by sampling only about 1% 3D points from the depth maps. Compared to the 2D silhouette based recognition, the recognition errors were halved. In addition, we demonstrate the potential of the bag of points posture model to deal with occlusions through simulation.
Hierarchical Filtered Motion for Action Recognition in Crowded Videos

Action recognition with cluttered and moving background is a challenging problem. One main difficulty lies in the fact that the motion field in an action region is contaminated by the background motions. We propose a hierarchical filtered motion (HFM) method to recognize actions in crowded videos by the use of motion history image (MHI) as basic representations of motion because of its robustness and efficiency. First, we detect interest points as the two-dimensional Harris corners with recent motion, e.g., locations with high intensities in the MHI. Then, a global spatial motion smoothing filter is applied to the gradients of the MHI to eliminate isolated unreliable or noisy motions. At each interest point, a local motion field filter is applied to the smoothed gradients of the MHI by computing structure proximity between any pixel in the local region and the interest point. Thus, the motion at a pixel is enhanced or weakened based on its structure proximity with the interest point. To validate its effectiveness, we characterize the spatial and temporal features by histograms of oriented gradient in the intensity image and the MHI, respectively, and use a Gaussian-mixture-model-based classifier for action recognition. The performance of the proposed approach achieves the state-of-the-art results on the KTH dataset that has clean background. More importantly, we perform cross-dataset action classification and detection experiments, where the KTH dataset is used for training, while the microsoft research (MSR) action dataset II that consists of crowded videos with people moving in the background is used for testing. Our experiments show that the proposed HFM method significantly outperforms existing techniques.
Microsoft Kinect Sensor and Its Effect

Recent advances in 3D depth cameras such as Microsoft Kinect sensors (www.xbox.com/en-US/kinect) have created many opportunities for multimedia computing. The Kinect sensor lets the computer directly sense the third dimension (depth) of the players and the environment. It also understands when users talk, knows who they are when they walk up to it, and can interpret their movements and translate them into a format that developers can use to build new experiences. While the Kinect sensor incorporates several advanced sensing hardware, this article focuses on the vision aspect of the Kinect sensor and its impact beyond the gaming industry.
A developing sensory mapping for robots

A sensory mapping method, called staggered hierarchical mapping (SHM), and its developmental algorithm are described in this paper SHM is a model motivated by human early, visual pathways including processing performed by the retina, lateral geniculate nucleus (LGN) and the primary visual cortex. The work reported here concerns not only the design of such a series of processors but also their autonomous development. A new incremental principal component analysis (IPCA) method is used to automatically develop orientation sensitive and other needed filters. A set of staggered receptive fields model the pattern of positioning of processing cells. From sequentially sensed video frames, the proposed algorithm develops a hierarch, of filters, whose outputs are uncorrelated within each layer but with increasing scale of receptive fields from low to high layers. To study, the completeness of the representation generated by, the SHM, we experimentally, show that the response produced at any layer is sufficient to reconstruct the corresponding "retinal" image to a great degree. In our experiment, we show that SHM can be used to perform local analysis.
Motion of an uncalibrated stereo rig: self-calibration and metric reconstruction

We address in this paper the problem of self-calibration and metric reconstruction (up to a scale factor) from one unknown motion of an uncalibrated stereo rig. The epipolar constraint is first formulated for two uncalibrated images. The problem then becomes one of estimating unknowns such that the discrepancy from the epipolar constraint, in terms of sum of squared distances between points and their corresponding epipolar lines, is minimized. Although the full self-calibration is theoretically possible, we assume in this paper that the coordinates of the principal point of each camera are known. Then, the initialization of the unknowns can be done based on our previous work on self-calibration of a single moving camera, which requires to solve a set of so-called Kruppa equations. Redundancy of the information contained in a sequence of stereo images makes this method more robust than using a sequence of monocular images. Real data has been used to test the proposed method, and the results obtained are quite good. We also show experimentally that it is very difficult to estimate precisely the coordinates of the principal points of cameras. A variation of as high as several dozen pixels in the principal point coordinates does not affect significantly the 3-D reconstruction.