nils-herrmann/clean_scirepeval huggingface dataset:
Eye gaze correction with stereovision for video-teleconferencing

The lack of eye contact in desktop video teleconferencing substantially reduces the effectiveness of video contents. While expensive and bulky hardware is available on the market to correct eye gaze, researchers have been trying to provide a practical software-based solution to bring video-teleconferencing one step closer to the mass market. This paper presents a novel approach: based on stereo analysis combined with rich domain knowledge (a personalized face model), we synthesize, using graphics hardware, a virtual video that maintains eye contact. A 3D stereo head tracker with a personalized face model is used to compute initial correspondences across two views. More correspondences are then added through template and feature matching. Finally, all the correspondence information is fused together for view synthesis using view morphing techniques. The combined methods greatly enhance the accuracy and robustness of the synthesized views. Our current system is able to generate an eye-gaze corrected video stream at five frames per second on a commodity 1 GHz PC.
Robust Local Scoring Function for Text-Independent Speaker Verification

Traditionally, the universal background model (UBM) is viewed as the background model of the entire acoustic feature space. We propose a novel interpretation of the UBM model, and consider it as a mapping function that transforms the variable length observations (speech utterances) into a fixed dimensional feature vector (sufficient statistics). After this mapping, a similarity measurement is computed on the fixed dimensional features. With this novel interpretation, we proposed a new similarity measurement which produces more than 10% relative improvement over the conventional UBM-MAP framework in both equal error rate and detection cost function
Robust head motion computation by taking advantage of physical properties

Head motion determination is an important problem for many applications including face modeling and tracking. We present a new algorithm to compute the head motion between two views from the correspondences of five feature points (eye corners, mouth corners and nose tip), and zero or more additional image point matches. The algorithm takes advantage of the physical properties of the feature points, such as symmetry, and it significantly improves the robustness of head motion estimation. This is achieved by reducing the number of unknowns to be estimated, thus increasing information redundancy. This idea can be easily extended to any number of feature point correspondences.
Image-Based Surface Detail Transfer

We present a novel technique, called Image-Based Surface Detail Transfer, to transfer geometric details from one surface to another with simple 2D image operations. The basic observation is that, without knowing its 3D geometry, geometric details (local deformations) can be extracted from a single image of an object in a way independent of its surface reflectance, and furthermore, these geometric details can be transferred to modify the appearance of other objects directly in images. We show examples including surface detail transfer between real objects, as well as between real and synthesized objects.
Leakage model and teeth clack removal for air- and bone-conductive integrated microphones

Continuing our previous work (Zhang et al. (2004), Liu et al. (2004)) on using air- and bone-conductive integrated microphones, and in particular on using the direct filtering approach (Liu et al. (2004)) for speech enhancement in noisy environments, we present in this paper a refined version of the direct filtering algorithm. The new algorithm takes into account explicitly the leakage of background noise into the bone channel. We also present a new algorithm that detects and removes an artifact known as teeth clacks. Experiments show that the addition of the above algorithms improves system performance to a large extent even in highly nonstationary noisy environments.
Robust and Accurate Visual Echo Cancelation in a Full-duplex Projector-Camera System

In this paper we study the problem of "visual echo" in a full-duplex projector-camera system for tele-collaboration applications. Visual echo is defined as the appearance of projected contents observed by the camera. It can potentially saturate the projected contents, similar to audio echo in telephone conversation. Our approach to visual echo cancelation includes an off-line calibration procedure that records the geometric and photometric transfer between the projector and the camera in a look-up table. During run-time, projected contents in the captured video are identified using the calibration information and suppressed, therefore achieving the goal of canceling visual echo. Our approach can accurately handle full color images under arbitrary reflectance of display surfaces and photometric response of the projector or camera. It is robust to geometric registration errors and quantization effect, therefore particularly effective for high-frequency contents such as texts and hand drawings. We demonstrate the effectiveness of our approach with a variety of real images in a full-duplex projector-camera system.
Auditory augmented reality: Object sonification for the visually impaired

Augmented reality applications have focused on visually integrating virtual objects into real environments. In this paper, we propose an auditory augmented reality, where we integrate acoustic virtual objects into the real world. We sonify objects that do not intrinsically produce sound, with the purpose of revealing additional information about them. Using spatialized (3D) audio synthesis, acoustic virtual objects are placed at specific real-world coordinates, obviating the need to explicitly tell the user where they are. Thus, by leveraging the innate human capacity for 3D sound source localization and source separation, we create an audio natural user interface. In contrast with previous work, we do not create acoustic scenes by transducing low-level (for instance, pixel-based) visual information. Instead, we use computer vision methods to identify high-level features of interest in an RGB-D stream, which are then sonified as virtual objects at their respective real-world coordinates. Since our visual and auditory senses are inherently spatial, this technique naturally maps between these two modalities, creating intuitive representations. We evaluate this concept with a head-mounted device, featuring modes that sonify flat surfaces, navigable paths and human faces.
A Flexible New Technique for Camera Calibration

We propose a flexible technique to easily calibrate a camera. It only requires the camera to observe a planar pattern shown at a few (at least two) different orientations. Either the camera or the planar pattern can be freely moved. The motion need not be known. Radial lens distortion is modeled. The proposed procedure consists of a closed-form solution, followed by a nonlinear refinement based on the maximum likelihood criterion. Both computer simulation and real data have been used to test the proposed technique and very good results have been obtained. Compared with classical techniques which use expensive equipment such as two or three orthogonal planes, the proposed technique is easy to use and flexible. It advances 3D computer vision one more step from laboratory environments to real world use.
Introduction of New Associate Editors

Kathryn E. Merrick received the bachelor’s degree in computer science and technology (Advanced, Hons., University Medal), from the University of Sydney, Sydney, Australia, in 2002, and the Ph.D. degree in computer science from the National ICT Australia and University of Sydney, in 2007. She is a Senior Lecturer in Information Systems and Computer Science from the University of New South Wales, Australian Defence Force Academy. Her research interests include the broad areas of artificial intelligence and machine learning, with applications in virtual characters, robotics, intelligent environments and network security. Her research is principally concerned with the development of algorithms and metrics for self-motivated learning agents. She is coauthor of the book “Motivated Reinforcement Learning: Curious Characters for Multiuser Games” (Berlin: Springer-Verlag, 2009) and over forty refereed conference and journal papers.
Energy-Based Sound Source Localization and Gain Normalization for Ad Hoc Microphone Arrays

We present an energy-based technique to estimate both microphone and speaker/talker locations from an ad hoc network of microphones. An example of such ad hoc microphone network is a set of microphones built in the laptops that some meeting participants bring in a meeting room. Compared with traditional sound source localization approaches based on time of flight, our technique does not require accurate synchronization, and it does not require each laptop to emit special signals. We estimate the meeting participants' positions based on average energies of their speech signals. In addition, we present a technique, which is independent of the volumes of the speakers, to estimate the relative gains of the microphones. This is crucial to aggregate various audio channels from the ad hoc microphone network into a single stream for audio conferencing.
Corner guided curve matching and its application to scene reconstruction

Corners and curves are important image features in many vision-based applications. Corners are usually more stable and easier to match than curves, while curves contain richer information of scene structure. In previously work, corners are often used to recover the epipolar geometry between two views, which is then used in curve matching to reduce the search space. However, information of the scene structure contained in this set of matched corners is ignored. In this paper we present a curve matching algorithm that is guided by a set of matched corners. Within a probabilistic framework, the role of the corner guidance is explicitly defined by a set of similarity-invariant unary measurements and by a similarity function. The similarity function provides stronger capability of resolving matching ambiguity than the epipolar constraint, and is integrated into a relaxation scheme to reduce computational complexity and improve accuracy of curve matching. Experimental results clearly demonstrate the benefit of integrating corner matches into the curve matching procedure.
Autonomous Mental Development: A New Interdisciplinary Transactions for Natural and Artificial Intelligence

A LTHOUGH some baby animals can get up and walk within hours after birth, what a human child learns during the first two years of life easily exceeds what those animals learn in their entire lifetime. Furthermore, besides the explosive growth that occurs during this period, it is now well documented that a human brain continues its life-long development and learning . The human brain is one of the most complex systems we know of in the world, composed of about 100 billion strongly interconnected neurons. A single neuron may have more than 10 000 connections to other neurons. For thousands of years, the mind has been the center of myths and human beings have endeavored to understand our own brain and the mind arising from it. With the recent advances in cognitive science and neuroscience, e.g., with the help of brain imaging technologies such as the fMRI, EEG and PET, and many other direct observation and intervention techniques, more and deeper details of the brain’s inner workings are being revealed. Together with the advances in computational intelligence, computer science, and robotics, these discoveries have stimulated the birth and rapid growth of a new interdisciplinary research field known as Autonomous Mental Development (AMD) . Mental development is a process during which a brain-like natural or artificial embodied system, under the control of its intrinsic species-specific developmental program, develops mental capabilities through its real-time interactions with its environment (including the brain’s own internal environment) using its own sensors and effectors. The mental capabilities that develop in this way include perceptual, cognitive, behavioral, motivational, and all other mental capabilities that are exhibited by humans, higher animals, and artificial systems. The intrinsic developmental program and the interaction with the environment are both important for normal mental development: The environment affects how the developmental program in the genes works, which in turn regulates how the environment and experience give rise to the brain’s internal representations, mental capabilities, and internal and external behaviors. In recognition of the gains made in this field and to support its further development, the IEEE has approved this new IEEE TRANSACTIONS ON AUTONOMOUS MENTAL DEVELOPMENT (TAMD). Published four times a year, it will serve as an archival repository for significant work on this subject. The scope of TAMD includes: — Computational modeling of mental development, including mental architecture, theories, algorithms, properties and experiments; — Experimental investigations relevant to the goal of achieving a computational understanding of develop-
A multiscale approach for terrain referenced underwater navigation

A terrain-based underwater navigation using sonar bathymetric profiles is presented. It deals with matching high resolution local depth maps against a large, on-board, low resolution reference map. The matching algorithm locates the local depth map within the a priori larger map to determine absolute position and heading of the vehicle. It uses cliff maps which are steep gradient contours extracted from both local and reference maps. This segmentation provides us with a means to extract critical points, which are defined as high curvature values. The problem is reduced to a singular point-based matching algorithm: given two point sets, find correspondences and estimate transformation between the two sets. In order to register maps at different scales, a part of this study focuses on partial differential equations based scale-space techniques. This approach is tested using real terrain data of the Var underwater canyon (France).
[CIS Publication Spotlight]

Digital Object Identifier: 10.1109/ TNN.2010.2060209 “Two wide families of soft computing models are studied. On the one hand, the regime-switching autoregressive paradigm is a recent development in s ta t i s t ica l t ime ser ie s modeling, and it includes a set of models closely related to artificial neural networks. On the other hand, fuzzy rule-based systems in the framework of time series analysis are considered. This work discloses original results establishing functional equivalences between models of these two classes, and hence opens the door to a productive line of research where results and techniques from one area can be applied in the other.”
A new and efficient iterative approach to image matching

This paper addresses the problem of matching two images with unknown epipolar geometry. A new and efficient iterative algorithm is proposed, which is a part of the author's project on developing a robust image matching technique. The author defines a new measure of matching support, which allows less contribution and higher tolerance of deformation with respect to affine transformations from distant matches than from nearby ones. A new strategy for updating matches is developed, which only selects those matches having both high matching support and low matching ambiguity. The update strategy is different from the classical "winner-take-all", which evolves too soon and is easily stuck at a local minimum, and also from "loser-take-nothing", which is usually very slow. The proposed algorithm has been tested with two dozen image pairs of very different types of scenes, and very good results have been obtained. It works remarkably well in a scene with many repetitive patterns.
Determining reflectance parameters and illumination distribution from a sparse set of images for view-dependent image synthesis

A framework for photo-realistic view-dependent image synthesis of a shiny object from a sparse set of images and a geometric model is proposed. Each image is aligned with the 3D model and decomposed into two images with regards to the reflectance components based on the intensity variation of object surface points. The view-independent surface reflection (diffuse reflection) is stored as one texture map. The view-dependent reflection (specular reflection) images are used to recover the initial approximation of the illumination distribution, and then a two step numerical minimization algorithm utilizing a simplified Torrance-Sparrow reflection model is used to estimate the reflectance parameters and refine the illumination distribution. This provides a very compact representation of the data necessary to render synthetic images from arbitrary view points. We have conducted experiments with real objects to synthesize photorealistic view-dependent images within the proposed framework.
Expandable Data-Driven Graphical Modeling of Human Actions Based on Salient Postures

This paper presents a graphical model for learning and recognizing human actions. Specifically, we propose to encode actions in a weighted directed graph, referred to as action graph, where nodes of the graph represent salient postures that are used to characterize the actions and are shared by all actions. The weight between two nodes measures the transitional probability between the two postures represented by the two nodes. An action is encoded as one or multiple paths in the action graph. The salient postures are modeled using Gaussian mixture models (GMMs). Both the salient postures and action graph are automatically learned from training samples through unsupervised clustering and expectation and maximization (EM) algorithm. The proposed action graph not only performs effective and robust recognition of actions, but it can also be expanded efficiently with new actions. An algorithm is also proposed for adding a new action to a trained action graph without compromising the existing action graph. Extensive experiments on widely used and challenging data sets have verified the performance of the proposed methods, its tolerance to noise and viewpoints, its robustness across different subjects and data sets, as well as the effectiveness of the algorithm for learning new actions.
Estimating Motion and Structure from Correspondences of Line Segments between Two Perspective Images

Presents an algorithm for determining 3D motion and structure from correspondences of line segments between two perspective images. To the author's knowledge, this paper is the first investigation of use of line segments in motion and structure from motion. Classical methods use their geometric abstraction, namely straight lines, but then three images are necessary for the motion and structure determination process. In this paper the author shows that it is possible to recover motion from two views when using line segments. The assumption used is that two matched line segments contain the projection of a common part of the corresponding line segment in space, i.e., they overlap. Indeed, this is what the author uses to match line segments between different views. This assumption constrains the possible motion between two views to an open set in motion parameter space. A heuristic, consisting of maximizing the overlap, leads to a unique solution. Both synthetic and real data have been used to test the proposed algorithm, and excellent results have been obtained with real data containing a relatively large set of line segments.
Cloning your own face with a desktop camera

We have developed an easy and cost-effective system that constructs textured 3D animated face models from videos with minimal user interaction. Our system first takes, with an ordinary video camera, images of a face of a person sitting in front of the camera turning the head from one side to the other. After five manual clicks on two images to tell the system where the eye corners, nose top and mouth corners are, the system automatically generates a realistic looking 3D human head model and the constructed model can be animated immediately (different poses, facial expressions and talking). A user, with a PC and a video camera, can use our system to generate hisher face model in a few minutes. The face model can then be imported in hisher favorite game, and the user sees themselves and their friends take part in the game they are playing. We will demonstrate the system on a laptop computer live at the conference, and participants can try it to model their own faces.
CIS Publication Spotlight [Publication Spotlight]

Digital Object Identifier: 10.1109/ TNNLS.2012.2210559 “A methodology for quickly learning to play games at a strong level is developed. The methodology consists of a novel combination of three techniques, and a variety of experiments on the game of Othello demonstrates their usefulness. Structures or topologies in neural network connectivity patterns are used to decrease the number of learning parameters and to deal more effectively with the structural credit assignment problem, which is to change individual network weights based on the obtained feedback. Furthermore, the structured neural networks are trained with the novel neural-fitted temporal difference (TD) learning algorithm to create a system that can exploit most of the training exper iences and enhance learning speed and performance. Finally, the neural-fitted TD-leaf algorithm is used to learn more effectively when lookahead search is performed by the game-playing program. Extensive experimental study indicated that the proposed method outperforms linear networks and fully connected neural networks or evaluation functions evolved with evolutionary algorithms.”
Incremental motion estimation through modified bundle adjustment

An incremental motion estimation scheme for long image sequence analysis is introduced. It applies to a sliding window of triplet images and maintains local motion consistency without resort to post-concatenation. This is possible thanks to our newly developed local process called three-view partial bundle adjustment. Unlike previous approaches that rely only on point matches across three or more views, our local process also takes into account all available two-view matches, leading to more accurate motion estimation. For sparse image sequences, two-view matches are very reliable and it becomes even more important to use them since the number of matches across more views decreases quickly. In this case, our incremental method can produce results very close to those obtained with a global bundle adjustment but in a fraction of time. Experiments with both synthetic and real data have been conducted to compare the proposed technique with other techniques, and have shown our technique to be clearly superior.