nils-herrmann/clean_scirepeval huggingface dataset:
CIS Publication Spotlight [Publication Spotlight]

Digital Object Identifier: 10.1109/ TNNLS.2013.2293637 “There has been a recent increase in researchers’ attention on connectionist models composed by several hidden layers. In fact, experimental results and heuristic considerations suggest that deep architectures are more suitable than shallow ones for modern applications, facing very complex problems, e.g., vision and human language understanding. However, the actual theoretical results supporting such a claim are still few and incomplete. In this paper, a new approach is proposed to study how the depth of feedforward neural networks impacts on their ability in implementing high complexity functions. First, a new measure based on topological concepts is introduced, aimed at evaluating the complexity of the function implemented by a neural network, used for classification purposes. Then, deep and shallow neural architectures with common sigmoidal activation functions are compared, by deriving upper and lower bounds on their complexity, and studying how the complexity depends on the number of hidden units and the used activation function. The obtained results seem to support the idea that deep networks actually implement functions of higher complexity, so that they are able, with the same number of resources, to address more difficult problems.”
Estimation of Displacements from Two 3-D Frames Obtained From Stereo

A method for estimating 3D displacements from two stereo frames is presented. It is based on the hypothesize-and-verify paradigm used to match 3D line segments between the two frames. In order to reduce the complexity of the method, an assumption is made that objects are rigid. The formulate a set of complete rigidity constraints for 3D line segments and integrate the uncertainty of measurements in this formation. The hypothesize-and-verify stages of the method use an extended Kalman filter to produce estimates of the displacements and of their uncertainty. The algorithm is shown to work on indoor and natural scenes. It is also shown to be easily extended to the case in which several mobile objects are present. The method is quite robust, fast, and has been thoroughly tested on hundreds of real stereo frames.
Model-based bundle adjustment with application to face modeling

We present a new model-based bundle adjustment algorithm to recover the 3D model of a scene/object from a sequence of images with unknown motions. Instead of representing scene/object by a collection of isolated 3D features (usually points), our algorithm uses a surface controlled by a small set of parameters. Compared with previous model-based approaches, our approach has the following advantages. First instead of using the model space as a regularizer we directly use it as our search space, thus resulting in a more elegant formulation with fewer unknowns and fewer equations. Second, our algorithm automatically associates tracked points with their correct locations on the surfaces, thereby eliminating the need for a prior 2D-to-3D association. Third, regarding face modeling, we use a very small set of face metrics (meaningful deformations) to parameterize the face geometry, resulting in a smaller search space and a better posed system. Experiments with both synthetic and real data show that this new algorithm is faster, more accurate and more stable than existing ones.
Efficient bundle adjustment with virtual key frames: a hierarchical approach to multi-frame structure from motion

In this paper we present an efficient hierarchical approach to structure from motion for long image sequences. There are two key elements to our approach: accurate 3D reconstruction for each segment and efficient bundle adjustment for the whole sequence. The image sequence is first divided into a number of segments so that feature points can be reliably tracked across each segment. Each segment has a long baseline to ensure accurate 3D reconstruction. To efficiently bundle adjust 3D structures from ail segments, we reduce the number of frames in each segment by introducing "virtual keyframes". The virtual frames encode the 3D structure of each segment along with its uncertainty but they form a small subset of the original frames. Our method achieves significant speedup over conventional bundle adjustment methods.
Interpolation of combined head and room impulse response for audio spatialization

Audio spatialization is becoming an important part of creating realistic experiences needed for immersive video conferencing and gaming. Using a combined head and room impulse response (CHRIR) has been recently proposed as an alternative to using separate head related transfer functions (HRTF) and room impulse responses (RIR). Accurate measurements of the CHRIR at various source and listener locations and orientations are needed to perform good quality audio spatialization. However, it is infeasible to accurately measure or model the CHRIR for all possible locations and orientations. Therefore, low-complexity and accurate interpolation techniques are needed to perform audio spatialization in real-time. In this paper, we present a frequency domain interpolation technique which naturally interpolates the interaural level difference (ILD) and interaural time difference (ITD) for each frequency component in the spectrum. The proposed technique allows for an accurate and low-complexity interpolation of the CHRIR as well as allowing for a low-complexity audio spatialization technique which can be used for both headphones as well as loudspeakers.
Modeling geometric structure and illumination variation of a scene from real images

We present in this paper a system which automatically builds, from real images, a scene model containing both 3D geometric information of the scene structure and its photometric information under various illumination conditions. The geometric structure is recovered from images taken from distinct viewpoints. Structure-from-motion and correlation-based stereo techniques are used to match pixels between images of different viewpoints and to reconstruct the scene in 3D space. The photometric property is extracted from images taken under different illumination conditions (orientation, position and intensity of the light sources). This is achieved by computing a low-dimensional linear space of the spatio-illumination volume, and is represented by a set of basis images. The model that has been built can be used to create realistic renderings from different viewpoints and illumination conditions. Applications include object recognition, virtual reality and product advertisement.
Visual echo cancellation in a projector-camera-whiteboard system

We propose to incorporate a whiteboard into a projector-camera system. The whiteboard serves as the writing surface (input) as well as the projecting surface (output). The ability to write and draw on top of computer-projected content opens up many new opportunities for real-time collaborations between people located on-site and remotely. Such applications inevitably require extracting handwritings from video images that contain both handwritings and the projected content. By analogy with echo cancellation in audio conferencing, we call this problem visual echo cancellation. This paper presents one approach to accomplish the task. Our visual echo cancellation algorithm estimates the incident light and derives the surface albedo based on both incident light and reflection. By estimating the albedo, we can extract the writings and recover their colors. Our approach includes two basic components of projector-camera systems: geometric calibration and color calibration. The first one solves the mapping between the position in the camera view and the position in the projector screen, while the second one solves the mapping between the actual color of the projected content and that seen by the camera.
Face Relighting from a Single Image under Arbitrary Unknown Lighting Conditions

In this paper, we present a new method to modify the appearance of a face image by manipulating the illumination condition, when the face geometry and albedo information is unknown. This problem is particularly difficult when there is only a single image of the subject available. Recent research demonstrates that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace using a spherical harmonic representation. Moreover, morphable models are statistical ensembles of facial properties such as shape and texture. In this paper, we integrate spherical harmonics into the morphable model framework by proposing a 3D spherical harmonic basis morphable model (SHBMM). The proposed method can represent a face under arbitrary unknown lighting and pose simply by three low-dimensional vectors, i.e., shape parameters, spherical harmonic basis parameters, and illumination coefficients, which are called the SHBMM parameters. However, when the image was taken under an extreme lighting condition, the approximation error can be large, thus making it difficult to recover albedo information. In order to address this problem, we propose a subregion-based framework that uses a Markov random field to model the statistical distribution and spatial coherence of face texture, which makes our approach not only robust to extreme lighting conditions, but also insensitive to partial occlusions. The performance of our framework is demonstrated through various experimental results, including the improved rates for face recognition under extreme lighting conditions.
Office presence detection using multimodal context information

An office presence detection system is presented. Context information from multi-sensory inputs is integrated to infer a user's activities in an office. We design a layered architecture to model human activities with different granularities. An IHDR (incremental hierarchical discriminant regression) tree is used to generate models automatically for acoustic signals from unsegmented auditory streams, with a high adaptive capability to new settings. Hidden Markov models (HMM) are implemented to detect human motion patterns. The outputs of the above two components are fed into high-level HMMs to analyze human activities. Experimental results of the real-time prototype system are reported.
CIS Publication Spotlight [Publication Spotlight]

Digital Object Identifier: 10.1109/ TNNLS.2011.2181866 “The definition of a new type of competitive learning scheme realized on large-scale networks is presented which has wide application in artificial neural networks. The model consists of several particles walking within the network and competing with each other to occupy as many nodes as possible, while attempting to reject intruder particles. The particle’s walking rule is composed of a stochastic combination of random and preferential movements. The model has been applied to solve community detection and data clustering problems. Computer simulations reveal that the proposed technique presents high precision of community and cluster detections, as well as low computational complexity. Moreover, an efficient method is developed for estimating the most likely number of clusters by using an evaluator index that monitors the information generated by the competition process itself.”
Maximum a posteriori estimation of room impulse responses

Estimating room impulse responses (RIRs) has a number of applications, including personalized audio, analyzing and improving acoustic behavior of concert halls, listening room compensation, sound source localization, and many others. RIRs have been estimated in essentially the same fashion for the last 50 years: Compute the cross correlation between a signal played at point A, and the signal received at point B. Best results are obtained when the signal played is white noise, or a maximum length sequence. No prior knowledge is exploited in computing the RIR, which is simply assumed to be the cross correlation between played and received signals. In contrast, research in adaptive RIR estimation (a.k.a. adaptive Acoustic Echo Cancellation) has made huge progress by (among other things) incorporating models for the RIR. In this paper we propose a new RIR estimation technique, based on a maximum a posteriori formulation. More specifically, we estimate the room reverberation time, as well as the room noise level, and use those as priors for the RIR estimation. Comparison with ground truth shows an average improvement of 12 dB compared to traditional methods.
Automatic Real-Time Barcode Localization in Complex Scenes

Many applications exist for automatically finding and reading barcodes in complex scenes with a camera. The key problem is to search barcodes in a complex scene and supply them for a reading subsystem. However, illumination, rotation, perspective distortion and multiple barcodes circumstances make barcode localization difficult. By jointly analyzing texture and shape, we propose a realtime barcode localization method that is robust to the above problems. A user only needs to put a barcode in front of a camera at around 15 cm for common Web cameras, and the system then automatically locates and decodes the barcode. This paper only focuses at automatic localization. We will demonstrate the complete system live at the conference.
CIS Publication Spotlight [Publication Spotlight]

Digital Object Identifier: 10.1109/ TNN.2011.2160459 “An ensemble of classifiers-based approach is introduced for incremental learning of concept drift, characterized by nonstationary environments (NSEs), where the underlying data distributions change over time. The proposed algorithm learns from consecutive batches of data without making any assumptions on the nature or rate of drift; it can learn from such environments that exper ience constant or variable rate of drift, addition or deletion of concept classes, as well as cyclical drift. The algorithm learns incremental ly without requiring access to previously seen data. The novelty of the approach is in determining the voting weights, based on each classifier’s time-adjusted accuracy on current and past environments. This approach allows the algorithm to recognize, and act accordingly, to the changes in underlying data distributions, as well as to a possible reoccurrence of an earlier distribution. The algorithm is evaluated on several synthetic datasets designed to simulate a variety of nonstationary environments, as well as a real-world weather prediction dataset.”
Building a 3D world model with a mobile robot: 3D line segment representation and integration

A system whereby a mobile robot in an unknown environment can incrementally build a world model is described. The model discussed is segment-based. A trinocular system is used to build a local map of the environment. A global map is obtained by integrating a sequence of stereo frames taken while the robot navigates in the environment. Emphasis is on the representation of the uncertainty of 3D segments from stereo and on the integration of segments from multiple views. The representation is simple and very convenient for characterizing the uncertainty of segments. A Kalman filter is used to merge matched line segments. An important characteristic of the integration strategy is that a segment observed by the stereo system corresponds only to one part of the segment in space, so that the union of different observations gives a better estimate on the segment in space. The integration of 35 stereo frames taken in a robot room is described.<<ETX>>
An integrated multiscale approach for terrain referenced underwater navigation

A terrain-based underwater navigation using sonar bathymetric profiles is presented. It deals with matching high resolution local depth maps against a large, on-board, low resolution reference map. The matching algorithm locates the local depth map within the a priori larger map to determine absolute position and heading of the vehicle. It uses cliff maps which are steep gradient contours extracted from both local and reference maps. This segmentation provides us with a means to extract critical points, which are defined as high curvature values. The problem is reduced to a singular point-based matching algorithm: given two point sets, find correspondences and estimate transformation between the two sets. In order to register maps at different scales, a part of this study concentrates on partial differential equations based scale-space techniques. This approach is tested using real terrain data of the Var underwater canyon (France).
Real time head pose tracking from multiple cameras with a generic model

We present a robust approach to real-time 3D head pose tracking using multiple cameras with unknown camera placements. Many important applications do not want prior multi-camera calibration. We exploit a generic face model to overcome the difficulties due to the lack of prior knowledge of camera placement and the severe head appearance difference across cameras. We propose a fast drift-free solution based on feature point tracking using reference frames of high confidence over the temporal and spatial domains. Our algorithm tracks feature points from Harris feature detector, but not necessarily points of face landmarks. The relative camera placement is refined progressively at the same time as the user's head pose is resolved. Compared to single camera tracking, the use of multiple cameras increases the reliability of tracking by covering a wider range of poses as well as providing more accurate head pose estimation. We have tested the algorithm on many subjects in a variety of environments. A live demonstration will be shown at the conference.
Tracking 3D line segments: new developments

Reports developments of research in analyzing long sequences of 3D frames obtained from stereo. The 3D primitives used are 3D line segments. Previously, the authors (1990) proposed a framework consisting of two levels. At the low level, each 3D segment is tracked from frame to frame independently and its motion is estimated by filtering over time. This is the temporal integration. At the second level, the main function is to group line segments into objects according to the similarity of their motion parameters. This is the spatial integration. The authors describe a new formulation in the same framework to track segments and estimate their motion. Using this formulation, they describe any complex motion if its angular and translational velocities can be expressed by time polynomials. They propose also a new technique based on a statistical test to decide whether a token has enough support of existence to continue being tracked or can be discarded. This is very important to deal with problems such as occlusion, disappearance and absence of tokens. Experimental results are provided.<<ETX>>
Highly realistic audio spatialization for multiparty conferencing using headphones

It is known that during multi-party conferencing spatialized audio which maps remote participants' voices to distinct virtual locations improves the listening experience. In this paper, we consider the case when the audio is rendered through headphones due to e.g. privacy reasons. Although existing headphone spatial audio techniques abound, most lack the desired realism dictated by listeners' expectation of naturalness in audio conferencing. In light of the situation, we propose a novel approach of spatial audio processing for headphones, where we measure the combined head and room impulse responses (CHRIRs) in an actual physical setting which are then directly used to spatialize remote participants' voices. Through proper processing of the CHRIRs, our solution is able to offer a higher degree of realism, closely approximating binaural recordings. We note that the success, however, comes with certain limitations and presented solutions to mitigate the shortcomings of the proposed technique. As a result, a user can adapt the source localization and the degree of reverberation to her own subjective preferences. We also show that the computation load can be effectively reduced and becomes very reasonable for modern computer hardware.
ViewMark: An interactive videoconferencing system for mobile devices

ViewMark, a server-client based interactive mobile videoconferencing system is proposed in this paper to enhance the remote meeting experience for mobile users. Compared with the state-of-the-art mobile videoconferencing technology, ViewMark is novel in allowing a mobile user to interactively change the viewpoint of the remote video, create viewmarks, and hear with spatial audio. In addition, ViewMark also streams the screen of the presentation slides to mobile devices. In this paper, we introduce the system design of ViewMark in details, compare the devices that can be used to implement interactive videoconferencing, and demonstrate the prototype system we have built on Windows Mobile platform.
On the Optimization Criteria Used in Two-View Motion Analysis

The three best-known criteria in two-view motion analysis are based, respectively, on the distances between points and their corresponding epipolar lines, on the gradient-weighted epipolar errors, and on the distances between points and the re-projections of their reconstructed points. The last one has a better statistical interpretation, but is significantly slower than the first two. The author shows that, given a reasonable initial guess of the epipolar geometry, the last two criteria are equivalent when the epipoles are at infinity, and differ from each other only a little even when the epipoles are in the image, as shown experimentally. The first two criteria are equivalent only when the epipoles are at infinity and when the observed object/scene has the same scale in the two images. This suggests that the second criterion is sufficient in practice because of its computational efficiency. Experiments with several thousand computer simulations and four sets of real data confirm the analysis.
Why take notes? Use the whiteboard capture system

The paper describes our recently developed system which captures both whiteboard content and audio signals of a meeting using a digital still camera and a microphone. Our system can be retrofitted to any existing whiteboard. It computes the time stamps of pen strokes on the whiteboard by analyzing the sequence of captured snapshots. It also automatically produces a set of key frames representing all the written content on the whiteboard before each erasure. Therefore, the whiteboard content serves as a visual index to browse the audio meeting efficiently. It is a complete system which not only captures the whiteboard content, but also helps users to view and manage the captured meeting content efficiently and securely.
Flexible camera calibration by viewing a plane from unknown orientations

Proposes a flexible new technique to easily calibrate a camera. It only requires the camera to observe a planar pattern shown at a few (at least two) different orientations. Either the camera or the planar pattern can be freely moved. The motion need not be known. Radial lens distortion is modeled. The proposed procedure consists of a closed-form solution followed by a nonlinear refinement based on the maximum likelihood criterion. Both computer simulation and real data have been used to test the proposed technique, and very good results have been obtained. Compared with classical techniques which use expensive equipment, such as two or three orthogonal planes, the proposed technique is easy to use and flexible. It advances 3D computer vision one step from laboratory environments to real-world use. The corresponding software is available from the author's Web page ().