nils-herrmann/clean_scirepeval huggingface dataset:
Parking space detection from video by augmenting training dataset

Auto parking techniques are attracting more attention these days. In this paper, we develop an image-based method to estimate the depth contour in parking areas. Our algorithm is an extension of the canonical appearance-based models for object recognition. One challenge in object recognition is that limited training dataset can hardly represent all kinds intra-class and inter-class variations. We propose to augment the limited training dataset by on-the-spot learning from test data. The information is obtained by applying a fast block based stereo algorithm to estimate a rough disparity map. New “soft” samples are created to augment the training sample library. We present improved classification performance by using the proposed technique.
Multi-view 3D reconstruction for scenes under the refractive plane with known vertical direction

Images taken from scenes under water suffer distortion due to refraction. While refraction causes magnification with mild distortion on the observed images, severe distortions in geometry reconstruction would be resulted if the refractive distortion is not properly handled. Different from the radial distortion model, the refractive distortion depends on the scene depth seen from each light ray as well as the camera pose relative to the refractive surface. Therefore, it's crucial to obtain a good estimate of scene depth, camera pose and optical center to alleviate the impact of refractive distortion. In this work, we formulate the forward and back projections of light rays involving a refractive plane for the perspective camera model by explicitly modeling refractive distortion as a function of depth. Furthermore, for cameras with an inertial measurement unit (IMU), we show that a linear solution to the relative pose and a closed-form solution to the absolute pose can be derived with known camera vertical directions. We incorporate our formulations with the general structure from motion framework followed by the patch-based multiview stereo algorithm to obtain a 3D reconstruction of the scene. We show through experiments that the explicit modeling of depth-dependent refractive distortion physically leads to more accurate scene reconstructions.
Finding rows of people in group images

People are among the most popular subjects in photography, and in many social settings, images of groups of people are captured. People often arrange themselves in a very structured manner in these group images. For example, taller peoplemight stand in a row behind smaller people. This structure is often exploited in captions that sequentially label the individuals in each row. We present an algorithm for automatically finding rows of people in group images. A graph is defined for the image, where each face is a vertex. Energy terms are learned from a training set of images. A minimum cut on this graph defines the rows of people in the image. On our test set, the algorithm achieves perfect results with 67.5% of the images. Detecting rows of people is useful for a number of applications.
Pose-robust face recognition using geometry assisted probabilistic modeling

Researchers have been working on human face recognition for decades. Face recognition is hard due to different types of variations in face images, such as pose, illumination and expression, among which pose variation is the hardest one to deal with. To improve face recognition under pose variation, this paper presents a geometry assisted probabilistic approach. We approximate a human head with a 3D ellipsoid model, so that any face image is a 2D projection of such a 3D ellipsoid at a certain pose. In this approach, both training and test images are back projected to the surface of the 3D ellipsoid, according to their estimated poses, to form the texture maps. Thus the recognition can be conducted by comparing the texture maps instead of the original images, as done in traditional face recognition. In addition, we represent the texture map as an array of local patches, which enables us to train a probabilistic model for comparing corresponding patches. By conducting experiments on the CMU PIE database, we show that the proposed algorithm provides better performance than the existing algorithms.
Retrieval of sketches based on spatial relation between strokes

Proposes a method to retrieve sketches stored in the form of multiple strokes, by extracting the shape information for each stroke and by considering the spatial relation between the strokes. The spatial relation between the strokes can be used to determine the stroke correspondence between the query sketch and the database sketch. Moreover, the similarity between the relationship among query strokes and database strokes can be used as part of the matching score. Sketch retrieval is useful in various applications, such as retrieval of free-form hand-drawings, hand-written or printed text, and trademarks. For freeform hand-drawings, such as lecture notes captured digitally from a whiteboard, a user can easily search through a database of hand-drawings by inputting a sketch about what he/she is looking for, without the trouble of describing it using keywords. For hand-written or printed text, such as a protest sign in a video scene, sketch retrieval can be used to recognize the text without the need to first identify each character. For trademarks, such as company logos, sketch retrieval is useful in adding automation to the feature extraction and the retrieval process.
Profile View Lip Reading

In this paper, we introduce profile view (PV) lip reading, a scheme for speaker-dependent isolated word speech recognition. We provide historic motivation for PV from the importance of profile images in facial animation for lip reading, and we present feature extraction schemes for PV as well as for the traditional frontal view (FV) approach. We compare lip reading results for PV and FV, which demonstrate a significant improvement for PV over FV. We show improvement in speech recognition with the integration of audio and visual features. We also found it advantageous to process the visual features over a longer duration than the duration marked by the endpoints of the speech utterance.
Hierarchical object groups for scene classification

The hierarchical structures that exist in natural scenes have been utilized for many tasks in computer vision. The basic idea is that instead of using strictly low level features it is possible to combine them into higher level hierarchical structures. These higher level structures provide a more specific feature and can thus lead to better results in classification or detection. Although most previous work has focused on hierarchical combinations of low level features, hierarchical structures exist on higher levels as well. In this work we attempt to automatically discover these higher level structures by finding meaningful object groups using the Minimum Description Length (MDL) principle. We then use these structures for scene classification and show that we can achieve a higher accuracy rate using them.
Structures for anticausal inverses and application in multirate filter banks

Anticausal or time-reversed inversion of digital filters has gained importance in the implementation of digital filter banks. Anticausal inversion has, in the past, been shown to be possible by using block processing with appropriate state initialization. With (A, B, C, D) denoting the state-space description of a structure implementing a filter G(z), the anticausal inverse can be essentially regarded as a filter structure having an inverted state-space description, which we denote as (A/spl circ/, B/spl circ/, C/spl circ/, D/spl circ/). It is usually not efficient to implement the state-space equations given by (A/spl circ/, B/spl circ/, C/spl circ/, D/spl circ/) directly because of excessive multiplier count. Rather, one seeks to find an efficient structure having the inverse description (A/spl circ/, B/spl circ/, C/spl circ/, D/spl circ/). While this can be done by inspection in simple cases such as the direct-form structure, systematic procedures for other important structures have yet to be developed. We derive anticausal inverse structures corresponding to several standard IIR filter structures such as the direct-form, cascade-form, coupled-form, and the entire family of IIR lattice structures including the tapped cascaded lattice. We introduce the notion of a causal dual, which we find convenient in the derivations. We show that the limit-cycle free property of the original structure is inherited by the causal dual in some but not all cases.
Principal component analysis for facial animation

This paper presents a technique for animating a three-dimensional face model through the application of principal component analysis (PCA). Using PCA has several advantages over traditional approaches to facial animation because it reduces the number of parameters needed to describe a face and confines the facial motion to a valid space to prevent unnatural contortions. First, real data is optically captured in real time from a human subject using infrared cameras and reflective trackers. This data is analyzed to find a mean face and a set of eigenvectors and eigenvalues that are used to perturb the mean face within the range described by the captured data. The result is a set of vectors that can be linearly combined and interpolated to represent different facial expressions and animations. We also show that it is possible to map the eigenvectors of one face onto another face or to change the eigenvectors to describe new motion.
Wide area video surveillance with spatial-temporal constraints

We propose a novel method for the problem of target retrieval for wide area video surveillance (WAVS). Different from previous methods, our algorithm does not require environment model construction nor the learning or usage of any transition probabilities. Furthermore, our method can be applied to an arbitrarily large area for surveillance. The essence of our method lies in an optimization problem formulation using only the shortest driving distance between pairs of cameras, and the provision of an algorithm that finds a global optimum to the optimization. Experiments show that our method can achieve 99% precision and > 80% recall on an input of thousands of frames, which far exceeds the performance of the baseline, whilst taking under 10 seconds to run on a 1.67GHz Intel Centrino laptop. This makes it feasibly deployable on handheld devices in a police car.
Rate-distortion efficiency of subband coding with integer coefficient filters

Discusses efficient image compression algorithms that are suitable for low-complexity implementation in deep space probes. Subband coding or equivalent wavelet transforms provide some advantages over traditional block transforms in terms of rate distortion performance under the mean square-error criterion and according to subjective visual evaluation. Furthermore, the possibility of choosing a desired resolution, independently in the time or frequency domain, is important to preserve the specific features that are most relevant for substantiating scientific findings. A low-complexity version of a JPEG-like algorithm based on the integer cosine transform (ICT), is being implemented in software on the Galileo spacecraft. The most promising candidate for improving the current ICT-based algorithm is a subband coding method that uses quadrature mirror filters (QMF) with lattice structure and integer coefficients. Lattice QMF filters are paraunitary perfect reconstruction (PR) filters. This provides a simple method to obtain PR filters with integer coefficients, without the need for any additional constraint to be satisfied for guaranteeing the PR property. Similarly, orthonormal wavelet transforms can be easily implemented by de signing a two-channel paraunitary QMF bank and then using a tree structure. This means that the wavelet orthonormality properties can be retained even when the coefficients are quantized or subject to certain constraints.<<ETX>>
Multimedia processing for building immersive environments

User study shows that when people communicate face-to-face, they utilize physical proximity, eye contact and other behavior to optimize the overall level of intimacy. Traditional videoconferencing lacks such a sense of immersion and therefore is not as effective as face-to-face communication. More effective communication can be achieved by immersing people in a virtual environment via computer graphics and image processing. We present multimedia processing techniques that are used to build these immersive environments.
Probabilistic relevance feedback with binary semantic feature vectors [image retrieval applications]

For information retrieval, relevance feedback is an important technique. This paper proposes a relevance feedback technique which is based on a probabilistic framework. The binary feature vectors in our experiment are high-level semantic features of trademark logo images, each feature representing the presence or absence of a certain shape or object. The images were labeled by human experts of the trademark office. We compared our probabilistic method with several existing methods such as MARS, MindReader, and one-class SVM. Our method outperformed the others.
3D-Based Reasoning with Blocks, Support, and Stability

3D volumetric reasoning is important for truly understanding a scene. Humans are able to both segment each object in an image, and perceive a rich 3D interpretation of the scene, e.g., the space an object occupies, which objects support other objects, and which objects would, if moved, cause other objects to fall. We propose a new approach for parsing RGB-D images using 3D block units for volumetric reasoning. The algorithm fits image segments with 3D blocks, and iteratively evaluates the scene based on block interaction properties. We produce a 3D representation of the scene based on jointly optimizing over segmentations, block fitting, supporting relations, and object stability. Our algorithm incorporates the intuition that a good 3D representation of the scene is the one that fits the data well, and is a stable, self-supporting (i.e., one that does not topple) arrangement of objects. We experiment on several datasets including controlled and real indoor scenarios. Results show that our stability-reasoning framework improves RGB-D segmentation and scene volumetric representation.
Incorporating Cloud Distribution in Sky Representation

Most sky models only describe the cloudiness of the overall sky by a single category or parameter such as sky index, which does not account for the distribution of the clouds across the sky. To capture variable cloudiness, we extend the concept of sky index to a random field indicating the level of cloudiness of each sky pixel in our proposed sky representation based on the Igawa sky model. We formulate the problem of solving the sky index of every sky pixel as a labeling problem, where an approximate solution can be efficiently found. Experimental results show that our proposed sky model has better expressiveness, stability with respect to variation in camera parameters, and geo-location estimation in outdoor images compared to the uniform sky index model. Potential applications of our proposed sky model include sky image rendering, where sky images can be generated with an arbitrary cloud distribution at any time and any location, previously impossible with traditional sky models.
Learning Patch Dependencies for Improved Pose Mismatched Face Verification

Most pose robust face verification algorithms, which employ 2D appearance, rely heavily on statistics gathered from offline databases containing ample facial appearance variation across many views. Due to the high dimensionality of the face images being employed, the validity of the assumptions employed in obtaining these statistics are essential for good performance. In this paper we assess three common approaches in 2D appearance pose mismatched face recognition literature. In our experiments we demonstrate where these approaches work and fail. As a result of this analysis, we additionally propose a new algorithm that attempts to learn the statistical dependency between gallery patches (i.e. local regions of pixels) and the whole appearance of the probe image. We demonstrate improved performance over a number of leading 2D appearance face recognition algorithms.
Unsupervised Learning of Hierarchical Semantics of Objects (hSOs)

A successful representation of objects in the literature is as a collection of patches, or parts, with a certain appearance and position. The relative locations of the different parts of an object are constrained by the geometry of the object. Going beyond the patches on a single object, consider a collection of images of a particular class of scenes containing multiple (recurring) objects. The parts belonging to different objects are not constrained by such a geometry. However the objects, arguably due to their semantic relationships, themselves demonstrate a pattern in their relative locations, which also propagates to their parts. Analyzing the interactions between the parts across the collection of images would reflect these patterns, and the parts can be grouped accordingly. These groupings are typically hierarchical. We introduce hSO: Hierarchical Semantics of Objects, which is learnt from a collection of images of a particular scene and captures this hierarchical grouping. We propose an approach for the unsupervised learning of the hSO. The hSO simply holds objects, as clusters of patches, at its nodes, but it goes much beyond that and also captures interactions between the objects through its structure. In addition to providing the semantic layout of the scene, learnt hSOs can have several useful applications such as providing context for enhanced object detection and compact scene representation for scene category classification.
Learning patch correspondences for improved viewpoint invariant face recognition

Variation due to viewpoint is one of the key challenges that stand in the way of a complete solution to the face recognition problem. It is easy to note that local regions of the face change differently in appearance as the viewpoint varies. Recently, patch-based approaches, such as those of Kanade and Yamada, have taken advantage of this effect resulting in improved viewpoint invariant face recognition. In this paper we propose a data-driven extension to their approach, in which we not only model how a face patch varies in appearance, but also how it deforms spatially as the viewpoint varies. We propose a novel alignment strategy which we refer to as ldquostack flowrdquo that discovers viewpoint induced spatial deformities undergone by a face at the patch level. One can then view the spatial deformation of a patch as the correspondence of that patch between two viewpoints. We present improved identification and verification results to demonstrate the utility of our technique.
Opportunistic Retransmission in WLANs

This paper presents an efficient opportunistic retransmission protocol (PRO, Protocol for Retransmitting Opportunistically) to improve the performance of IEEE 802.11 WLANs. PRO is a link-layer protocol that allows overhearing nodes to function as relays that retransmit on behalf of a source after they learn about a failed transmission. Relays with better connectivity to the destination have a higher chance of delivering the packet than the source, thereby resulting in a more efficient use of the channel. PRO has four main features. First, channel reciprocity coupled with a runtime calibration process is used to estimate the instantaneous link quality to the destination. Second, a local qualification process filters out poor relays early. Third, a distributed relay selection algorithm chooses the best set of eligible relays among all qualified relays and prioritizes them. Finally, 802.11e Enhanced Distributed Channel Access (EDCA) is leveraged to make sure high-quality relays transmit with higher probability. PRO is designed to coexist with legacy 802.11 stations. Our extensive evaluation on both a controlled testbed and in the real world shows that PRO can improve throughput in diverse wireless environments. PRO helps the most when there is significant contention for the ether, under fading, and with user mobility.
Automatic discovery of groups of objects for scene understanding

Objects in scenes interact with each other in complex ways. A key observation is that these interactions manifest themselves as predictable visual patterns in the image. Discovering and detecting these structured patterns is an important step towards deeper scene understanding. It goes beyond using either individual objects or the scene as a whole as the semantic unit. In this work, we promote "groups of objects". They are high-order composites of objects that demonstrate consistent spatial, scale, and viewpoint interactions with each other. These groups of objects are likely to correspond to a specific layout of the scene. They can thus provide cues for the scene category and can also prime the likely locations of other objects in the scene. It is not feasible to manually generate a list of all possible groupings of objects we find in our visual world. Hence, we propose an algorithm that automatically discovers groups of arbitrary numbers of participating objects from a collection of images labeled with object categories. Our approach builds a 4-dimensional transform space of location, scale and viewpoint, and efficiently identifies all recurring compositions of objects across images. We then model the discovered groups of objects using the deformable parts-based model. Our experiments on a variety of datasets show that using groups of objects can significantly boost the performance of object detection and scene categorization.
Camera free 3-dimensional virtual touch display with multi-user identification

A camera free 3D interactive system for multi-user/multi-touch was proposed. By embedding the optical sensors in display pixels, the flat panel can sense images projected by infrared(IR) light. A light pen which can project T-mark IR image was utilized to determine the 5-axis (x, y, z, θ, φ) information of the user. Additionally, multiple T-marks with sequentially illuminating method can further identify multi-users even the marks are overlapped. Therefore, a 3-dimensional virtual touch display for 5-axis information with multi-user system can be achieved.
Fast bilateral filtering by adapting block size

Direct implementations of bilateral filtering show O(r2) computational complexity per pixel, where r is the filter window radius. Several lower complexity methods have been developed. State-of-the-art low complexity algorithm is an O(1) bilateral filtering, in which computational cost per pixel is nearly constant for large image size. Although the overall computational complexity does not go up with the window radius, it is linearly proportional to the number of quantization levels of bilateral filtering computed per pixel in the algorithm. In this paper, we show that overall runtime depends on two factors, computing time per pixel per level and average number of levels per pixel. We explain a fundamental trade-off between these two factors, which can be controlled by adjusting block size. We establish a model to estimate run time and search for the optimal block size. Using this model, we demonstrate an average speedup of 1.2–26.0x over the previous method for typical bilateral filtering parameters.