nils-herrmann/clean_scirepeval huggingface dataset:
Integrating Face and Gait for Human Recognition

This paper introduces a new video based recognition method to recognize non-cooperating individuals at a distance in video, who expose side views to the camera. Information from two biometric sources, side face and gait, is utilized and integrated for recognition. For side face, we construct Enhanced Side Face Image (ESFI), a higher resolution image compared with the image directly obtained from a single video frame, to fuse information of face from multiple video frames. For gait, we use Gait Energy Image (GEI), a spatio-temporal compact representation of gait in video, to characterize human walking properties. The features of face and the features of gait are obtained separately using Principal Component Analysis (PCA) and Multiple Discriminant Analysis (MDA) combined method from ESFI and GEI, respectively. They are then integrated at match score level. Our approach is tested on a database of video sequences corresponding to 46 people. The different fusion methods are compared and analyzed. The experimental results show that (a) Integrated information from side face and gait is effective for human recognition in video; (b) The idea of constructing ESFI from multiple frames is promising for human recognition in video and better face features are extracted from ESFI compared to those from original face images.
On computing a 'fuzzy' focus of expansion for autonomous navigation

The focus of expansion (FOE) is an important concept in dynamic scene analysis, particularly where translational motion is dominant, as in mobile robot applications. In practice, it is difficult to determine the exact location of the FOE from a given set of displacement vectors due to the effects of camera rotation, digitization, and noise. Instead of a single image location, the authors propose to compute a connected region, termed fuzzy FOE, that marks the approximate direction of camera heading. The fuzzy FOE provides numerous clues about 3D scene structure and independent object motion. The main problems of the classic FOE approach are discussed, concentrating on the details of computing the fuzzy FOE for a camera undergoing translation and rotation in 3D space. Results on real outdoor images are presented. Experiments show that even erroneous point matches in the given image displacement vector field can be tolerated, which is a necessity in a fully automated process.<<ETX>>
Robust guidance of a conventionally steered vehicle using destination bearing

An algorithm for camera-based navigation of a conventionally steered vehicle is presented. The destination bearing, obtained by actively panning the camera to fixate on the destination, is the primary measurement. A steering sequence designed to direct a moving vehicle towards a visible destination is generated using an extension of predictive control techniques. A variable sampling interval, based on distance traveled instead of elapsed time, provides robustness to destination range errors. Additional robustness is achieved by incorporating nonlinearities into the steering algorithm, ensuring that the moving vehicle originating from an awkward position and heading will not diverge from the destination. A two-stage extended Kalman filter, which uses the destination bearing angle and known vehicle motion, provides estimates of range. A criterion for shifting the attention of the camera between intermediate destinations is presented. Simulation results demonstrate the effectiveness of the proposed steering algorithm.<<ETX>>
Integrating Face and Gait for Human Recognition at a Distance in Video

This paper introduces a new video-based recognition method to recognize noncooperating individuals at a distance in video who expose side views to the camera. Information from two biometrics sources, side face and gait, is utilized and integrated for recognition. For side face, an enhanced side-face image (ESFI), a higher resolution image compared with the image directly obtained from a single video frame, is constructed, which integrates face information from multiple video frames. For gait, the gait energy image (GEI), a spatio-temporal compact representation of gait in video, is used to characterize human-walking properties. The features of face and gait are obtained separately using the principal component analysis and multiple discriminant analysis combined method from ESFI and GEI, respectively. They are then integrated at the match score level by using different fusion strategies. The approach is tested on a database of video sequences, corresponding to 45 people, which are collected over seven months. The different fusion methods are compared and analyzed. The experimental results show that: 1) the idea of constructing ESFI from multiple frames is promising for human recognition in video, and better face features are extracted from ESFI compared to those from the original side-face images (OSFIs); 2) the synchronization of face and gait is not necessary for face template ESFI and gait template GEI; the synthetic match scores combine information from them; and 3) an integrated information from side face and gait is effective for human recognition in video.
Super-resolution Restoration of Facial Images in Video

Reconstruction-based super-resolution has been widely treated in computer vision. However, super-resolution of facial images has received very little attention. Since different parts of a face may have different motions in normal videos, this paper proposes a new method for enhancing the resolution of low-resolution facial image by handling the facial image non-uniformly. We divide low-resolution face image into different regions based on facial features and estimate motions of each of these regions using different motion models. Our experimental results show we can achieve better results than applying super-resolution on the whole face image uniformly
Bayesian based 3D shape reconstruction from video

In a video sequence with a 3D rigid object moving, changing shapes of the 2D projections provide interrelated spatio-temporal cues for incremental 3D shape reconstruction. This paper describes a probabilistic approach for intelligent view-integration to build 3D model of vehicles from traffic videos collected from an uncalibrated static camera. The proposed Bayesian net framework allows the handling of uncertainties in a systematic manner. The performance is verified with several types of vehicles in different videos.
Recognition of occluded objects: A cluster structure paradigm

Clustering techniques have been used to perform image segmentation, to detect lines and curves in the images and to solve several other problems in pattern recognition and image analysis. In this paper we apply clustering methods to a new problem domain and present a new method based on a cluster-structure paradigm for the recognition of 2-D partially occluded objects. The cluster-structure paradigm entails the application of clustering concepts in a hierarchical manner. The amount of computational effort decreases as the recognition algorithm progresses. As compared to some of the earlier methods, which identify an object based on only one sequence of matched segments, the new technique allows the identification of all parts of the model which match with the apparent object. Also the method is able to tolerate a moderate change in scale and a significant amount of shape distortion arising as a result of segmentation and/or the polygonal approximation of the boundary of the object. The method has been evaluated with respect to a large number of examples where several objects partially occlude one another. A summary of the results is presented.
Facial expression recognition using emotion avatar image

Existing facial expression recognition techniques analyze the spatial and temporal information for every single frame in a human emotion video. On the contrary, we create the Emotion Avatar Image (EAI) as a single good representation for each video or image sequence for emotion recognition. In this paper, we adopt the recently introduced SIFT flow algorithm to register every frame with respect to an Avatar reference face model. Then, an iterative algorithm is used not only to super-resolve the EAI representation for each video and the Avatar reference, but also to improve the recognition performance. Subsequently, we extract the features from EAIs using both Local Binary Pattern (LBP) and Local Phase Quantization (LPQ). Then the results from both texture descriptors are tested on the Facial Expression Recognition and Analysis Challenge (FERA2011) data, GEMEP-FERA dataset. To evaluate this simple yet powerful idea, we train our algorithm only using the given 155 videos of training data from GEMEP-FERA dataset. The result shows that our algorithm eliminates the person-specific information for emotion and performs well on unseen data.
Dynamic-Scene and Motion Analysis Using Passive Sensors - Part II: Displacement-Field and Feature-Based Approaches

For pt.I see ibid., vol.7, no.1, p.45-52 (1992). Dynamic-scene and motion analysis techniques developed to support the DARPA Strategic Computing program's autonomous land vehicle effort are described. Motion-understanding techniques using displacement fields, 3-D motion and structures, and feature correspondence, which show promise, are discussed, and the issues to be addressed and problems to be solved before robust dynamic-scene and motion analysis is achieved are examined.
Face recognition from face profile using dynamic time warping

Most of the current profile recognition algorithms depend on the correct detection of fiducial points and the determination of relationships among these fiducial points. Unfortunately, some features such as concave nose, protruding lips, flat chin, etc., make detection of such points difficult and unreliable. Also, the number and position of fiducial points vary when expression changes even for the same person. In this paper, a curvature-based matching approach is presented, which does not require the extraction of all the fiducial points, but uses information contained in the profile. The scale space filtering is used to smooth the profile and then the curvature of the filtered profile is computed. Using the curvature value, the fiducial points, such as nasion and throat can be reliably extracted using a fast and simple method. Then a dynamic time warping method is applied to match the face profile portion from nasion to throat based on the curvature value. Experiments are performed on two profile face image databases. Recognition rates and conclusion are presented and discussed.
Understanding Discrete Facial Expressions in Video Using an Emotion Avatar Image

Existing video-based facial expression recognition techniques analyze the geometry-based and appearance-based information in every frame as well as explore the temporal relation among frames. On the contrary, we present a new image-based representation and an associated reference image called the emotion avatar image (EAI), and the avatar reference, respectively. This representation leverages the out-of-plane head rotation. It is not only robust to outliers but also provides a method to aggregate dynamic information from expressions with various lengths. The approach to facial expression analysis consists of the following steps: 1) face detection; 2) face registration of video frames with the avatar reference to form the EAI representation; 3) computation of features from EAIs using both local binary patterns and local phase quantization; and 4) the classification of the feature as one of the emotion type by using a linear support vector machine classifier. Our system is tested on the Facial Expression Recognition and Analysis Challenge (FERA2011) data, i.e., the Geneva Multimodal Emotion Portrayal-Facial Expression Recognition and Analysis Challenge (GEMEP-FERA) data set. The experimental results demonstrate that the information captured in an EAI for a facial expression is a very strong cue for emotion inference. Moreover, our method suppresses the person-specific information for emotion and performs well on unseen data.
Delayed reinforcement learning for closed-loop object recognition

Object recognition is a multi-level process requiring a sequence of algorithms at low, intermediate and high levels. Generally, such systems are open loop with no feedback between levels and assuring their robustness is a key challenge in computer vision research. A robust closed-loop system based on "delayed" reinforcement learning is introduced in this paper. The parameters of a multi-level system employed for model-based object recognition are learned. The method improves recognition results over time by using the output at the highest level as feedback for the learning system. It has been experimentally validated by learning the parameters of image segmentation and feature extraction and thereby recognizing 2D objects. The approach systematically controls feedback in a multi-level vision system and provides a potential solution to a long-standing problem in the field of computer vision.
Tracking People by Evolving Social Groups: An Approach with Social Network Perspective

We address the problem of multi-people tracking in unconstrained and semi-crowded scenes. People typically walk in groups that split and merge over time. The evolving or dynamic social group property embodies pedestrians' connections and interactions during walking which we attempt to identify and exploit in this paper. To this end, instead of seeking more robust appearance or motion models to track each person as an isolated moving entity, we pose the multi-people tracking problem as a group-based tracklets association problem using the discovered social groups of track lets as the contextual cues. We formulate tracking the evolution of social groups of tracklets as detecting closely connected communities in a "tracklet interaction network" (TIN) with nodes standing for the tracklets and edges denoting the spatio-temporal co-occurrence correlations measured by the edge weights. We incorporate the detected social groups in the tracklet interaction network to improve multi-people tracking performance. We evaluate our approach against state-of-the-art and show improvements on three real-world datasets.
Improved image super-resolution by Support Vector Regression

Support Vector Machine (SVM) can construct a hyperplane in a high or infinite dimensional space which can be used for classification. Its regression version, Support Vector Regression (SVR) has been used in various image processing tasks. In this paper, we develop an image super-resolution algorithm based on SVR. Experiments demonstrated that our proposed method with limited training samples outperforms some of the state-of-the-art approaches and during the super-resolution process the model learned by SVR is robust to reconstruct edges and fine details in various testing images.
Closed-loop adaptive image segmentation

A closed-loop image segmentation system that incorporates a genetic algorithm to adapt the segmentation process to changes in image characteristics caused by variable environmental conditions is presented. The genetic algorithm efficiently searches the hyperspace of segmentation parameter combinations to determine the parameter set which maximizes the segmentation quality criteria. A summary of the experimental results that demonstrates the ability to perform adaptive image segmentation and to learn from experience using a collection of outdoor color imagery is given.<<ETX>>
Integrating relevance feedback techniques for image retrieval using reinforcement learning

Relevance feedback (RF) is an interactive process which refines the retrievals to a particular query by utilizing the user's feedback on previously retrieved results. Most researchers strive to develop new RF techniques and ignore the advantages of existing ones. In this paper, we propose an image relevance reinforcement learning (IRRL) model for integrating existing RF techniques in a content-based image retrieval system. Various integration schemes are presented and a long-term shared memory is used to exploit the retrieval experience from multiple users. Also, a concept digesting method is proposed to reduce the complexity of storage demand. The experimental results manifest that the integration of multiple RF approaches gives better retrieval performance than using one RF technique alone, and that the sharing of relevance knowledge between multiple query sessions significantly improves the performance. Further, the storage demand is significantly reduced by the concept digesting technique. This shows the scalability of the proposed model with the increasing-size of database.
Object detection via feature synthesis using MDL-based genetic programming

In this paper, we use genetic programming (GP) to synthesize composite operators and composite features from combinations of primitive operations and primitive features for object detection. The motivation for using GP is to overcome the human experts' limitations of focusing only on conventional combinations of primitive image processing operations in the feature synthesis. GP attempts many unconventional combinations that in some cases yield exceptionally good results. To improve the efficiency of GP and prevent its well-known code bloat problem without imposing severe restriction on the GP search, we design a new fitness function based on minimum description length principle to incorporate both the pixel labeling error and the size of a composite operator into the fitness evaluation process. To further improve the efficiency of GP, smart crossover, smart mutation and a public library ideas are incorporated to identify and keep the effective components of composite operators. Our experiments, which are performed on selected training regions of a training image to reduce the training time, show that compared to normal GP, our GP algorithm finds effective composite operators more quickly and the learned composite operators can be applied to the whole training image and other similar testing images. Also, compared to a traditional region-of-interest extraction algorithm, the composite operators learned by GP are more effective and efficient for object detection.
Gabor wavelets for 3-D object recognition

This paper presents a model-based object recognition approach that uses a hierarchical Gabor wavelet representation. The key idea is to use magnitude, phase and frequency measures of Gabor wavelet representation in an innovative flexible matching approach that can provide robust recognition. A Gabor grid a topology-preserving map, efficiently encodes both signal energy and structural information of an object in a sparse multi-resolution representation. The Gabor grid subsamples the Gabor wavelet decomposition of an object model and is deformed to allow the indexed object model match with the image data. Flexible matching between the model and the image minimizes a cost function based on local similarity and geometric distortion of the Gabor grid. Grid erosion and repairing is performed whenever a collapsed grid, due to object occlusion, is detected. The results on infrared imagery are presented. Where objects undergo rotation, translation, scale, occlusion and aspect variations under changing environmental conditions.<<ETX>>
A comparison of techniques for camera selection and handoff in a video network

Video networks are becoming increasingly important for solving many real-world problems, Multiple video sensors, usually cameras, require collaboration when performing various tasks. One of the most basic tasks is the tracking of objects, which requires mechanisms to select a camera for a certain object and hand-off this object from one camera to another so as to accomplish seamless tracking. In this paper, we provide a comprehensive comparison of current and emerging camera selection and hand-off techniques. We consider geometry, statistics, and game theory-based approaches and provide both theoretial and experimental comparison using centralized and distributed computational models. We provide simulation and experimental results using real data for various scenarios of a large number of cameras and objects for in-depth understanding of strengths and weaknesses of these techniques.
Attributes co-occurrence pattern mining for video-based person re-identification

Person re-identification has received considerable attention in the image processing, computer vision and pattern recognition communities because of its huge potential for video-based surveillance applications and the challenges it presents due to illumination, pose and viewpoint changes among non-overlapping cameras. Being different from the widely used low-level descriptors, visual attributes (e.g., hair and shirt color) offer a human understandable way to recognize people. In this paper, a new way to take advantage of them is proposed. First, convolutional neural networks are adopted to detect the attributes. Second, the dependencies among attributes are obtained by mining association rules, and they are used to refine the attributes classification results. Third, metric learning technique is used to transfer the attribute learning task to person re-identification. Finally, the approach is integrated into an appearance-based method for video-based person re-identification. Experimental results on two benchmark datasets indicate that attributes can provide improvements both in accuracy and generalization capabilities.
Reference Face Graph for Face Recognition

Face recognition has been studied extensively; however, real-world face recognition still remains a challenging task. The demand for unconstrained practical face recognition is rising with the explosion of online multimedia such as social networks, and video surveillance footage where face analysis is of significant importance. In this paper, we approach face recognition in the context of graph theory. We recognize an unknown face using an external reference face graph (RFG). An RFG is generated and recognition of a given face is achieved by comparing it to the faces in the constructed RFG. Centrality measures are utilized to identify distinctive faces in the reference face graph. The proposed RFG-based face recognition algorithm is robust to the changes in pose and it is also alignment free. The RFG recognition is used in conjunction with DCT locality sensitive hashing for efficient retrieval to ensure scalability. Experiments are conducted on several publicly available databases and the results show that the proposed approach outperforms the state-of-the-art methods without any preprocessing necessities such as face alignment. Due to the richness in the reference set construction, the proposed method can also handle illumination and expression variation.
Semantic-visual concept relatedness and co-occurrences for image retrieval

This paper introduces a novel approach that allows the retrieval of complex images by integrating visual and semantic concepts. The basic idea consists of three aspects. First, we measure the relatedness of semantic and visual concepts and select the visually separable semantic concepts as elements in the proposed image signature representation. Second, we demonstrate the existence of concept co-occurrence patterns. We propose to uncover those underlying patterns by detecting the communities in a network structure. Third, we leverage the visual and semantic correspondence and the co-occurrence patterns to improve the accuracy and efficiency for image retrieval. We perform experiments on two popular datasets that confirm the effectiveness of our approach.