nils-herrmann/clean_scirepeval huggingface dataset:
Is alice chasing or being chased?: Determining subject and object of activities in videos

Recent progress in video description has shown promising results by combining object/action recognition and natural language processing techniques. However, even the most simplest form of the generated sentence, the SVO triplet (Subject/Verb/Object), can be misleading for its lack of role relationship analysis. When the system detects keywords "person", "baby" and "feed", we do not want the system to generate "a person feeding a baby" when the actual screen is a scene where the baby is trying to share the food. In this paper, we explore role relationships between objects/persons and their usage in generating a more meaningful video description. More specifically, we confine ourselves on the following problem: identifying subject and object roles in two-person activities. We argue that the subject and object roles have consistent properties across different activities. To that end, we cast this problem as a domain adaptation problem. A novel Youtube SVO dataset is proposed for evaluating methods developed for this problem. The performance of the proposed method is compared against several baseline methods.
Segmentation of non-stationary signals with applications

Nonstationary signals are partitioned into near-stationary segments using a modified Appel and Brandt algorithm. The modification requires two spectral distance measures to be used to produce an algorithm which is insensitive to changes in signal energy level which are irrelevant in this application. Performance on real and simulated data is presented. Segmentation has been used to provide an estimator of the evolutive spectrum, an application to a noisy communication signal is presented.<<ETX>>
An Early Experience Toward Developing Computer Aided Diagnosis for Gram-Stained Smears Images

Gram stained direct smears test is clinically useful in early identification of infections. Unfortunately, this practice is considered time consuming and labour intensive. Most existing effort in this area is to perform highmagnification analysis of images taken from manually selected areas. In this paper, we address the problem of the automatic selection of candidate areas (or patches) for subsequent high-magnification analysis. Specifically, we explore and study the possibility of selecting good working areas based on low-magnification images, where bacteria are likely to be found when viewed in high-magnification images. To this end, we develop an approach to classify the areas of interest according to the textural information of the image patch. We explore and study the efficacy of traditional textural features such as Histogram of Gradients, Local Binary Patterns, and 2 Dimensional Discrete Cosine Transform. Experiments show that the best variant method is able to select working areas where it is likely to find bacteria in high-powered objective images in a wide range of images.
Pedestrian Tracking Based on Colour and Spatial Information

This paper describes a tracking with appearance modelling system for pedestrians. A cascade of boosted classifiers and Haar-like rectangular features [6, 12] are used for the pedestrian detection. Statistical modelling in the HSV colour space is used for adaptive background modelling and subtraction, where the use of circular statistics for hue is proposed. By using the background model in combination with the detector, the system extracts a feature vector based on colour statistics and the spatial information. Circular [9] and linear statistics are applied on the extracted features to robustly track the pedestrians and other moving objects through the scene. An adaptive appearance model copes with partial or full occlusions and addresses the problem of missing or wrong detections in single frames.
A Bayesian network-based framework with Constraint Satisfaction Problem (CSP) formulations for FPGA system design

In recent years, there has been a growing interest in IP-reuse for SoCs in order to bridge the gap between the silicon capacity and the design productivity. This research work investigates how our proposed methodology can be used to partition and schedule a JPEG encoder IP core onto an FPGA. We will also describe a novel Constraint Satisfaction Problem (CSP) formulations that are used in the proposed framework. At the same time, we will also demonstrate the effectiveness of CSP in the Bayesian Network-based framework.
Model structure selection & training algorithms for an HMM gesture recognition system

Hidden Markov models using the fully-connected, left-right and left-right banded model structures are applied to the problem of alphabetical letter gesture recognition. We examine the effect of training techniques, in particular the Baum-Welch and Viterbi path counting techniques, on each of the model structures. We show that recognition rates improve when moving from a fully-connected model to a left-right model and a left-right banded 'staircase' model with peak recognition rates of 84.8%, 92.31% and 97.31% respectively. The left-right banded model in conjunction with the Viterbi path counting present the best performance. Direct calculation of model parameters from analysis of the physical system was also tested, yielding a peak recognition rate of 92%, but the simplicity and efficiency of this approach is of interest.
Clustering on Grassmann manifolds via kernel embedding with application to action analysis

With the aim of improving the clustering of data (such as image sequences) lying on Grassmann manifolds, we propose to embed the manifolds into Reproducing Kernel Hilbert Spaces. To this end, we define a measure of cluster distortion and embed the manifolds such that the distortion is minimised. We show that the optimal solution is a generalised eigenvalue problem that can be solved very efficiently. Experiments on several clustering tasks (including human action clustering) show that in comparison to the recent intrinsic Grassmann k-means algorithm, the proposed approach obtains notable improvements in clustering accuracy, while also being several orders of magnitude faster.
Tracking with Multiple Cameras for Video Surveillance

The large shape variability and partial occlusions challenge most object detection and tracking methods for non- rigid targets such as pedestrians. Single camera tracking is limited in the scope of its applications because of the limited field of view (FOV) of a camera. This initiates the need for a multiple-camera system for completely monitoring and tracking a target, especially in the presence of occlusion. When the object is viewed with multiple cameras, there is a fair chance that it is not occluded simultaneously in all the cameras. In this paper, we developed a method for the fusion of tracks obtained from two cameras placed at two different positions. First, the object to be tracked is identified on the basis of shape information measured by MPEG- 7 ART shape descriptor. After this, single camera tracking is performed by the unscented Kalman filter approach and finally the tracks from the two cameras are fused. A sensor network model is proposed to deal with the situations in which the target moves out of the field of view of a camera and reenters after sometime. Experimental results obtained demonstrate the effectiveness of our proposed scheme for tracking objects under occlusion.
Automatic Segmentation and Recognition of Bank Cheque Fields

This paper describes a novel method for automatically segmenting and recognizing the various information fields present on a bank cheque. The uniqueness of our approach lies in the fact that it doesn’t necessitate any prior information and requires minimum human intervention. The extraction of segmented fields is accomplished by means of a connectivity based approach. For the recognition part, we have proposed four innovative features, namely; entropy, energy, aspect ratio and average fuzzy membership values. Though no particular feature is pertinent in itself but a combination of these is used for differentiating between the fields. Finally, a fuzzy neural network is trained to identify the desired fields. The system performance is quite promising on a large dataset of real and synthetic cheque images.
Improved Foreground Detection via Block-Based Classifier Cascade With Probabilistic Decision Integration

Background subtraction is a fundamental low-level processing task in numerous computer vision applications. The vast majority of algorithms process images on a pixel-by-pixel basis, where an independent decision is made for each pixel. A general limitation of such processing is that rich contextual information is not taken into account. We propose a block-based method capable of dealing with noise, illumination variations, and dynamic backgrounds, while still obtaining smooth contours of foreground objects. Specifically, image sequences are analyzed on an overlapping block-by-block basis. A low-dimensional texture descriptor obtained from each block is passed through an adaptive classifier cascade, where each stage handles a distinct problem. A probabilistic foreground mask generation approach then exploits block overlaps to integrate interim block-level decisions into final pixel-level foreground segmentation. Unlike many pixel-based methods, ad-hoc postprocessing of foreground masks is not required. Experiments on the difficult Wallflower and I2R datasets show that the proposed approach obtains on average better results (both qualitatively and quantitatively) than several prominent methods. We furthermore propose the use of tracking performance as an unbiased approach for assessing the practical usefulness of foreground segmentation methods, and show that the proposed approach leads to considerable improvements in tracking accuracy on the CAVIAR dataset.
How do you develop a face detector for the unconstrained environment?

With the growing of available large datasets for evaluation, face detection in recent literature has progressed rapidly. However, little research has been dedicated to develop a face detector robust to all possible variations. To address this problem, novel unconstrained datasets containing faces with more challenging variations are proposed. We notice that some recent face detectors have not been evaluated against recent unconstrained datasets. In this work, we analyse recent state-of-the-art face detectors, Headhunter and Normalized Pixel Difference (NPD), under the unconstrained conditions. We perform a detailed evaluation by proposing our evaluation protocol derived from two unconstrained datasets: (1) IARPA Janus Benchmark A (IJB-A) and (2) WIDER FACE. From our study, we find that the multiple model based detector achieves superior detection rates across most of the domains studied and requires more computational power. However, the unique combination of normalised pixel difference, soft cascade and regression trees within a face detector can achieve similar performance with much less computation.
Content-Based Video Retrieval (CBVR) System for CCTV Surveillance Videos

The inherent nature of image and video and its multi-dimension data space makes its processing and interpretation a very complex task, normally requiring considerable processing power. Moreover, understanding the meaning of video content and storing it in a fast searchable and readable form, requires taking advantage of image processing methods, which when running them on a video stream per query, would not be cost effective, and in some cases is quite impossible due to time restrictions. Hence, to speed up the search process, storing video and its extracted meta-data together is desired. The storage model itself is one of the challenges in this context, as based on the current CCTV technology; it is estimated to require a petabyte size data management system. This estimate however, is expected to grow rapidly as current advances in video recording devices are leading to higher resolution sensors, and larger frame size. On the other hand, the increasing demand for object tracking on video streams has invoked the research on Content-Based Image Retrieval (CBIR) and Content-Based Video Retrieval (CBVR). In this paper, we present the design and implementation of a framework and a data model for CCTV surveillance videos on RDBMS which provides the functions of a surveillance monitoring system, with a tagging structure for event detection. On account of some recent results, we believe this is a promising direction for surveillance video search in comparison to the existing solutions.
Hand Gesture Extraction by Active Shape Models

The paper applied active statistical model for hand gesture extraction and recognition. After the hand contours are found out by a real-time segmenting and tracking system, a set of feature points (Landmarks) are marked out automatically and manually along the contour. A set of feature vectors will be normalized and aligned and then trained by Principle Component Analysis (PCA). Mean shape, eigen-values and eigenvectors are computed out and composed of active shape model. When the model parameter is adjusted continually, various shape contours are generated to match the hand edges extracted from the original images. The gesture is finally recognized after well matching.
Matching image sets via adaptive multi convex hull

Traditional nearest points methods use all the samples in an image set to construct a single convex or affine hull model for classification. However, strong artificial features and noisy data may be generated from combinations of training samples when significant intra-class variations and/or noise occur in the image set. Existing multi-model approaches extract local models by clustering each image set individually only once, with fixed clusters used for matching with various image sets. This may not be optimal for discrimination, as undesirable environmental conditions (eg. illumination and pose variations) may result in the two closest clusters representing different characteristics of an object (eg. frontal face being compared to non-frontal face). To address the above problem, we propose a novel approach to enhance nearest points based methods by integrating affine/convex hull classification with an adapted multi-model approach. We first extract multiple local convex hulls from a query image set via maximum margin clustering to diminish the artificial variations and constrain the noise in local convex hulls. We then propose adaptive reference clustering (ARC) to constrain the clustering of each gallery image set by forcing the clusters to have resemblance to the clusters in the query image set. By applying ARC, noisy clusters in the query set can be discarded. Experiments on Honda, MoBo and ETH-80 datasets show that the proposed method outperforms single model approaches and other recent techniques, such as Sparse Approximated Nearest Points, Mutual Subspace Method and Manifold Discriminant Analysis.
An Abandoned Object Detection System Based on Dual Background Segmentation

An abandoned object detection system is presented and evaluated using benchmark datasets. The detection is based on a simple mathematical model and works efficiently at QVGA resolution at which most CCTV cameras operate. The pre-processing involves a dual-time background subtraction algorithm which dynamically updates two sets of background, one after a very short interval (less than half a second) and the other after a relatively longer duration. The framework of the proposed algorithm is based on the Approximate Median model. An algorithm for tracking of abandoned objects even under occlusion is also proposed. Results show that the system is robust to variations in lighting conditions and the number of people in the scene. In addition, the system is simple and computationally less intensive as it avoids the use of expensive filters while achieving better detection results.
The relationship between instantaneous frequency and time-frequency representations

The relationship between instantaneous frequency estimation based on the derivative of the phase of the analytic signal and the first moment of the general time-frequency representation from Cohen's (1989) class is given in the continuous- and discrete-time domains. Application of the standard linear definition of first moment to discrete-time-frequency representations leads to biased instantaneous frequency estimators with high variance. To correct this problem, it is shown that periodic (circular) definitions of moments must be used to account for the periodization of the frequency variable due to sampling.
Effect of initial HMM choices in multiple sequence training for gesture recognition

We present several ways to initialize and train hidden Markov models (HMMs) for gesture recognition. These include using a single initial model for training (re-estimation), multiple random initial models, and initial models directly computed from physical considerations. Each of the initial models is trained on multiple observation sequences using both Baum-Welch and the Viterbi path counting algorithm on three different model structures: fully connected (or ergodic), left-right, and left-right banded. After performing many recognition trials on our video database of 780 letter gestures, results show that a) the simpler the structure is, the less the effect of the initial model, b) the direct computation method for designing the initial model is effective and provides insight into HMM learning, and c) Viterbi path counting performs best overall and depends much less on the initial model than does Baum-Welch training.
Automatic and quantitative evaluation of attribute discovery methods

Many automatic attribute discovery methods have been developed to extract a set of visual attributes from images for various tasks. However, despite good performance in some image classification tasks, it is difficult to evaluate whether these methods discover meaningful attributes and which one is the best to find the attributes for image descriptions. An intuitive way to evaluate this is to manually verify whether consistent identifiable visual concepts exist to distinguish between positive and negative images of an attribute. This manual checking is tedious, labor intensive and expensive and it is very hard to get quantitative comparisons between different methods. In this work, we tackle this problem by proposing an attribute meaningfulness metric, that can perform automatic evaluation on the meaningfulness of attribute sets as well as achieving quantitative comparisons. We apply our proposed metric to recent automatic attribute discovery methods and popular hashing methods on three attribute datasets. A user study is also conducted to validate the effectiveness of the metric. In our evaluation, we gleaned some insights that could be beneficial in developing automatic attribute discovery methods to generate meaningful attributes. To the best of our knowledge, this is the first work to quantitatively measure the semantic content of automatically discovered attributes.
Ensemble of furthest subspace pairs for enhanced image set matching

Recently it has been shown that the performance of image set matching methods can be improved by clustering set samples into smaller and more coherent groups. Typically, set samples are treated independently during clustering, ie., clustering criteria have not been defined to exploit set characteristics. In this paper we introduce a novel approach to image set clustering by considering the similarities between subspaces instead of similarities between samples. We exploit an ensemble learning technique to create an ensemble of subspace pairs. Each pair has the property that its members are located at the furthest distance in the sense of distances between subspaces. Object recognition experiments on the CMU-MoBO and ETH-80 datasets show that the proposed method obtains higher discrimination accuracy in comparison to several benchmark methods as well as the recently proposed Kernel Affine Hull Method.
Classifying and tracking multiple persons for proactive surveillance of mass transport systems

We describe a pedestrian classification and tracking system that is able to track and label multiple people in an outdoor environment such as a railway station. The features selected for appearance modelling are circular colour histograms for the hue and conventional colour histograms for the saturation and value components. We combine blob matching with a particle filter for tracking and augment these algorithms with colour appearance models to track multiple people in the presence of occlusion. In the object classification stage, hierarchical chamfer matching combined with particle filtering is applied to classify commuters in the railway station into several classes. Classes of interest include normal commuters, commuters with backpacks, commuters with suitcases, and mothers with their children.
Building detection by Dempster-Shafer fusion of LIDAR data and multispectral aerial imagery

A method for the classification of land cover in urban areas by the fusion of first and last pulse LIDAR data and multi-spectral images is presented. Apart from buildings, the classes "tree", "grass land", and "bare soil" are also distinguished by a classification method based on the theory of Dempster-Shafer for data fusion. Examples are given for a test site in Germany.
A face biometric benchmarking review and characterisation

In order to advance face recognition research, algorithm performance has to be measured and compared using a range of metrics and operating characteristics. While public challenges such as the NIST-sponsored FERET, FRVT, FRGC, and MBGC are helpful to gauge comparative performance and improvement for a particular scenario, they typically are not sufficient to fully characterise the strengths and weaknesses of the face recognition algorithm, thus researchers need to do additionally benchmarking independently. This paper provides: (1) a detailed review and categorisation of publicly available face biometrics benchmarks; (2) a discussion of metrics and performance factors to consider; (3) a proposal for a meta-face biometric benchmarking regime which suggests guidelines for benchmarking across multiple datasets to more fully characterise and quantify face recognition performance across various operating characteristics; and (4) a sample demonstration which compare the performance of a face recognition algorithm before and after inclusion of a face quality metric.
Combined Learning of Salient Local Descriptors and Distance Metrics for Image Set Face Verification

In contrast to comparing faces via single exemplars, matching sets of face images increases robustness and discrimination performance. Recent image set matching approaches typically measure similarities between subspaces or manifolds, while representing faces in a rigid and holistic manner. Such representations are easily affected by variations in terms of alignment, illumination, pose and expression. While local feature based representations are considerably more robust to such variations, they have received little attention within the image set matching area. We propose a novel image set matching technique, comprised of three aspects: (i) robust descriptors of face regions based on local features, partly inspired by the hierarchy in the human visual system, (ii) use of several subspace and exemplar metrics to compare corresponding face regions, (iii) jointly learning which regions are the most discriminative while finding the optimal mixing weights for combining metrics. Experiments on LFW, PIE and MOBIO face datasets show that the proposed algorithm obtains considerably better performance than several recent state of-the-art techniques, such as Local Principal Angle and the Kernel Affine Hull Method.