nils-herrmann/clean_scirepeval huggingface dataset:
On the application of Bayes networks to semantic understanding of consumer photographs

Belief networks, such as Bayes nets, have emerged as an effective knowledge representation and inference engine in artificial intelligence and expert systems research. Their effectiveness is due to the ability to explicitly integrate domain knowledge in the network structure and to reduce a joint probability distribution to conditionally independence relationships. Current research in content-based image processing and analysis is largely limited to low-level feature extraction and classification. The ability to extract both low-level and semantic features and perform knowledge integration of different types of features would be very useful. We present a general knowledge integration framework that incorporates Bayes networks and has been used in two applications involving semantic understanding of consumer photographs. The first application aims at detecting main photographic subjects in an image and the second aims at selecting the most appealing image in an event. With these diverse examples, we demonstrate that effective inference engines can be built according to specific domain knowledge and available training data to solve inherently uncertain vision problems.
Robust Domain Adaptation on the L1-Grassmannian Manifold

Domain adaptation aims to remedy the loss in classification performance that often occurs due to domain shifts between training and testing datasets. This problem is known as the dataset bias attributed to variations across datasets. Domain adaptation methods on Grassmann manifolds are among the most popular, including Geodesic Subspace Sampling and Geodesic Flow Kernel. Grassmann learning facilitates compact characterization by generating linear subspaces and representing them as points on the manifold. However, Grassmannian construction is based on PCA which is sensitive to outliers. This motivates us to find linear projections that are robust to noise, outliers, and dataset idiosyncrasies. Hence, we combine L1-PCA and Grassmann manifolds to perform robust domain adaptation. We present empirical results to validate improvements and robustness for domain adaptation in object class recognition across datasets.
Manifold based Sparse Representation for robust expression recognition without neutral subtraction

This paper exploits the discriminative power of manifold learning in conjunction with the parsimonious power of sparse signal representation to perform robust facial expression recognition. By utilizing an ℓ1 reconstruction error and a statistical mixture model, both accuracy and tolerance to occlusion improve without the need to perform neutral frame subtraction. Initially facial features are mapped onto a low dimensional manifold using supervised Locality Preserving Projections. Then an ℓ1 optimization is employed to relate surface projections to training exemplars, where reconstruction models on facial regions determine the expression class. Experimental procedures and results are done in accordance with the recently published extended Cohn-Kanade and GEMEP-FERA datasets. Results demonstrate that posed datasets overemphasize the mouth region, while spontaneous datasets rely more on the upper cheek and eye regions. Despite these differences, the proposed method overcomes previous limitations to using sparse methods for facial expression and produces state-of-the-art results on both types of datasets.
Blur identification by residual spectral matching

The estimation of the point spread function (PSF) for blur identification, often a necessary first step in the restoration of real images, method is presented. The PSF estimate is chosen from a collection of candidate PSFs, which may be constructed using a parametric model or from experimental measurements. The PSF estimate is selected to provide the best match between the restoration residual power spectrum and its expected value, derived under the assumption that the candidate PSF is equal to the true PSF. Several distance measures were studied to determine which one provides the best match. The a priori knowledge required is the noise variance and the original image spectrum. The estimation of these statistics is discussed, and the sensitivity of the method to the estimates is examined analytically and by simulations. The method successfully identified blurs in both synthetically and optically blurred images.
Indoor vs outdoor classification of consumer photographs using low-level and semantic features

Scene categorization to indoor vs outdoor may be approached by using low-level features for inferring high-level information about the image. Low-level features such as color and texture have been used extensively in image understanding research, however, they cannot solve the problem completely. We propose the use of a Bayesian network for integrating knowledge from low-level and semantic features for indoor vs outdoor classification of images. Using ground truth data for sky and grass detection, we demonstrate that the classification performance can be significantly improved when semantic features are employed in the classification process.
River flow estimation using video data

An image-based framework for river flow monitoring based on a statistical estimation technique for fluid flow estimation is presented. This approach uses subsequent gray-scale video frames along with a statistical estimation method to extract the optical flow. An average velocity estimate is computed using the velocity vectors of the main motion trend, which is extracted using classification methods. The corresponding real-world surface velocity is computed using velocity-area transformations. The use of only two subsequent video frames and the lack of tracers in the flow are the key features of this technique in order to extract an accurate estimate of the real surface velocity. We compare our real-world surface velocity estimate with traditional current meter measurements, made on the site of Pinios river, Thessaly, Greece using the Q-liner 2 Doppler device.
A computationally efficient approach to indoor/outdoor scene classification

Prior research in scene classification has shown that high-level information can be inferred from low-level image features. Classification rates of roughly 90% have been reported using low-level features to predict indoor scenes vs. outdoor scenes. However, the high classification rates are often achieved by using computationally expensive, high-dimensional feature sets, thus limiting the practical implementation of such systems. We show that a more computationally efficient approach to indoor/outdoor classification can yield classification rates comparable to the best methods reported in the literature. A low complexity, low-dimensional feature set is used in conjunction with a two-stage support vector machine classification scheme to achieve a classification rate of 90.2% on a large database of consumer photographs.
L1-Grassmann manifolds for robust face recognition

Classification on Grassmann manifolds has found application in computer vision problems because it yields improved accuracy and fast computation times. Grassmann manifolds map subspaces to single points, which involves solving for a unit vector representation that is obtained using principal component analysis (PCA). However, PCA may suffer from the presence of outliers due to noise and occlusions often encountered in unconstrained settings. We address this problem by introducing L1-Grassmann manifolds where L1-PCA is used for subspace generation during the mapping process. We utilize a new approach to L1-PCA and demonstrate the effectiveness of L1-Grassmann manifolds for robust face recognition. Results using the Yale face database and the ORL database of faces show that L1-Grassmann manifolds outperform traditional L1-Grassmann manifolds for face recognition and are more robust to noise and occlusions.
Interactive display using depth and RGB sensors for face and gesture control

This paper introduces an interactive display system guided by a human observer's gesture, facial pose, and facial expression. The Kinect depth sensor is used to detect and track an observer's skeletal joints while the RGB camera is used for detailed facial analysis. The display consists of active regions that the observer can manipulate with body gestures and secluded regions that are activated through head pose and facial expression. The observer receives realtime feedback allowing for intuitive navigation of the interface. A storefront interactive display was created and feedback was collected from over one hundred subjects. Promising results demonstrate the potential of the proposed approach for human-computer interaction applications.
Adaptive document image thresholding using foreground and background clustering

Two algorithms for document image thresholding are presented, that are suitable for scanning document images at high-speed. They are designed to operate on a portion of the image while scanning the document, thus, they fit a pipeline architecture and lend themselves to real-time implementation. The first algorithm is based on adaptive thresholding and uses local edge information to switch between global thresholding and adaptive local thresholding determined from the statistics of a local image window. The second thresholding algorithm is based on tracking the foreground and background levels using clustering based on a variant of the K-means algorithm. The two approaches may be used independently or may be combined for improved performance. Results are presented illustrating the algorithms' performance for document and pictorial images.
Low vision assistance using face detection and tracking on android smartphones

This paper presents a low vision assistance system for individuals with blind spots in their visual field. The system identifies prominent faces in the field of view and redisplays them in regions that are visible to the user. As part of the system performance evaluation, we compare various algorithms for face detection and tracking on an Android smartphone, a netbook and a high-performance workstation representative of cloud computing. We examine processing time and energy consumption on all three platforms to determine the tradeoff between processing on a smartphone versus a cloud-desktop after compression and transmission. Our results demonstrate that Viola-Jones face detection along with Lucas-Kanade tracking achieve the best performance and efficiency.
Grassmannian Sparse Representations and Motion Depth Surfaces for 3D Action Recognition

Manifold learning has been effectively used in computer vision applications for dimensionality reduction that improves classification performance and reduces computational load. Grassmann manifolds are well suited for computer vision problems because they promote smooth surfaces where points are represented as subspaces. In this paper we propose Grassmannian Sparse Representations (GSR), a novel subspace learning algorithm that combines the benefits of Grassmann manifolds with sparse representations using least squares loss L1-norm minimization for optimal classification. We further introduce a new descriptor that we term Motion Depth Surface (MDS) and compare its classification performance against the traditional Motion History Image (MHI) descriptor. We demonstrate the effectiveness of GSR on computationally intensive 3D action sequences from the Microsoft Research 3D-Action and 3D-Gesture datasets.
Multi-view action classification using sparse representations on Motion History Images

Multi-view action classification is an important component of real world applications such as automatic surveillance and sports analysis. Motion History Images capture the location and direction of motion in a scene and sparse representations provide a compact representation of high dimensional signals. In this paper, we propose a multi-view action classification algorithm based on sparse representation of spatio-temporal action representations using motion history images. We find that this approach is effective at multi-view action classification and experiments with the i3DPost Multi-view Dataset achieve high classification rates.
On the accuracy of PSF representation in image restoration

Point spread function (PSF) models derived from physical optics provide a more accurate representation of real blurs than simpler models based on geometrical optics. However, the physical PSF models do not always result in a significantly better restoration, due to the coarse sampling of the recording device and insufficiently high signal-to-noise ratio (SNR) levels. Low recording resolutions result in aliasing errors in the PSF and suboptimal restorations. A high-resolution representation of the PSF where aliasing errors are minimized is used to obtain improved restorations. The SNR is the parameter which ultimately limits the restoration quality and determines the need for an accurate PSF model. As a rule of thumb, the geometrical PSF can be used in place of the physical PSF without significant loss in restoration quality when the SNR is less than 30 dB.
Pose estimation using facial feature points and manifold learning

This paper presents robust facial pose estimation techniques based on the underlying low dimensional manifolds embedded in facial images of varying pose. In our approach, facial feature points of training faces are converted to a low dimensional projection space to form a smooth manifold surface. Subsequent faces are automatically detected, facial feature points are extracted and mapped onto the low dimensional projection surface, where regression models robustly estimate pose. The benefit of using facial feature points for manifold learning over raw facial images is demonstrated by a variety of experiments. Linear, nonlinear, unsupervised and supervised methods are considered including Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Locally Linear Embedding (LLE), Isomap, unsupervised Locality Preserving Projections (LPP), and supervised LPP (SLPP).
Fast L1-eigenfaces for robust face recogntion

Face recognition using eigenfaces is a popular technique based on principal component analysis (PCA). However, its performance suffers from the presence of outliers due to occlusions and noise often encountered in unconstrained settings. We address this problem by utilizing L1-eigenfaces for robust face recognition. We introduce an effective approach for L1-eigenfaces based on combining fast computation of L1-PCA with a greedy search technique. Experimental results demonstrate that L1-eigenfaces outperform traditional L2-eigenfaces for face recognition and reconstruction on the Yale face database corrupted with random occlusions.
Automatic image event segmentation and quality screening for albuming applications

In this paper, a system for automatic albuming of consumer photographs is described and its specific core components of event segmentation and screening of low quality images are discussed. A novel event segmentation algorithm was created to automatically cluster pictures into events and sub-events for albuming, based on date/time meta data information as well as color content of the pictures. A new quality-screening is developed based on object quality to detect problematic images due to underexposure, low contrast, and camera defocus or movement. Performance testing of these algorithms was conducted using a database of real consumer photos and showed that these functions provide a useful first-cut album layout for typical rolls of consumer pictures. A first version of the automatic albuming application software was rested through a consumer trial in the United States from August to December 1999.
LGE-KSVD: Flexible Dictionary Learning for Optimized Sparse Representation Classification

Sparse representations have successfully been exploited for the development of highly accurate classifiers. Unfortunately, these classifiers are computationally intensive and subject to the adverse effects of coefficient contamination, where for example variations in pose may affect identity and expression recognition. We propose a technique, called LGE-KSVD, that addresses both problems and attains state-of-the-art results for face and gesture classification problems. Specifically, LGE-KSVD utilizes variants of Linear extension of Graph Embedding to optimize K-SVD, an iterative technique for small yet overcomplete dictionary learning. The dimensionality reduction matrix, sparse representation dictionary, sparse coefficients, and sparsity-based linear classifier are jointly learned through LGE-KSVD. The atom optimization process is redefined to have variable support using graph embedding techniques to produce a more flexible and elegant dictionary learning algorithm. Results are obtained for a wide variety of facial and activity recognition problems to demonstrate the robustness of the proposed method.
A framework for object class recognition with no visual examples

Traditional approaches in object class recognition utilize a large number of labeled visual examples in order to train classifiers to recognize the category of an object in a test image. However, the need for a large number of training data makes the scalability of this approach problematic. In this paper, we explore the recently proposed paradigm of attribute based category recognition for object category recognition without using any visual examples. This goal is achieved by introducing a textual based attribute representation of an image and using these attributes for object categorization. We propose the Sparse Representations (SRs) framework to achieve training-free and highly scalable attribute prediction. We investigate different approaches in mapping the predicted attributes to object classes using Nearest Neighbors and Support Vector Machines. Experimental results suggest that the use of the SRs framework in conjunction with an appropriate Nearest Neighbors scheme can improve prediction accuracy at a much lower computational cost.
Robust Image Hashing Via Non-Negative Matrix Factorizations

In this paper, we propose the use of non-negative matrix factorization (NMF) for robust image hashing. In particular, we view images as matrices and the goal of hashing as a randomized dimensionality reduction that retains the essence of the original image matrix while preventing against intentional attacks of guessing and forgery. Our work is motivated by the fact that standard-rank reduction techniques such as the QR, and singular value decomposition (SVD), produce low rank bases which do not respect the structure (i.e. non-negativity for images) of the original data. We observe that NMFs have two very desirable properties for secure image hashing applications: 1) The additivity property resulting from the non-negativity constraints results in bases that capture local characteristics of the image, thereby significantly reducing misclassification, and 2) the effect of geometric attacks on images in the spatial domain manifests (approximately) as independent identically distributed noise on NMF vectors, allowing design of detectors that are both computationally simple and at the same time optimal in the sense of minimizing error probabilities. ROC (receiver operating characteristics) analysis over a large image database reveals that the proposed algorithms significantly outperform existing approaches for robust image hashing
Automatic rank estimation for practical STAP covariance estimation via an expected likelihood approach

We address the problem of estimation of structured covariance matrices for radar space-time adaptive processing (STAP)1. The knowledge of the interference environment has been exploited in many previous works to accurately estimate a structured disturbance covariance matrix. In particular, it has been shown that employing the rank of clutter subspace, i.e. rank constrained maximum likelihood (RCML) estimation, leads to a practically powerful estimator as well as a closed form solution. While the rank is a very effective constraint, often practical non-idealities make it difficult to be known precisely using physical models. We propose an automatic rank estimation method in STAP via an expected likelihood (EL) approach. We formulate rank estimation as an optimization problem with the expected likelihood criterion and formally prove that the proposed optimization has a unique solution. Through experimental results from a simulation model and KASSPER dataset, we show the RCML estimator with the rank obtained via the EL approach outperforms RCML estimators with the other rank selection methods in the sense of a normalized signal-to-interference and noise ratio (SINR) and the probability of detection.
Meta-classifiers for exploiting feature dependencies in automatic target recognition

Of active interest in automatic target recognition (ATR) is the problem of combining the complementary merits of multiple classifiers. This is inspired by decades of research in the area which has seen a variety of fairly successful feature extraction techniques as well as decision engines being developed. While heuristically based fusion techniques are omnipresent, this paper explores a principled meta-classification strategy that is based on the exploitation of correlation between multiple feature extractors as well as decision engines. We present two learning algorithms respectively based on support vector machines and AdaBoost, which combine soft-outputs of state of the art individual classifiers to yield an overall improvement in recognition rates. Experimental results obtained from benchmark SAR image databases reveal that the proposed meta-classification strategies are not only asymptotically superior but also have better robustness to choice of training over state-of-the art individual classifiers.
Twofold video hashing with automatic synchronization

Video hashing finds a wide array of applications in content authentication, robust retrieval and anti-piracy search. While much of the existing research has focused on extracting robust and secure content descriptors, a significant open challenge still remains: Most existing video hashing methods are fallible to temporal desynchronization. That is, when the query video results by deleting or inserting some frames from the reference video, most existing methods assume the positions of the deleted (or inserted) frames are either perfectly known or reliably estimated. This assumption may be okay under typical transcoding and frame-rate changes but is highly inappropriate in adversarial scenarios such as anti-piracy video search. For example, an illegal uploader will try to bypass the ‘piracy check’ mechanism of YouTube/Dailymotion etc by performing a cleverly designed non-uniform resampling of the video. We present a new solution based on dynamic time warping (DTW), which can implement automatic synchronization and can be used together with existing video hashing methods. The second contribution of this paper is to propose a new robust feature extraction method called flow hashing (FH), based on frame averaging and optical flow descriptors. Finally, a fusion mechanism called distance boosting is proposed to combine the information extracted by DTW and FH. Experiments on real video collections show that such a hash extraction and comparison enables unprecedented robustness under both spatial and temporal attacks.