nils-herrmann/clean_scirepeval huggingface dataset:
Nonnegative Matrix Factorization in Polynomial Feature Space

Plenty of methods have been proposed in order to discover latent variables (features) in data sets. Such approaches include the principal component analysis (PCA), independent component analysis (ICA), factor analysis (FA), etc., to mention only a few. A recently investigated approach to decompose a data set with a given dimensionality into a lower dimensional space is the so-called nonnegative matrix factorization (NMF). Its only requirement is that both decomposition factors are nonnegative. To approximate the original data, the minimization of the NMF objective function is performed in the Euclidean space, where the difference between the original data and the factors can be minimized by employing L 2-norm. In this paper, we propose a generalization of the NMF algorithm by translating the objective function into a Hilbert space (also called feature space) under nonnegativity constraints. With the help of kernel functions, we developed an approach that allows high-order dependencies between the basis images while keeping the nonnegativity constraints on both basis images and coefficients. Two practical applications, namely, facial expression and face recognition, show the potential of the proposed approach.
Appearance based object tracking in stereo sequences

A novel algorithm is proposed, that performs tracking of rigid objects in 3D videos, without knowledge of the camera calibration parameters, by exploiting only visual information obtained from the left and right video channels, namely luminance and disparity information. The proposed algorithm exploits noisy disparity maps that have been extracted by a real-time disparity estimation algorithm. The algorithm employs two appearance-based representation methods for describing the object texture. The first one combines luminance with disparity information and the second one employs Local Steering Kernel (LSK) descriptors.
3-D Head Pose Estimation in Monocular Video Sequences Using Deformable Surfaces and Radial Basis Functions

This paper presents a novel approach for estimating 3-D head pose in single-view video sequences. Following initialization by a face detector, a tracking technique that utilizes a 3-D deformable surface model to approximate the facial image intensity is used to track the face in the video sequence. Head pose estimation is performed by using a feature vector which is a byproduct of the equations that govern the deformation of the surface model used in the tracking. The afore-mentioned vector is used as input in a radial basis function interpolation network in order to estimate the 3-D head pose. The proposed method was applied to IDIAP head pose estimation database. The obtained results show that the method can estimate the head direction vector with very good accuracy.
Automatic 3D defects identification in stereoscopic videos

3DTV and 3D cinema have become quite popular during the last few years. It is now well understood that certain 3D video quality issues may have a negative effect in the 3D viewing experience. In this paper, we propose two novel algorithms that exploit available disparity information, in order to detect two disturbing stereoscopic issues, namely Stereoscopic Window Violations (SWV) and bent window effects. The algorithms' performance is tested on a number of examples. The proposed algorithms can be used for assessing the overall quality of stereoscopic video content or in order to enable fixing the detected issues in a post-production stage.
Visual Lip Activity Detection and Speaker Detection Using Mouth Region Intensities

In this letter, we introduce a novel approach for lip activity detection and speaker detection, using solely visual information. The main idea in this work is to apply signal detection algorithms to a simple and easily extracted feature from the mouth region. We argue that the increased average value and standard deviation of the number of pixels with low intensities that the mouth region of a speaking person demonstrates can be used as visual cues for detecting visual speech. We then proceed in deriving a statistical algorithm that utilizes this fact for the efficient characterization of visual speech and silence in video sequences. Furthermore, we employ the lip activity detection method in order to determine the active speaker(s) in a multi-person environment.
Video shot detection and condensed representation. a review

There is an urgent need to develop techniques that organize video data into more compact forms or extract semantically meaningful information. Such operations can serve as a first step for a number of different data access tasks such as browsing, retrieval, genre classification, and event detection. In this paper, we focus not on the high-level video analysis task themselves but on the common basic techniques that have been developed to facilitate them. These basic tasks are shot boundary detection and condensed video representation
Frontal facial pose recognition using a discriminant splitting feature extraction procedure

Frontal facial pose recognition deals with classifying facial images into two-classes: frontal and non-frontal. Recognition of frontal poses is required as a preprocessing step to face analysis algorithms (e.g. face or facial expression recognition) that can operate only on frontal views. A novel frontal facial pose recognition technique that is based on discriminant image splitting for feature extraction is presented in this paper. Spatially homogeneous and discriminant regions for each facial class are produced. The classical image splitting technique is used in order to determine those regions. Thus, each facial class is characterized by a unique region pattern which consist of homogeneous and discriminant 2-D regions. The mean intensities of these regions are used as features for the classification task. The proposed method has been tested on data from the XM2VTS facial database with very satisfactory results.
Exploiting disparity information in visual object tracking

A novel method is proposed for visual object tracking in stereo videos. The algorithm employs Local Steering Kernel features and 2-dimensional color-disparity histograms for object texture description. The proposed framework requires no information about the intrinsic and extrinsic parameters of the stereo camera system. Therefore, it can be applied on 3D video content captured by commercial stereo cameras, as well as 3D movies and 3D TV programs. Experiments showed that the proposed method is effective in tracking objects under partial occlusion and changes in the object view angle.
Multichannel L filters based on reduced ordering

Nonlinear multichannel signal processing is an emerging research topic with numerous applications. In this paper we use the so-called reduced ordering (R-ordering) principle to introduce a new family of L filters for vector-valued observations. The coefficients of the proposed filters can be deduced so that the filters are optimal with respect to the output mean squared error. Expressions for the unconstrained, unbiased and location invariant optimal filter coefficients are derived. The calculation of moments of the R-ordered vectors that are involved in these expressions is also discussed. Experiments with noisy two-channel vector fields and noisy color images are presented in order to demonstrate the superiority of the proposed filters over other multichannel filters.
Video shot segmentation using fusion of SVD and mutual information features

A new method for detecting shot boundaries in video sequences by fusing features obtained by singular value decomposition (SVD) and mutual information (MI) is proposed. The first method relies on performing singular value decomposition on a matrix created from 3D color histograms of single frames. The method can detect cuts and gradual transitions, such as dissolves, fades and wipes. The second method relies on evaluating mutual information between two consecutive frames. It can detect abrupt cuts, fade-ins and fade-outs with very high accuracy. A combination of features derived from these methods and subsequent processing through a clustering procedure results in very efficient detection of abrupt cuts and gradual transitions, as demonstrated by experiments on the TRECVID2004 video test set containing different types of shots with significant object and camera motion inside the shots.
The use of Audio-Visual Description Profile in 3D video content description

In this paper we propose a way of using the Audio-Visual Description Profile (AVDP) of the MPEG-7 standard for stereo video content description. Our aim is to provide means of using AVDP in such a way that 3D video content can be correctly and consistently described. Since, AVDP semantics do not include ways for dealing with 3D video content, a new semantic framework within AVDP is proposed. Finally, we show some examples of using AVDP to describe the results of semantic analysis algorithms on stereo video content.
Label propagation on data with multiple representations through multi-graph locality preserving projections

In this paper a novel method is introduced for propagating label information on data with multiple representations. The method performs dimensionality reduction of the data by calculating a projection matrix that preserves locality information and a priori pairwise information, in the form of must-link and cannot-link constraints between the various data representations. The final data representations are then fused, in order to perform label propagation. The performance of the proposed method was evaluated on facial images extracted from stereo movies and on the UCF11 action recognition database. Experimental results showed that the proposed method outperforms state of the art methods.
A class of order statistics learning vector quantizers

A novel class of Learning Vector Quantizers (LVQs) based on multivariate order statistics is proposed in order to overcome the drawback that the estimators for obtaining the reference vectors in LVQ do not have robustness either against erroneous choices for the winner vector or against the outliers that may exist in vector-valued observations. The performance of the proposed variants of LVQ is demonstrated by experiments. In the case of marginal median LVQ, its asymptotic properties are derived as well.<<ETX>>
Anthropocentric descriptors and description schemes for multi-view video content

In this paper a new framework for multi-view video content is discussed. The latter framework is based on the MPEG-7 description schemes and is an extension of the Anthropos-7 framework. Furthermore, we propose a new structure that is based on Anthropos-7 and extends the description from single-view to multi-view multimedia content. Moreover, we show that the proposed structure can be used to describe stereo, video plus depth and multi-view video content. The aim of this proposal is to achieve better results in the indexing, filtering and retrieval processes of multi-capturing systems in terms of time complexity.
The i3DPost Multi-View and 3D Human Action/Interaction Database

In this paper a new multi-view/3D human action/interaction database is presented. The database has been created using a convergent eight camera setup to produce high definition multi-view videos, where each video depicts one of eight persons performing one of twelve different human motions. Various types of motions have been recorded, i.e., scenes where one person performs a specific movement, scenes where a person executes different movements in a succession and scenes where two persons interact with each other. Moreover, the subjects have different body sizes, clothing and are of different sex, nationalities, etc.. The multi-view videos have been further processed to produce a 3D mesh at each frame describing the respective 3D human body surface. To increase the applicability of the database, for each person a multi-view video depicting the person performing sequentially the six basic facial expressions separated by the neutral expression has also been recorded. The database is freely available for research purposes.
A technique for fake 3D (2D-to-3D converted) video recognition

In this paper, we propose a technique for the automatic recognition of "fake" stereoscopic videos/movies i.e., videos which result from classic 2D videos through a 2D to 3D conversion process. Essentially, the proposed technique distinguishes between 2D movies converted to 3D and real stereoscopic ones. It is based on the difference in sharpness around foreground objects in a converted stereo frame pair caused from the in-painting step that takes place after the generation of the right frame (rendered view) from the left frame (source view). The two variants of the algorithm, one utilizing a two-class Support Vector Machine and another one that follows a threshold based classification approach, use a sharpness metric evaluated on a stripe created around foreground objects such as human figures. Experimental evaluation of the proposed algorithm, which can serve as 3D quality characterization tool, is conducted on several stereoscopic movies with very promising results.
A model-based facial expression recognition algorithm using Principal Components Analysis

In this paper, we propose a new method for facial expression recognition. We utilize the Candide facial grid and apply Principal Components Analysis (PCA) to find the two eigenvectors of the model vertices. These eigenvectors along with the barycenter of the vertices are used to define a new coordinate system where vertices are mapped. Support Vector Machines (SVMs) are then used for the facial expression classification task. The method is invariant to in-plane translation and rotation as well as scaling of the face and achieves very satisfactory results.
A perceptual hashing algorithm using latent dirichlet allocation

This paper investigates the possibility of extracting latent aspects of a video, using visual information about humans (e.g. actors' faces), in order to develop a fingerprinting (replica detection) framework. We employ a generative probabilistic model, namely Latent Dirichlet Allocation (LDA), so as to capture latent aspects of a video, using facial semantic information derived from the video. We use the bag-of-words concept, (bag-of-faces in our case) in order to ensure exchangeability of the latent variables (e.g. topics). The video topics are modeled as a mixture of distributions of faces in each video. This generative probabilistic model has already been used in the case of text modeling with good results. Experimental results provide evidence that the proposed method performs very efficiently for video fingerprinting.
Fusion of Geometrical and Texture Information for Facial Expression Recognition

A novel method based on geometrical and texture information is proposed for facial expression recognition from video sequences. The discriminant non-negative matrix factorization (DNMF) algorithm is applied at the image of the last frame of the video sequence, corresponding to the greatest intensity of the facial expression, thus extracting the texture information. A support vector machines (SVMs) system is used for the classification of the geometrical information derived from tracking the Candide grid over the video sequence. The geometrical information consists of the differences of the node coordinates between the neutral (first) and the fully expressed facial expression (last) video frame. The fusion of texture and geometrical information obtained is performed using SVMs. The accuracy achieved is 98,7% when recognizing the six basic facial expressions.
2D visual tracking for sports UAV cinematography applications

In this paper, we provide a preliminary study of basic requirements for autonomous UAV cinematography via 2D target tracking. Our contribution is two-fold. First, we develop a mathematical framework so as to determine hardware camera requirements (specifically, focal length), on a representative case study, i.e., orbiting a still or moving target. Second, we examine the on-board software requirements in order to successfully achieve autonomous target following. To this end, we evaluate the performance of state-of-the-art realtime 2D visual trackers in videos captured by commercial drones. Overall, it was found that state-of-the-art 2D visual trackers are dependable and fast enough to be used in drone cinematography, particularly when combined with periodic target re-detection. A proposed variant of the Staple tracker achieved the best balance between real-time performance and tracking accuracy, on a dataset composed of 31 sports videos recorded by commercial drones.
Incremental Training of Multiclass Support Vector Machines

We present a new method for the incremental training of multiclass Support Vector Machines that provides computational efficiency for training problems in the case where the training data collection is sequentially enriched and dynamic adaptation of the classifier is required. An auxiliary function that incorporates some desired characteristics in order to provide an upper bound of the objective function which summarizes the multiclass classification task has been designed and the global minimizer for the enriched dataset is found using a warm start algorithm, since faster convergence is expected when starting from the previous global minimum. Experimental evidence on two data collections verified that our method is faster than retraining the classifier from scratch, while the achieved classification accuracy is maintained at the same level.
On the initialization of the DNMF algorithm

A subspace supervised learning algorithm named discriminant non-negative matrix factorization (DNMF) has been recently proposed for classifying human facial expressions. It decomposes images into a set of basis images and corresponding coefficients. Usually, the algorithm starts with random basis image and coefficient initialization. Then, at each iteration, both basis images and coefficients are updated to minimize the underlying cost function. The algorithm may need several thousands of iterations to obtain cost function minimization. We provide a way to significantly improve the speed of the algorithm convergence by constructing initial basis images that meet the sparseness and orthogonality requirements and approximate the final minimization solution. To experimentally evaluate the new approach, we have applied DNMF using the random and the proposed initialization procedure to recognize six basic facial expressions. While fewer iteration steps are needed with the proposed initialization, the recognition accuracy remains within satisfactory levels
Towards automated post-production and semantic annotation of films

Film post-production nowadays can be, and most often is, done in the digital domain using computers. There are plenty of tools available for 3D CGI effects, 2D image manipulation of individual frames, color correction and grading, audio-visual synchronization and video editing. Every task, however, still requires a considerable amount of human interaction. In this paper, we will provide an overview of the relevant computer vision technologies which could be used to automate the more time intensive parts of these tasks, provided their performance reaches the high quality standards of the film industry, and semantically annotate the films.
3D Human Action Recognition for Multi-view Camera Systems

This paper presents a novel approach for combining optical flow into enhanced 3D motion vector fields for human action recognition. Our approach detects motion of the actors by computing optical flow in video data captured by a multi-view camera setup with an arbitrary number of views. Optical flow is estimated in each view and extended to 3D using 3D reconstructions of the actors and pixel-to-vertex correspondences. The resulting 3D optical flow for each view is combined into a 3D motion vector field by taking the significance of local motion and its reliability into account. 3D Motion Context (3D-MC) and Harmonic Motion Context (HMC) are used to represent the extracted 3D motion vector fields efficiently and in a view-invariant manner, while considering difference in anthropometry of the actors and their movement style variations. The resulting 3D-MC and HMC descriptors are classified into a set of human actions using normalized correlation, taking into account the performing speed variations of different actors. We compare the performance of the 3D-MC and HMC descriptors, and show promising experimental results for the publicly available i3DPost Multi View Human Action Dataset.
Studying the effectiveness of using linear subspace techniques to improve SVM classifiers in facial image classification

In this paper we investigate the potential benefits of combining, within a classification task, a discriminant linear subspace feature extraction technique, namely Discriminant Nonnegative Matrix Factorization (Discriminant NMF or DNMF), with a Support Vector Machine (SVM) classifier. The aim was to investigate whether this combination provides better classification results compared to a template matching method operating on the DNMF space or on the raw data and an SVM classifier operating on the raw data, when applied on the frontal facial pose recognition problem. The latter is a two-class problem (frontal and non-frontal facial images). DNMF is based on a supervised training procedure and works by imposing additional criteria on the NMF objective function that aim at increasing class seperability in the lower dimensionality space. Results on face images extracted from the XM2VTS dataset show that feeding the DNMF subspace data into the SVM is the approach that provides the best results.
Variational Bayesian inference for stereo object tracking

In this paper, we deal with object tracking in stereo video sequences. We introduce a Bayesian framework for utilizing the results of any conventional single channel object tracker, in order to accomplish the refinement of the tracking accuracy in the left/right video channel. In this Bayesian framework, a variational Bayesian algorithm is employed to this end, where a priori information about the object displacement (movement) over time is incorporated by means of a prior distribution. This a priori information is obtained in a pre-processing step, in which the object displacement over time is estimated. Experiments demonstrate the efficiency of the proposed post-processing methodology in terms of tracking accuracy.
Video replica detection utilizing R-trees and frame-based voting

A novel color-based two-step, coarse-to-fine video replica detection system is proposed in this paper. The first step uses an R-tree in order to perform a coarse selection of the database (original) videos that potentially match the query video. A training procedure that utilizes attacked versions of the database videos and aims at achieving robustness to attacks is being used. A frame-based voting procedure is also involved. A refinement step that processes the set of videos returned by the first step in order to select the final matching video (if any) follows. The performance of the system has been evaluated on a database of short videos with good results.
Feature extraction and statistical analysis of videos for cinemetric applications

In this paper, we describe a framework for the extraction of low-level and high level information from movies in order to be used for cinemetric applications. The developed framework analyses the available video content and extracts characteristics related to color, motion, contrast, shot length, tempo, face to frame ratios etc. The extracted information is stored in MPEG 7 AVDP profile format, which is a standard description format that can be imported to related cinemetric applications. We applied the developed framework in a collection of downloaded videos, as well as 3 stereoscopic movies.