waashk/dblp huggingface dataset:
drowsy driver detection through facial movement analysis. the advance of computing technology has provided the means for building intelligent vehicle systems. drowsy driver detection system is one of the potential applications of intelligent vehicle systems. previous approaches to drowsiness detection primarily make preassumptions about the relevant behavior, focusing on blink rate, eye closure, and yawning. here we employ machine learning to datamine actual human behavior during drowsiness episodes. automatic classifiers for 30 facial actions from the facial action coding system were developed using machine learning on a separate database of spontaneous expressions. these facial actions include blinking and yawn motions, as well as a number of other facial movements. in addition, head motion was collected through automatic eye tracking and an accelerometer. these measures were passed to learning-based classifiers such as adaboost and multinomial ridge regression. the system was able to predict sleep and crash episodes during a driving computer game with 96% accuracy within subjects and above 90% accuracy across subjects. this is the highest prediction rate reported to date for detecting real drowsiness. moreover, the analysis revealed new information about human behavior during drowsy driving.
utilization of stereo disparity and optical flow information for human interaction. to attain smooth human interaction, we propose a system which simultaneously utilizes the stereo disparity and optical flow information of real-time stereo gray multiresolution images to recognize objects and gestures. for real-time calculation of the disparity and optical flow information of a stereo image, the system first creates pyramid images by utilizing a gaussian filter. the system then determines the disparity and optical flow of a low density image and extracts regions in front of a certain depth. the three foremost regions are recognized by higher order local auto-correlation features and a linear discriminant analysis. with this process, the system recognizes the face and hand signs of users, which are displayed foremost, and roughly recognizes novements within the region in real-time. with this framework, the system can discriminate the face of a user, can monitor the basic movements of the user, can smoothly learn a presented object by users, and can communicate with users from hand signs learned in advance.
multiclass spectral clustering. we propose a principled account on multiclass spectral clustering. given a discrete clustering formulation, we first solve a relaxed continuous optimization problem by eigen-decomposition. we clarify the role of eigenvectors as a generator of all optimal solutions through orthonormal transforms. we then solve an optimal discretization problem, which seeks a discrete solution closest to the continuous optima. the discretization is efficiently computed in an iterative fashion using singular value decomposition and non-maximum suppression. the resulting discrete solutions are nearly global-optimal. our method is robust to random initialization and converges faster than other clustering methods. experiments on real image segmentation are reported.
nonparametric modelling and tracking with -gng. in this paper we address the correspondence problem, with its application to nonrigid tracking and unsupervised modelling, as a nonparametric, active-linking topology learning problem. unlike existing soft competitive learning methods, active growing neural gas (a-gng) has both global and local properties which allows part of the network to reconfigure while tracking. in addition, a-gng uses a number of features (e.g. topographic product, local grey-level and map transformation) so that the topological relations are preserved and nodes correspondences are retained between tracked configurations. experimental results in a sequence of hand gestures and artificial data have shown the superiority of our proposed method over the original gng.
visual motion estimation and prediction: a probabilistic network model for temporal coherence. we develop a theory for the temporal integration of visual motion motivated by psychophysical experiments. the theory proposes that input data are temporally grouped and used to predict and estimate motion flows in the image sequences. our theory is expressed in terms of the bayesian generalization [10] of standard kalman filtering which allows us to solve temporal grouping in conjunction with prediction and estimation. as demonstrated for tracking isolated contours [12], the bayesian formulation is superior to approaches which use data association as a first stage followed by conventional kalman filtering. our computer simulations demonstrate that our theory qualitatively accounts for several psychophysical experiments on motion occlusion and motion outliers [15], [20].
signfinder: using color to detect, localize and identify informational signs. we describe an approach to detecting, locating and normalizing road signs. the approach will apply provided: (i) the signs have stereotypical boundary shapes (i.e. rectangular, or hexagonal ¿ of course, we allow for these shapes to be distorted by projection to unknown viewpoint), (ii) the writingon the sign has one uniform color and the rest of the sign has a second uniform color (we allow for the color of the illuminant to be unknown). we show that the approach works even under significant illuminant color changes, viewpoint direction, shadowing, and occlusion. this work is part of a project intended to help people who are blind, or whose sight is impaired.
a unifying approach to hard and probabilistic clustering. we derive the clustering problem from first principles showing that the goal of achieving a probabilistic, or "hard", multi class clustering result is equivalent to the algebraic problem of a completely positive factorization under a doubly stochastic constraint. we show that spectral clustering, normalized cuts, kernel k-means and the various normalizations of the associated affinity matrix are particular instances and approximations of this general principle. we propose an efficient algorithm for achieving a completely positive factorization and extend the basic clustering scheme to situations where partial label information is available.
real time body pose tracking in an immersive training environment. we describe a visual communication application for a dark, theaterlike interactive virtual simulation training environment. our system visually estimates and tracks the body position, orientation and the arm-pointing direction of the trainee. this system uses a near-ir camera array to capture images of the trainee from different angles in the dim-lighted theater. image features like silhouettes and intermediate silhouette body axis points are then segmented and extracted from image backgrounds. 3d body shape information such as 3d body skeleton points and visual hulls can be reconstructed from these 2d features in multiple calibrated images. we proposed a particle-filtering based method that fits an articulated body model to the observed image features. currently we focus on the arm-pointing gesture of either limb. from the fitted articulated model we can derive the position on the screen the user is pointing to. we use current graphic hardware to accelerate the processing speed so the system is able to work in real-time. the system serves as part of multi-modal user-input device in the interactive simulation.
multi-view subspace constraints on homographies. the motion of a planar surface between two camera views induces a homography. the homography depends on the camera intrinsic and extrinsic parameters, as well as on the 3d plane parameters. while camera parameters vary across different views, the plane geometry remains the same. based on this fact, we derive linear subspace constraints on the relative motion of multiple (\math) planes across multiple views.the paper has three main contributions: (i) we show that the collection of all "relative homographies" of a pair of planes (homologies) across multiple views, spans a 4-dimensional linear subspace. (ii) we show how this constraint can be extended to the case of multiple planes across multiple views. (iii) we suggest two potential application areas which can benefit from these constraints: (a) the accuracy of homography estimation can be improved by enforcing the multi-view subspace constraints. (b) violations of these multi-view constraints can be used as a cue for moving object detection. all the results derived in this paper are true for uncalibrated cameras.
image based regression using boosting method. we present a general algorithm of image based regression that is applicable to many vision problems. the proposed regressor that targets a multiple-output setting is learned using boosting method. we formulate a multiple-output regression problem in such a way that overfitting is decreased and an analytic solution is admitted. because we represent the image via a set of highly redundant haar-like features that can be evaluated very quickly and select relevant features through boosting to absorb the knowledge of the training data, during testing we require no storage of the training data and evaluate the regression function almost in no time. we also propose an efficient training algorithm that breaks the computational bottleneck in the greedy feature selection process. we validate the efficiency of the proposed regressor using three challenging tasks of age estimation, tumor detection, and endocardial wall localization and achieve the best performance with a dramatic speed, e.g., more than 1000 times faster than conventional data-driven techniques such as support vector regressor in the experiment of endo-cardial wall localization.
squaring the circles in panoramas. pictures taken by a rotating camera cover the viewing sphere surrounding the center of rotation. having a set of images registered and blended on the sphere what is left to be done, in order to obtain a flat panorama, is projecting the spherical image onto a picture plane. this step is unfortunately not obvious ¿ the surface of the sphere may not be flattened onto a page without some form of distortion. the objective of this paper is discussing the difficulties and opportunities that are connected to the projection from viewing sphere to image plane. we first explore a number of alternatives to the commonly used linear perspective projection. these are ¿global¿ projections and do not depend on image content. we then show that multiple projections may coexist successfully in the same mosaic: these projections are chosen locally and depend on what is present in the pictures. we show that such multi-view projections can produce more compelling results than the global projections.
progressive surface reconstruction from images using a local prior. this paper introduces a new method for surface reconstruction from multiple calibrated images. the primary contribution of this work is the notion of local prior to combine the flexibility of the carving approach with the accuracy of graph-cut optimization. a progressive refinement scheme is used to recover the topology and reason the visibility of the object. within each voxel, a detailed surface patch is optimally reconstructed using a graph-cut method. the advantage of this technique is its ability to handle complex shape similarly to level sets while enjoying a higher precision. compared to carving techniques, the addressed problem is well-posed, and the produced surface does not suffer from aliasing. in addition, our approach seamlessly handles complete and partial reconstructions: if the scene is only partially visible, the process naturally produces an open surface; otherwise, if the scene is fully visible, it creates a complete shape. these properties are demonstrated on real image sequences.
a general framework for temporal video scene segmentation. videos are composed of many shots caused by different camera operations, e.g., on/off operations and switching between cameras. one important goal in video analysis is to group the shots into temporal scenes, such that all the shots in a single scene are related to a particular physical setting, an on-going action or a theme. in this paper, we present a general framework for temporal scene segmentation for various video types. the proposed method is formulated in a statistical fashion and uses the markov chain monte carlo (mcmc) technique to determine the boundaries between video scenes. in this approach, an arbitrary number of scene boundaries are randomly initialized and automatically updated using two types of updates: diffuse and jumps. the posterior probability on the number of scenes and their boundary locations is computed based on the model priors and the data likelihood. the updates of the model parameters are controlled by the hypothesis ratio test in the mcmc process. the proposed framework has been experimented on two types of videos, home videos and feature films, and accurate results have been obtained.
estimating motion and structure from correspondences of line segments between two perspective images. presents an algorithm for determining 3d motion and structure from correspondences of line segments between two perspective images. to our knowledge, the paper is the first investigation of use of line segments in motion and structure from motion. classical methods use their geometric abstraction, namely straight lines, but then three images are necessary for the motion and structure determination process. we show that two views are in general sufficient when we use line segments. the assumption we use is that two matched line segments contain the projection of a common part of the corresponding line segment in space. indeed this is what we use to match line segments between different views. both synthetic and real data have been used to test the proposed algorithm, and excellent results have been obtained with real data containing a relatively large set of line segments. the results are comparable with those obtained using stereo calibration.
understanding the relationship between the optimization criteria in two-view motion analysis. the three best known criteria in two-view motion analysis are based, respectively, on the distances between points and their corresponding epipolar lines, on the gradient-weighted epipolar errors, and on the distances between points and the reprojections of their reconstructed points. the last one has a better statistical interpretation, but is, however, much slower than the first two. in this paper, we show that the last two criteria are equivalent when the epipoles are at infinity, and differ from each other only a little even when the epipoles are in the image. the first two criteria are equivalent only when the epipoles are at infinity and when the observed object has the same scale in the two images. this suggests that the second criterion is sufficient in practice because of its computational efficiency. the resultis valid for both calibrated and uncalibrated images.
modeling geometric structure and illumination variation of a scene from real images. we present in this paper a system which automatically builds, from real images, a scene model containing both 3d geometric information of the scene structure and its photometric information under various illumination conditions. the geometric structure is recovered from images taken from distinct viewpoints. structure-from-motion and correlation-based stereo techniques are used to match pixels between images of different viewpoints and to reconstruct the scene in 3d space. the photometric property is extracted from images taken under different illumination conditions (orientation, position and intensity of the light sources). this is achieved by computing a low-dimensional linear space of the spatio-illumination volume, and is represented by a set of basis images. the model that has been built can be used to create realistic renderings from different viewpoints and illumination conditions. applications include object recognition, virtual reality and product advertisement.
shape and motion under varying illumination: unifying structure from motion, photometric stereo, and multi-view stereo. this paper presents an algorithm for computing optical flow,shape, motion, lighting, and albedo from an image sequenceof a rigidly-moving lambertian object under distant illumination.the problem is formulated in a manner that subsumesstructure from motion, multi-view stereo, and photometricstereo as special cases. the algorithm utilizes bothspatial and temporal intensity variation as cues: the formerconstrains flow and the latter constrains surface orientation;combining both cues enables dense reconstruction ofboth textured and texture-less surfaces. the algorithm worksby iteratively estimating affine camera parameters, illumination,shape, and albedo in an alternating fashion. results aredemonstrated on videos of hand-held objects moving in frontof a fixed light and camera.
bayesian body localization using mixture of nonlinear shape models. we present a 2dmodel-based approach to localizing human body in images viewed from arbitrary and unknown angles. the central component is a statistical shape representation of the nonrigid and articulated body contours, where a non-linear deformation is decomposed based on the concept of parts. several image cues are combined to relate the body configuration to the observed image, with self-occlusion explicitly treated. to accommodate large viewpoint changes, a mixture of view-dependent models is employed. inference is done by direct sampling of the posterior mixture, using sequential monte carlo (smc) simulation enhanced with annealing and kernel move. the fitting method is independent of the number of mixture components, and does not require the preselection of a "correct" viewpoint. the models were trained on a large number of interactively labeled gait images. preliminary tests demonstrated the feasibility of the proposed approach.
tracking objects using density matching and shape priors. we present a novel method for tracking objects by combiningdensity matching with shape priors. density matchingis a tracking method which operates by maximizing thebhattacharyya similarity measure between the photometricdistribution from an estimated image region and a modelphotometric distribution. such trackers can be expressed aspde-based curve evolutions, which can be implemented usinglevel sets. shape priors can be combined with this level-setimplementation of density matching by representing theshape priors as a series of level sets; a variational approachallows for a natural, parametrization-independentshape term to be derived. experimental results on real imagesequences are shown.
euclidean structure from uncalibrated images using fuzzy domain knowledge: application to facial images synthesis. use of uncalibrated images has found many applications such as image synthesis. however, it is not easy to specify the desired position of the new image in projective or af.ne space. this paper proposes to recover euclidean structure from uncalibrated images using domain knowledge such as distances and angles. the knowledge we have is usually about an object category, but not very precise for the particular object being considered. the variation (fuzziness) is modeled as a gaussian variable. six types of common knowledge are formulated. once we have a euclidean description, the task to specify the desired position in euclidean space becomes trivial. the proposed technique is then applied to synthesis of new facial images. a number of dif.culties existing in image synthesis are identified and solved. for example, we propose to use edge points to deal with occlusion.
facial expression understanding in image sequences using dynamic and active visual information fusion. this paper explores the use of multisensory information fusiontechnique with dynamic bayesian networks (dbns)for modeling and understanding the temporal behaviors offacial expressions in image sequences. our approach tothe facial expression understanding lies in a probabilisticframework by integrating the dbns with the facial actionunits (aus) from psychological view. the dbns provide acoherent and unified hierarchical probabilistic frameworkto represent spatial and temporal information related to facialexpressions, and to actively select the most informativevisual cues from the available information to minimize theambiguity in recognition. the recognition of facial expressionsis accomplished by fusing not only from the currentvisual observations, but also from the previous visual evidences.consequently, the recognition becomes more robustand accurate through modeling the temporal behavior of facialexpressions. experimental results demonstrate that ourapproach is more admissible for facial expression analysisin image sequences.
local gabor binary pattern histogram sequence (lgbphs): a novel non-statistical model for face representation and recognition. for years, researchers in face recognition area have been representing and recognizing faces based on subspace discriminant analysis or statistical learning. nevertheless, these approaches are always suffering from the generalizability problem. this paper proposes a novel non-statistics based face representation approach, local gabor binary pattern histogram sequence (lgbphs), in which training procedure is unnecessary to construct the face model, so that the generalizability problem is naturally avoided. in this approach, a face image is modeled as a "histogram sequence" by concatenating the histograms of all the local regions of all the local gabor magnitude binary pattern maps. for recognition, histogram intersection is used to measure the similarity of different lgbphses and the nearest neighborhood is exploited for final classification. additionally, we have further proposed to assign different weights for each histogram piece when measuring two lgbphses. our experimental results on ar and feret face database show the validity of the proposed approach especially for partially occluded face images, and more impressively, we have achieved the best result on feret face database.
a probabilistic semantic model for image annotation and multi-modal image retrieva. this paper addresses automatic image annotation problem and its application to multi-modal image retrieval. the contribution of our work is three-fold. (1) we propose a probabilistic semantic model in which the visual features and the textual words are connected via a hidden layer which constitutes the semantic concepts to be discovered to explicitly exploit the synergy among the modalities. (2) the association of visual features and textual words is determined in a bayesian framework such that the confidence of the association can be provided. (3) extensive evaluation on a large-scale, visually and semantically diverse image collection crawled from web is reported to evaluate the prototype system based on the model. in the proposed probabilistic model, a hidden concept layer which connects the visual feature and the word layer is discovered by fitting a generative model to the training image and annotation words through an expectation-maximization (em) based iterative learning procedure. the evaluation of the prototype system on 17,000 images and 7,736 automatically extracted annotation words from crawled web pages for multi-modal image retrieval has indicated that the proposed semantic model and the developed bayesian framework are superior to a state-of-the-art peer system in the literature.