waashk/dblp huggingface dataset:
combined support vector machines and hidden markov models for modeling facial action temporal dynamics. the analysis of facial expression temporal dynamics is of great importance for many real-world applications. being able to automatically analyse facial muscle actions (action units, aus) in terms of recognising their neutral, onset, apex and offset phases would greatly benefit application areas as diverse as medicine, gaming and security. the base system in this paper uses support vector machines (svms) and a set of simple geometrical features derived from automatically detected and tracked facial feature point data to segment a facial action into its temporal phases. we propose here two methods to improve on this base system in terms of classification accuracy. the first technique describes the original time-independent set of features over a period of time using polynomial parametrisation. the second technique replaces the svm with a hybrid svm/hidden markov model (hmm) classifier to model time in the classifier. our results show that both techniques contribute to an improved classification accuracy. modeling the temporal dynamics by the hybrid svm-hmm classifier attained a statistically significant increase of recall and precision by 4.5% and 7.0%, respectively.
is ica significantly better than pca for face recognition?. the standard pca was always used as baseline algorithm to evaluate ica-based face recognition systems in the previous research. in this paper, we examine the two architectures of ica for image representation and find that ica architecture i involves a pca process by vertically centering (pca i), while ica architecture ii involves a whitened pca process by horizontally centering (pca ii). so, it is reasonable to use these two pca versions as baseline algorithms to revaluate the ica-based face recognition systems. the experiments were performed on the feret face database. the experimental results show there is no significant performance differences between ica architecture i (ii) and pca i (ii), although ica architecture ii significantly outperforms the standard pca. it can be concluded that the performance of ica strongly depends on its involved pca process. the pure ica projection has little effect on the performance of face recognition.
an artificial imagination for interactive search. in this paper we take a look at the predominant form of human computer interaction as used in image retrieval, called interactive search, and discuss a new approach called artificial imagination. this approach addresses two of the grand challenges in this field as identified by the research community: reducing the amount of iterations before the user is satisfied and the small sample problem. artificial imagination will deepen the level of interaction with the user by giving the computer the ability to think along by synthesizing ('imagining') example images that ideally match all or parts of the picture the user has in mind. we discuss two methods of how to synthesize new images, of which the evolutionary synthesis approach receives our main focus.
assessing accuracy factors in deformable 2d/3d medical image registration using a statistical pelvis model. deformable 2d-3d medical image registration is anessential technique in computer integrated surgery (cis)to fuse 3d pre-operative data with 2d intra-operativedata. several factors may affect the accuracy of 2d-3dregistration, including the number of 2d views, the anglebetween views, the view angle relative to anatomical objects,the co-registration error between views, the imagenoise, and the image distortion. in this paper, we investigateand assess the relationship between these factors andthe accuracy of 2d-3d registration. we proposed a deformable2d-3d registration method based on a statisticalmodel. we conducted experiments using a hemi-pelvismodel and simulated x-ray images. some discussions areprovided on how to improve the accuracy of 2d-3d registrationbased on our assessment.
a system for hybrid vision- and sound-based interaction with distal and proximal targets on wall-sized, high-resolution tiled displays. when interacting with wall-sized, high-resolution tiled displays, users typically stand or move in front of it rather than sit at fixed locations. using a mouse to interact can be inconvenient in this context, as it must be carried around and often requires a surface to be used. even for devices that work in mid-air, accuracy when trying to hit small or distal targets becomes an issue. ideally, the user should not need devices to interact with applications on the display wall. we have developed a hybrid vision- and sound-based system for device-free interaction with software running on a 7×4 tile 220-inch display wall. the system comprises three components that together enable interaction with both distal and proximal targets: (i) a camera determines the direction in which a user is pointing, allowing distal targets to be selected. the direction is determined using edge detection followed by applying the hough transform. (ii) using four microphones, a user double-snapping his fingers is detected and located, before the selected target is moved to the location of the snap. this is implemented using correlation and multilateration. (iii) 16 cameras detect objects (fingers, hands) in front of the display wall. the 1d positions of detected objects are then used to triangulate object positions, enabling touch-free multi-point interaction with proximal content. the system is used on the display wall in three contexts to (i) move and interact with windows from a traditional desktop interface, (ii) interact with a whiteboard-style application, and (iii) play two games.
object detection in aerial imagery based on enhanced semi-supervised learning. object detection in aerial imagery has been well studied in computer vision for years. however, given the complexity of large variations of the appearance of the object and the background in a typical aerial image, a robust and efficient detection is still considered as an open and challenging problem. in this paper, we present the enhanced semi-supervised learning (esl) framework and apply this framework to revising an object detection methodology we have developed in a previous effort. theoretic analysis and experimental evaluation using the uci machine learning repository clearly indicate the superiority of the esl framework. the performance evaluations of the revised object detection methodology against the original one clearly demonstrate the promise and superiority of this approach.
interactive feedback for video tracking using a hybrid maximum likelihood similarity measure. in this article, we present an object tracking system which allows interactive user feedback to improve the accuracy of the tracking process in real-time video. in addition, we describe the hybrid maximum likelihood similarity, which integrates traditional metrics with the maximum likelihood estimated metric. the hybrid similarity measure is used to improve the dynamic relevance feedback process between the human user and the objects detected by our system.
automatic generation of robot program code: learning from perceptual data. we propose a novel appr oach to programa robot by demonstrating the task multiple number of times in front of a vision system. here we integrate human dexterity with sensory data using computer vision techniques in a single platform. a simultaneous feature detection and tracking framework is used to track various features (finger tips and the wrist joint). a kalman filter does the tracking by predicting the tentative feature location and a hos-based data clustering algorithm extracts the feature. color information of the features are used for establishing correspondences. a fast, efficient and robust algorithm for the vision system thus developed process a binocular video sequence to obtain the trajectories and the orientation information of the end effector. the concept of a trajectory bundle is introduced to avoid singularities and to obtain an optimal path.
multiple cue integrated action detection. we present an action recognition scheme that integrates multiple modality of cues that include shape, motion and depth to recognize human gesture in the video sequences. in the proposed approach we extend classification framework that is commonly used in 2d object recognition to 3d spatio-temporal space for recognizing actions. specifically, a boosting-based classifier is used that learns spatio-temporal features specific to target actions where features are obtained from temporal patterns of shape contour, optical flow and depth changes occuring at local body parts. the individual features exhibit different strength and sensitivity depending on many factors that include action, underlying body parts and background. in the current method, the multiple cues of different modalities are combined optimally by fisher linear discriminant to form a strong feature that preserve strength of individual cues. in the experiment, we apply the integrated action classifier on a set of target actions and evaluate its performance by comparing with single cue-based cases and present qualitative analysis of performance gain.
conformal metrics and true "gradient flows" for curves. we wish to endow the manifold m of smooth curves in ¿¿ with a riemannian metric that allows us to treat continuous morphs (homotopies) between two curves c¿ and c¿ as trajectories with computable lengths which are independent of the parameterization or representation of the two curves (and the curves making up the morph between them). we may then define the distance between the two curves using the trajectory of minimal length (geodesic) between them, assuming such a minimizing trajectory exists. at first we attempt to utilize the metric structure implied rather unanimously by the past twenty years or so of shape optimization literature in computer vision. this metric arises as the unique metric which validates the common references to a wide variety of contour evolution models in the literature as "gradient flows" to various formulated energy functionals. surprisingly, this implied metric yields a pathological and useless notion of distance between curves. in this paper, we show how this metric can be minimally modified using conformal factors the depend upon a curve¿s total arclength. a nice property of these new conformal metrics is that all active contour models that have been called "gradient flows" in the past will constitute true gradient flows with respect to these new metrics under specfic time reparameterizations.
human-computer intelligent interaction: a survey. human-computer interaction (hci) is one of the foremost challenges of our society. new paradigms for interacting with computers are being developed which will define the 21st century and enable the world to communicate and interact effortlessly and intuitively. in this short survey, we explain the major research clusters comprising the state of the art, and indicate promising future research directions.
stereoscopic segmentation. we cast the problem of multiframe stereo reconstruction of a smooth surface as the global region segmentation of a collection of images of the scene. dually, the problem of segmenting multiple calibrated images of an object becomes that of estimating the solid shape that gives rise to such images. we assume that the radiance of the scene results in piecewise homogeneous image statistics. this simplifying assumption covers lambertian scenes with constant albedo as well as fine homogeneous textures, which are known challenges to stereo algorithms based on local correspondence. we pose the segmentation problem within a variational framework, and use fast level set methods to find the optimal solution numerically. our algorithm does not work in the presence of strong photometric features, where traditional reconstruction algorithms do. it enjoys significant robustness to noise under the assumptiong it is designed for.
non-intrusive physiological monitoring for automated stress detection in human-computer interaction. affective computing, one of the frontiers of human-computer interaction studies, seeks to provide computers with the capability to react appropriately to a user's affective states. in order to achieve the required on-line assessment of those affective states, we propose to extract features from physiological signals from the user (blood volume pulse, galvanic skin response, skin temperature and pupil diameter), which can be processed by learning pattern recognition systems to classify the user's affective state. an initial implementation of our proposed system was set up to address the detection of "stress" states in a computer user. a computer-based "paced stroop test" was designed to act as a stimulus to elicit emotional stress in the subject. signal processing techniques were applied to the physiological signals monitored to extract features used by three learning algorithms: naïve bayes, decision tree and support vector machine to classify relaxed vs. stressed states.
a statistical approach to snakes for bimodal and trimodal imagery. in this paper, we describe a new region-based approach to active contours for segmenting images composed of two or three types of regions characterizable by a given statistic. the essential idea is to derive curve evolutions which separate two or more values of a predetermined set of statistics computed over geometrically determined subsets of the image. both global and local image information is used to evolve the active contour. image derivatives, however, are avoided, thereby giving rise to a further degree of noise robustness compared to most edge-based snake algorithms.
pose and gaze estimation in multi-camera networks for non-restrictive hci. multi-camera networks offer potentials for a variety of novel human-centric applications through provisioning of rich visual information. in this paper, face orientation analysis and posture analysis are combined as components of a human-centered interface systemthat allows the user's intentions and region of interest to be estimated without requiring carried or wearable sensors. in pose estimation, image observations at the cameras are first locally reduced to parametrical descriptions, and particle swarm optimization (pso) is then used for optimization of the kinematics chain of the 3d human model. in face analysis, a discrete-time linear dynamical system (lds), based on kinematics of the head, combines the local estimates of the user's gaze angle produced by the cameras and employs spatiotemporal filters to correct any inconsistencies. knowing the intention and the region of interest of the user facilitates further interpretation of human behavior, which is the key to non-restrictive and intuitive human-centered interfaces. applications in assisted living, speaker tracking, and gaming can benefit from such unobtrusive interfaces.
recognizing human actions in videos acquired by uncalibrated moving cameras. most work in action recognition deals with sequences acquired by stationary cameras with fixed viewpoints. due to the camera motion, the trajectories of the body parts contain not only the motion of the performing actor but also the motion of the camera. in addition to the camera motion, different viewpoints of the same action in different environments result in different trajectories, which can not be matched using standard approaches. in order to handle these problems, we propose to use the multi-view geometry between two actions. however, well known epipolar geometry of the static scenes where the cameras are stationary is not suitable for our task. thus, we propose to extend the standard epipolar geometry to the geometry of dynamic scenes where the cameras are moving. we demonstrate the versatility of the proposed geometric approach for recognition of actions in a number of challenging sequences.
peye: toward a visual motion based perceptual interface for mobile devices. we present the architecture and algorithm design of a visual motion based perceptual interface for mobile devices with cameras. in addition to motion vector, we use the term "visual motion" to be any dynamic changes on consecutive image frames. in the lower architectural hierarchy, visual motion events are defined by identifying distinctive motion patterns. in the higher hierarchy, these visual events are used for interacting with user applications. we present an approach to context aware motion vector estimation to better tradeoff between speed and accuracy. it switches among a set of motion estimation algorithms of different speeds and precisions based on system context such as computation load and battery level. for example, when the cpu is heavily loaded or the battery level is low, we switch to a fast but less accurate algorithm, and vice versa. moreover, to obtain more accurate motion vectors, we propose to adapt the search center of fast block matching methods based on previous motion vectors. both quantitative evaluation of algorithms and subjective usability study are conducted. it is demonstrated that the proposed approach is very robust yet efficient.
reinforcement learning for combining relevance feedback techniques. relevance feedback (rf) is an interactive process which refines the retrievals by utilizing user's feedback history. most researchers strive to develop new rf techniques and ignore the advantages of existing ones. in this paper, we propose an image relevance reinforcement learning (irrl) model for integrating existing rf techniques. various integration schemes are presented and a long-term shared memory is used to exploit the retrieval experience from multiple users. also, a concept digesting method is proposed to reduce the complexity of storage demand. the experimental results manifest that the integration of multiple rf approaches gives better retrieval performance than using one rf technique alone, and that the sharing of relevance knowledge between multiple query sessions also provides significant contributions for improvement. further, the storage demand is significantly reduced by the concept digesting technique. this shows the scalability of the proposed model against a growing-size database.
large lexicon detection of sign language. this paper presents an approach to large lexicon sign recognition that does not require tracking. this overcomes the issues of how to accurately track the hands through self occlusion in unconstrained video, instead opting to take a detection strategy, where patterns of motion are identified. it is demonstrated that detection can be achieved with only minor loss of accuracy compared to a perfectly tracked sequence using coloured gloves. the approach uses two levels of classification. in the first, a set of viseme classifiers detects the presence of sub-sign units of activity. the second level then assembles visemes into word level sign using markov chains. the system is able to cope with a large lexicon and is more expandable than traditional word level approaches. using as few as 5 training examples the proposed system has classification rates as high as 74.3% on a randomly selected 164 sign vocabulary performing at a comparable level to other tracking based systems.
real-time automatic kinematic model building for optical motion capture using a markov random field. we present a completely autonomous algorithm for the real-time creation of a moving subject's kinematic model from optical motion capture data and with no a priori information. our approach solves marker tracking, the building of the kinematic model, and the tracking of the body simultaneously. the novelty lies in doing so through a unifying markov random field framework, which allows the kinematic model to be built incrementally and in real-time. we validate the potential of this method through experiments in which the system is able to accurately track the movement of the human body without an a priori model, as well as through experiments on synthetic data.
catadioptric camera calibration using geometric invariants. central catadioptric cameras are imaging devices thatuse mirrors to enhance the field of view while preservinga single effective viewpoint. in this paper, we propose anovel method for the calibration of central catadioptriccameras using geometric invariants. lines in space areprojected into conics in the catadioptric image plane aswell as spheres in space. we proved that the projection ofa line can provide three invariants whereas the projectionof a sphere can provide two. from these invariants,constraint equations for the intrinsic parameters ofcatadioptric camera are derived. therefore, there are twovariants of this novel method. the first one uses theprojections of lines and the second one uses theprojections of spheres. in general, the projections of twolines or three spheres are sufficient to achieve thecatadioptric camera calibration. one importantobservation in this paper is that the method based on theprojections of spheres is more robust and has higheraccuracy than that using the projections of lines. theperformances of our method are demonstrated by theresults of simulations and experiments with real images.
detecting salient motion by accumulating directionally-consistent flow. motion detection can play an important role in many vision tasks. yet image motion can arise from "uninteresting" events as well as interesting ones. in this paper, salient motion is defined as motion that is likely to result from a typical surveillance target (e.g., a person or vehicle traveling with a sense of direction through a scene) as opposed to other distracting motions (e.g., the scintillation of specularities on water, the oscillation of vegetation in the wind).we propose an algorithm for detecting this salient motion that is based on intermediate-stage vision integration of optical flow. empirical results are presented that illustrate the applicability of the proposed methods to real-world video. unlike many motion detection schemes, no knowledge about expected object size or shape is necessary for rejecting the distracting motion.
linear approaches to camera calibration from sphere images or active intrinsic calibration using vanishing points. spherical objects and vanish points are often used for camera calibration. an occluding contour of a sphere is projected to a conic in the perspective image, and using a moving active camera, the trajectory of a vanishing point in the perspective images is also a conic when the camera is rotated about a fixed 3d axis whereas the translation of the camera is arbitrary. in fact, the problems of camera calibration using conics from spheres or vanishing points can be described by same mathematic representations. two linear approaches to the problems are proposed in this paper: one based on the geometric interpretation of the relation between image conics and the image of the absolute conic, and the other using the special structure of the problems in algebra. only three such conics are needed for the two linear approaches, and the minimum number for previous nonlinear optimization methods is also three. all five intrinsic parameters are recovered linearly without making assumptions, such as, zero-skew or unitary aspect ratio which are often used in previous methods. the two linear algorithms have been tested in extensive experiments with respect to noise sensitivity and also made comparisons with recent calibration techniques.