nils-herrmann/clean_scirepeval huggingface dataset:
Towards an integrated personalized interactive video environment

One of the most interesting topics in modern multimedia research, as well as one of the most important tasks in modern audiovisual content providing systems is the treatment of content and users at a semantic level. In this framework, mapping user profiles to multimedia content is a challenging and important problem, as the new generation of home television viewers is currently being confronted with a series of technological developments and improvements, targeted towards their expectations from TV broadcasts. This paper presents initial results from our ongoing work in the field of semantic multimedia analysis, retrieval and user profile extraction in the framework of an interactive video environment, within the MELISA project (E. Papaioannou et al., 2004); this project aims at the cross-media broadcasting of sports events featuring interactive advertising and sports-related games over digital television and provides services for personalized presentation of interactive time video content. Initially, it extends on previous work on low level multimedia content, like scene and shot detection, contour extraction and object tracking, descriptor extraction and matching, and semantic document analysis, in the direction of extraction of semantic user preferences. Such preferences can then be utilized towards the personalization of the overall retrieval process and the multimedia content offering to the end-users. To tackle the latter issue, we propose a methodology based mainly on the utilization of a novel mechanism of weights definition, which are combined with the automatically extracted profiling information.
3D cylindrical trace transform based feature extraction for effective human action classification

Human action recognition is currently one of the hottest areas in pattern recognition and machine intelligence. Its applications vary from console and exertion gaming and human computer interaction to automated surveillance and assistive environments. In this paper, we present a novel feature extraction method for action recognition, extending the capabilities of the Trace transform to the 3D domain. We define the notion of a 3D form of the Trace transform on discrete volumes extracted from spatio-temporal image sequences. On a second level, we propose the combination of the novel transform, named 3D Cylindrical Trace Transform, with Selective Spatio-Temporal Interest Points, in a feature extraction scheme called Volumetric Triple Features, which manages to capture the valuable geometrical distribution of interest points in spatio-temporal sequences and to give prominence to their action-discriminant geometrical correlations. The technique provides noise robust, distortion invariant and temporally sensitive features for the classification of human actions. Experiments on different challenging action recognition datasets provided impressive results indicating the efficiency of the proposed transform and of the overall proposed scheme for the specific task.
Adaptive gesture recognition in Human Computer Interaction

An adaptive, invariant to user performance fluctuation or noisy input signal, gesture recognition scheme is presented based on Self Organizing Maps, Markov Models and Levenshtein sequence distance. Multiple modalities, all based on the hand position during gesturing, train different classifiers which are then fused in a weak classifier boosting-like setup by weight assignment to each stream. The adaptability of the proposed approach consists of the incorporation of Self Organizing Maps during training, the exploitation of neighboring relations between states of the Markov models and the modified Levenshtein distance algorithm. The main focus of current work is to tackle intra and inter user variability during gesture performance by adding flexibility to the decoding procedure and allowing the algorithm to perform an optimal trajectory search while the processing speed of both the feature extraction and the recognition process indicate that the proposed architecture is appropriate for real time and large scale lexicon applications.
User Modelling and Adaptive, Natural Interaction for Conflict Resolution

Modern school environments are usually populated with children from diverse ethnic, cultural and social backgrounds, bringing in different social norms and skills, diverse behaviours and often contradicting cooperation strategies. As a result, conflicts are inevitable and should be resolved as quickly and painlessly as possible, making sure that school life and the learning process continue as intended. The Siren serious game aims to educate 10-14 year old students on conflict management and resolution, presenting them with user- and culture-adaptive mini game scenarios, based on popular game genres and taking into account their affective expressivity and in-game behaviour to adjust the intensity of the conflict to better suit their needs and competencies.
Confidence-Based Fusion of Multiple Feature Cues for Facial Expression Recognition

Since facial expressions are a key modality in human communication, the automated analysis of facial images for the estimation of the displayed expression is essential in the design of intuitive and accessible human computer interaction systems. In most existing rule-based expression recognition approaches, analysis is semiautomatic or requires high quality video. In this paper we propose a feature extraction system which combines analysis from multiple channels based on their confidence, to result in better facial feature boundary detection. The facial features are then used for expression estimation. The proposed approach has been implemented as an extension to an existing expression analysis system in the framework of the IST ERMIS project
The Importance of Eye Gaze and Head Pose to Estimating Levels of Attention

This paper explores, from a theoretical and technical point of view, the role of head rotation and eye gaze directionality to the human perception of attention from non-verbal cues. We have annotated two different versions of the same dataset, in order to correlate the above parameters with the degree people have considered people in the dataset pay attention to a hypothetical task they have in front of them. Based on our findings, we investigate the role of eye gaze directionality in relation to head rotations and, based on previous studies, we developed an algorithm for estimating attention levels from head pose, eye gaze and facial feature spatial locations. With the help of our AI system and people's annotation, we have made a first attempt towards quantifying the role of each cue to the overall estimation of attention. One of the important properties of the technical part of this work is that all systems we used were non-intrusive and did not demand any personal training or calibration phase, constituting themselves ideal for Game playing. Knowing the behavioral state of a player can be of vital importance for adapting the game design during interaction, or building personal profiles, leading to appropriate game features aiming at maximizing player satisfaction.
Image and Signal Processing for Networked E-Health Applications

E-health is closely related with networks and telecommunications when dealing with applications of collecting or transferring medical data from distant locations for performing remote medical collaborations and diagnosis. In this book we provide an overview of the fields of image and signal processing for networked and distributed e-health applications and their supporting technologies. The book is structured in 10 chapters, starting the discussion from the lower end, that of acquisition and processing of biosignals and medical images and ending in complex virtual reality systems and techniques providing more intuitive interaction in a networked medical environment. The book also discusses networked clinical decision support systems and corresponding medical standards, WWW-based applications, medical collaborative platforms, wireless networking, and the concepts of ambient intelligence and pervasive computing in electronic healthcare systems.
MPEG-4: one multimedia standard to unite all

A wide variety of multimedia coding and representation standards have emerged during the past years, in the quest to establish a common denominator for both the research community and the related industry to incorporate results and algorithms in commercial products. MPEG-4 aims to improve where past standards proved ineffective or isolated and integrate different modalities to supply different forms of material, suitable for diverse applications and devices. In this paper, we describe the general concepts that MPEG-4 builds upon and how they are related to multi-user interactive environments and gaming.
Fusing Visual and Behavioral Cues for Modeling User Experience in Games

Estimating affective and cognitive states in conditions of rich human-computer interaction, such as in games, is a field of growing academic and commercial interest. Entertainment and serious games can benefit from recent advances in the field as, having access to predictors of the current state of the player (or learner) can provide useful information for feeding adaptation mechanisms that aim to maximize engagement or learning effects. In this paper, we introduce a large data corpus derived from 58 participants that play the popular Super Mario Bros platform game and attempt to create accurate models of player experience for this game genre. Within the view of the current research, features extracted both from player gameplay behavior and game levels, and player visual characteristics have been used as potential indicators of reported affect expressed as pairwise preferences between different game sessions. Using neuroevolutionary preference learning and automatic feature selection, highly accurate models of reported engagement, frustration, and challenge are constructed (model accuracies reach 91%, 92%, and 88% for engagement, frustration, and challenge, respectively). As a step further, the derived player experience models can be used to personalize the game level to desired levels of engagement, frustration, and challenge as game content is mapped to player experience through the behavioral and expressivity patterns of each player.
A non-intrusive method for user focus of attention estimation in front of a computer monitor

In this work, we present a system that estimates a user's focus of attention in front of a computer screen, using a Web camera, based on detection and tracking of the user's head position and eye movements. Utilizing machine learning concepts, the system gives real time feedback on the user's attention, by combining information coming from eye gaze, head pose, and distance from the screen. The system is completely un-intrusive and no special hardware (such as infrared cameras or wearable devices) is needed. Furthermore, it adjusts to every user, not necessitating initial calibration, and can work under real and unconstrained conditions in terms of lighting.
Identifying unintentional falls in action videos using the 3D Cylindrical Trace Transform

Identification of unintentional falls is a critical application in smart assistive environments, especially in the context of elderly care. However, visually discriminating between falls and fall-like, intentional activities is a challenging task. In this paper, we propose the utilization of a novel feature extraction scheme based on the newly formulated 3D Cylindrical Trace Transform, on spatio-temporal interest points, for the task of fall detection. Using this pipeline, we are able to produce features invariant to occlusion, viewpoint, camera placement and other distortions. Experimentation on two publicly available datasets, on a number of different conditions, proved the efficiency of the proposed methodology for the task at hand.
Detecting human behavior emotional cues in Natural Interaction

Current work focuses on the detection of human behavior emotional cues and their incorporation into affect aware Natural Interaction. Techniques for extracting emotional cues based on visual non verbal human behavior are presented. Namely, gesture qualitative expressivity features and head pose and eye gaze estimation are derived from hand and facial movement respectively. Extracted emotional cues are employed in expressive synthesis on virtual agents, based on the analysis of actions performed by human users, in a Human-Virtual Agent Interaction setting and in Assistive Technologies aiming to infer in real time the degree of attention or frustration of children with reading difficulties.
Adaptation of facial feature extraction and rule generation in emotion-analysis systems

The paper addresses the problem of emotion recognition in faces through an intelligent neuro-fuzzy system, where the extraction of facial features follows the MPEG-4 standard and is adapted to particular environmental conditions and specific persons. These features are associated to symbolic fuzzy predicates providing the classification of facial images according to the underlying emotional states. For this classification we use rules extracted from psychological studies and expression databases including extreme expressions such as those illustrated in Ekman's database. The rules are then refined in realistic conditions, taking into account the extracted features. The experimental results, based both in extreme and naturalistic databases developed in the frameworks of IST ERMIS and NoE HUMAINE, illustrate the capability of the developed system to analyse and recognise facial expressions in human computer interaction applications.
Variance-based shape descriptors for determining the level of expertise of tennis players

Exertion games form a vastly expanding field, crossing over to machine learning and user studies, with studies of qualitative traits of actions, such as the player's level of expertise. In this work, we show how simple shape descriptors based on variance features fare on such a demanding task. We formulate two variance-based features and experiment on a demanding sports related dataset, captured with a Kinect sensor, in an action-specific k-NN classification scheme. Results show that simple shape features can produce meaningful results on determining a player's experience level, further encouraging their incorporation in more intricate schemes and real-world applications.
Emotion representation for virtual environments

AbwaclResearch on networked applications that utilize multimodal information about their users' current emotional state a re presently a t the forefront of interest of the computer vision a n d artificial intelligence communities. Human faces may act as visual interfaces that help users feel at home when interacting with a computer because they a re accepted as the most expressive means for communicating and recognizing emotions. Thus, a lifelike human face can enhance interactive applications by providing straightforward feedback to and from the users and stimulating emotional responses f rom them. Thus, virtual environments can employ believable, expressive characters since such features significantly enhance the atmosphere of a virtual world and Communicate messages f a r more vividly than any textual or speech information. In this paper, we present an abstract means of description of facial expressions, by utilizing concepts included in the MPEG-4 standard. Furthermore, we exploit these concepts to synthesize a wide variety of expressions using a reduced representation, suitable for networked and lightweight applications.
Combination of multiple extraction algorithms in the detection of facial features

Automated analysis of facial images for the estimation of the displayed expression is essential in the design of intuitive and accessible human computer interaction systems. In existing rule-based expression recognition approaches, different feature extraction techniques have been tested that allow for the automatic detection of feature points, providing the required input for a rule based expression analysis; each one of these techniques outperforms others under specific constraints. In this paper we propose a feature extraction system which combines analysis from multiple channels based on their confidence, to result in better, error resilient facial feature boundary detection. The proposed approach has been implemented as an extension to an existing expression analysis system in the framework of the 1ST ERMIS project.
MPEG-21 concepts for personalized consumption of heterogeneous multimedia

In the framework of digital TV, viewers are currently being presented with technological developments and a plethora of content sources that create diverse possibilities for both entertainment and profit. Dynamic domains such as sports broadcasting create even higher expectations, forcing broadcasting corporations and content providers to seek new ways of integrating and presenting enhanced content to their customers, while modifying their production cycle as less as possible. In this paper, we present an outline of a system that integrates heterogeneous multimedia content and performs adaptive packaging operations, while preserving the intellectual property rights (IPR) of the respective contributors and taking into account the cumulative profiles of the viewers. To cater for this, the MELISA system makes extensive use of concepts included in the MPEG-21 ¿Multimedia Framework¿ standard, such as Digital Item Adaptation (DIA).
Emotion representation for online gaming

The ability to simulate lifelike interactive characters has many applications in the gaming industry. Human faces may act as visual interfaces that help users feel at home when interacting with a computer because they are accepted as the most expressive means for communicating and recognizing emotions. Thus, a lifelike human face can enhance interactive applications by providing straightforward feedback to and from the users and stimulating emotional responses from them. Thus, the gaming and entertainment industries can benefit from employing believable, expressive characters since such features significantly enhance the atmosphere of a virtual world and communicate messages far more vividly than any textual or speech information. In this paper, we present an abstract means of description of facial expressions, by utilizing concepts included in the MPEG-4 standard. Furthermore, we exploit these concepts to synthesize a wide variety of expressions using a reduced representation, suitable for networked and lightweight applications.
Robust validation of Visual Focus of Attention using adaptive fusion of head and eye gaze patterns

We propose a framework for inferring the focus of attention of a person, utilizing information coming both from head rotation and eye gaze estimation. To this aim, we use fuzzy logic to estimate confidence on the gaze of a person towards a specific point, and results are compared to human annotation. For head pose we propose Bayesian modality fusion of both local and holistic information, while for eye gaze we propose a methodology that calculates eye gaze directionality, removing the influence of head rotation, using a simple camera. For local information, feature positions are used, while holistic information makes use of face region. Holistic information uses Convolutional Neural Networks which have been shown to be immune to small translations and distortions of test data. This is vital for an application in an unpretending environment, where background noise should be expected. The ability of the system to estimate focus of attention towards specific areas, for unknown users, is grounded at the end of the paper.
Emotion synthesis in the MPEG-4 framework

Man-machine interaction (MMI) systems that utilize multimodal information about users' current emotional state are presently at the forefront of interest of the computer vision and artificial intelligence communities. A lifelike human face can enhance interactive applications by providing straightforward feedback to and from the users and stimulating emotional responses from them. In this paper, we present an abstract means of description of facial expressions, by utilizing concepts included in the MPEG-4 standard to synthesize expressions using a reduced representation, suitable for networked and lightweight applications.
Natural Interaction Multimodal Analysis: Expressivity Analysis towards Adaptive and Personalized Interfaces

Intelligent personalized systems often ignore the affective aspectof human behavior and focus more on tactile cues of the useractivity. A complete user modeling, though, should also incorporatecues such as facial expressions, speech prosody and gesture orbody posture expressivity features, in order to dynamically profile the user, fusing all available modalities since these qualitative affective cues contain significant information about the user's on verbal behavior and communication. Towards this direction, this work focuses on automatic extraction of gestural and headexpressivity features and related statistical processing. The perspective of adopting a common formalization of using expressivity features for a multitude of visual, emotional modalities is explored and grounded through an overview of experiments on appropriate corpora and the corresponding analysis.
Towards Interactive Video Browsing

Almost all video players use interaction features similar to those invented for traditional, sequential VCRs. In our current world where video is intensively used in many areas and in many different ways, existing features are not sufficient anymore. Although video abstraction is a means to ease the handling of videos, it lacks rich interactivity. We propose a video browsing tool that provides more flexibility and interactivity to the end-user. It should enable a user to get knowledge quickly about the content of a video and to easily find relevant parts in a video. We will evaluate our current implementation with user studies performed in a usability-lab.
Tier-selection algorithms with multi-class traffic in CDMA hierarchical cellular systems

In this paper, we investigate several tier selection strategies in a hierarchical cellular system where a hotspot microcell service data users demanding high capacity, and a macrocell services low capacity voice users. A new "relative pilot based (RPB)" tier selection scheme is proposed to achieve the goals of balancing traffic load between a macrocell and a microcell, and reducing the interference between a macrocell and a microcell. The simulation results show that the RPB scheme improves both the blocking performance and outage performance with both data and voice users compared to traditional service based tier selection scheme.
Evolutionarily multiobjective PID control for linear brushless DC motors

This paper presents a robust output tracking control design for a linear brushless DC motor with modeling uncertainties. Frequency-domain design specifications directly related to the mixed sensitivity function and control energy consumption are imposed to ensure stability and performance robustness. A generalized two-parameter PID control framework is developed via an evolution algorithm, which searches the available solutions over certain specified domain. The proposed design paradigm is intuitive and practical in the sense that it offers an effective way to implement simple but robust solutions covering a wide range of plant perturbation and, in addition, provides excellent tracking performance without resorting to excessive control. Experimental and numerical studies are discussed.