# Sensors, Techniques, and Future Trends of Human-Engagement-Enabled Applications: A Review
## 2 Background  
Engagement is often characterised as a variable state which is determined by users' presence, affect, participation, and motion [1]. A popular theory explaining engagement is _flow_, constructed upon the user-system-experience (USE) model [6]. This theory frames engagement as an optimal and enjoyable experience characterised by a tractable challenge, immersion, and immediate feedback [9], which result from users' typology, preferences, and motivations together with system design. Cognitive load theory [8] describes engagement as mental resources of a definite magnitude, which could be drained by poor system design and extraneous distraction. Both theories would back up the variable nature of engagement and its impact on experience, performance, and productivity. [PERSON] et al. [10] classify engagement into _task engagement_ and _social engagement_. Human users' _social engagement_, with counterpart agents for curiosity, companionship, or amusement, is not considered unless networking is the goal per se. One may argue that, e.g., tutors' characters in e-learning or cockpit design for drivers have profound influence on engagement. Hereby, this article concentrates on _task engagement_ and _productive engagement_[7] for the purpose of measurability and objectivity of engagement. For instance, we think that engagement in e-learning should be reflected by learning outcomes. Driver engagement comes down to safety, responsiveness, and ability to comply with traffic rules. [PERSON] [11] regards engagement as comprising three perspectives: _behavioural_, _emotional_, and _cognitive_. We follow said taxonomy to identify sensors and to group domains of application. Figure 1: An illustration of the sub-processes of engagement. The human appraisal system, cognitive system, motivation system, and motor system all reveal important information about engagement. Emotions are part of the appraisal process of the behaviour of the interaction partner. A certain level of cognitive load will be associated with engagement. Motor responses such as gross motor, gaze, and facial expression can be captured through observation. Many techniques can measure engagement sub-processes even while not directly measuring engagement as whole. Behavioural engagement refers to human agents' active participation in the task at hand, such as an operator striking a keyboard, a learner taking notes, and a driver steering the wheel. Audiovisual data provide invaluable information to analyze behavioural factors [12; 13; 14], thanks to the rich information contained in the visible and audible spectra. To name a few, eye tracking [12; 15], gaze [16; 17], posture [18], gesture [4], speech [19; 20; 21], and observed behavioural changes [22; 23] reveal a large amount of information about users' engagement state. _Emotional_ engagement describes human agents' affective reactions disclosed by facial expressions in conjunction with body pose and speech [3]. Emotions are part of the appraisal process of the behaviour of the interaction partner. Such emotions can cover all emotion classes/dimensions. Emotion can be part of the idiothetic emotions to appraisal of the engagement process itself. For instance, happiness and contentment usually indicate high-level engagement; boredom or frustration with shaking head often indicate a decrease in engagement. Note that emotion classification is regarded a standalone downstream task often requiring recognition and feature engineering of the Facial Action Unit (FAU). Engagement is correlated to but not necessarily determined by emotion classification. For example,'surprised' or 'nervous' emotions may indicate strong engagement [24]. _Cognitive_ engagement relates to human agents' internal state of mind. A certain level of cognitive load [8] will be associated with engagement which involves sensory processing and perception. _Cognitive_ engagement can be difficult to measure. One can easily relate to the experience of mind-wandering in a class while behaving as if paying attention by gazing at the tutor.