# Sensors, Techniques, and Future Trends of Human-Engagement-Enabled Applications: A Review
## 7 Discussion
### Reliability  
Measuring and detecting human engagement accurately faces multifaceted challenges. A prime concern comes from established datasets. As discussed in Section 5, popular datasets of human engagement adopt naive labels, such as a value between 0 and 1 or binary/four-category/six-category classification of engagement level, without evidencing these labels from a cognitive science perspective. These labels can be overlapping, meaning boundaries between classes, such as _Not engaged_ and _Barely engaged_, are fuzzy, which makes a source of confusion for data-driven methods.  
Human engagement manifests in different ways, from frequent eyelid closure to head moving away from area of interest. The complexity of human engagement makes it difficult to estimate from one or multiple cues. Taking mind wandering for example, it is difficult to obtain ground truth of the internal state of mind even by interviewing human participants directly or by measuring physiological cues.  
There is a rising consensus that the reliable engagement estimation requires multi-modality sensing [12; 13; 118]. Using multi-modality sensors has the advantage of gaining multidimensional information from the scene allowing collective evaluation and enhanced robustness. For instance, when facial cues are occluded, continued conversation and gesture may be signs of sustained engagement. Thereafter, this complementary information could be merged in the learning-based pipelines through early fusion or late fusion. Early fusion refers to combining low-level sensory features from different modalities at the input level before feeding the concatenated input into a neural network [13; 14]. This allows the model to learn joint representations across modalities from raw data with a simple training pipeline. Late fusion processes each modality separately and combines them at the decision level [160; 161], e.g., through averaging or voting. However, this means networks for representation learning of each modality must be trained separately. Attention [162] is a common mechanism embedded in state-of-the-art neural networks to focus more on informative modalities in different contexts. There are also hybrid approaches combining elements of both early and late fusion.  
Engagement should be carefully put in the context of long-term versus short-term estimation. Diagnosis of ADHD could be a short-term practice, whereas monitoring it is considered a long-term campaign. User adhesiveness to a digital platform is typically deemed long term, but measuring engagement in a single usage is possible. Although attention lapses in class can be short term [12], learning effectiveness and learner's performance are usually assessed in the long run. In contrary, driver drowsiness detection and HRI design demand real-time responsiveness. Drivers should be warned immediately for confirmed disengagement caused by, e.g., sleepiness or fatigue. Social robots are expected to be exceptionally acute in discerning engaged users and floor changes. Real-time engagement recognition is paramount for industrial robotic agents to collaborate with humans safely on the shop floor. There is more to be accomplished to close the gap of real-time engagement estimation.