Development of an open affective computing environment

Affective computing facilitates more intuitive, natural computer interfaces by enabling the communication of the user’s emotional state. Despite rapid growth in recent years, affective computing is still an under-explored field, which holds promise to be a valuable direction for future software development. An area which may particularly benefit is e-learning. The fact that interaction with computers is often a fundamental part of study, coupled with the interaction between affective state and learning, makes this an ideal candidate for affective computing developments. The overall aim of the research described in this thesis is to advance the field and promote the uptake of affective computing applications both within the domain of e-learning, as well as in other problem domains. This aim has been addressed with contributions in the areas of tools to infer affective state through physiology, an architecture of a re-usable component based model for affective application development and the construction and subsequent empirical evaluation of a tutoring system that responds to the learner’s affective state. The first contribution put forward a solution that is able to infer the user’s affective state by measuring subtle physiological signals using relatively unobtrusive and low-cost equipment. An empirical study was conducted to evaluate the success of this solution. Results demonstrated that the physiological signals did respond to affective state, and that the platform and methodology was sufficiently robust to detect changes in affective state. The second contribution addressed the ad-hoc and sometimes overly complex nature of affective application development, which may be hindering progress in the field. A conceptual model for affective software development called the Affective Stack Model was introduced. This model supports a logical separation and loose coupling of reusable functional components to ensure that they may be developed and refined independently of one another in an efficient and streamlined manner. The third major contribution utilized the proposed Affective Stack Model, and the physiological sensing platform, to construct an e-learning tutor that was able to detect and respond to the learner’s affective state in real-time. This demonstrated the real-world applicability and success of the conceptual model, whilst also providing a proof of concept test-bed in which to evaluate the theorized learning gains that may be realized by affective tutoring strategies. An empirical study was conducted to assess the effectiveness of this tutoring system as compared to a non-affect sensing implementation. Results confirmed that there were statistically significant differences whereby students who interacted with the affective tutor had greater levels of perceived learning than students who used the non-affective version. This research has theoretical and practical implications for the development of affective computing applications. The findings confirmed that underlying affective state can be inferred with two physiological signals, paving the way for further evaluation and research into the applications of physiological computing. The Affective Stack Model has also provided a framework to support future affective software development. A significant aspect of this contribution is that this is the first such model to be created which is compatible with the use of third-party, closed source software. This should make a considerable impact in the future as vast possibilities for future affective interfaces have been opened up. The development and subsequent evaluation of the affective tutor has substantial practical implications by demonstrating that the Affective Stack Model can be successfully applied to a real-world application to augment traditional learning materials with the capability for affect support. Furthermore, the empirical support that learning gains are attainable should spur new interest and growth in this area.