## 3 Results and Discussion
### Interactive Learning/Human-In-The-Loop
Although human engagement can aid the training of the machine learning model and increase performance, it also introduces challenges. In active learning, a user is prompted to label the data [9]. However, treating the end user as a passive information oracle has proved problematic because it is not always realistic [21; 26]. For instance, in a real-world scenario, a user might not always respond as expected. [PERSON] et al. [24] explore how the performance of the machine learning classifications is affected if the reliability level of the user is varied. Experiments were conducted where the user did not always respond in accordance with what was expected and where the label provided was not always correct. The interactive learning strategies and machine learning algorithms affected the performance in the experiments, where the combination of Naive Bayes classifier (compared to SVN and k-NN) and the machine teaching strategy triggered by error resulted in the overall highest performance. Figure 4: IDIVS employing active learning (an extension of earlier work on DIVS [2]). There might be many reasons for applying a human-centred design to the interactions between users and a machine learning system. Considering the IDIVS concept, this aspect might be decisive in getting the user to provide relevant and correct feedback. A human-centred approach is often necessary to apply when designing interactions in order to improve the quality of user experiences in this context [27]. It may create a frustrating experience for the end user by neglecting the human ability to experiment and revise, as relevant for the IDIVS concept, thus hindering the user's ability to provide relevant and correct user feedback. We argue that the IDIVS concept allows for a human-centred design by allowing multiple ways to trigger users to provide feedback. Further, it may engage users to provide labels, which is important since labelling is often regarded as tedious and/or costly. In addition, interactive machine learning models, which may support explanations, feedback, or both, facilitate the interaction of non-expert users with systems for the purpose of improving the learning process, thus increasing system transparency and understandability. This is an approach that can benefit from designing interactions based on user needs and perceptions [28]. Importantly, these systems often rely on user interfaces and interaction modalities to interact with people--visual, audible, tactual, etc. [29]. It has been confirmed that explanations as a dialogue could improve the algorithmic experience in AI systems. For instance, in music recommendation systems, a study [30] confirms that designing experiences that help users understand the underlying algorithmic workings will make recommendations from intelligent systems more usable, thereby providing users with better experiences. It goes beyond the scope of this paper to discuss how to potentially make the IDIVS more explainable. However, we argue that the concept of providing rather minimalistic output, i.e., a virtual sensor value, using appropriate interaction modalities in comparison to a complex information type as output, alleviates the difficulty for a user to understand the output, particularly if the user is also engaged in the teaching of the system. Figure 5: The accumulated average accuracy of different learning algorithms for different labelling budgets and interactive machine learning strategies for the Opportunity dataset. Recompiled results from [PERSON] et al. [22] into average values of the different learning algorithms (Naïve Bayes (NB), k-Nearest Neighbour (k-NN), and Support Vector Machine, SVM).