# A Framework for Frugal Supervised Learning with Incremental Neural Networks
## 3 Results
### Application to Affective Computing  
The INN implementation was initially designed to address multimodal classification problems in affective computing, and more specifically concerning emotion and depression classification. Several results have been published [11,21], including a live prototype demonstration [30]. This experiment was previously published in [10], and consists of the use of the INN to predict the depressive score of subjects in the dataset of the AVEC2014 challenge. The dataset consists of 150 task-oriented videos of human-computer interaction scenarios. In each video, there is only one subject (see Figure 5), and the whole dataset contains 84 subjects. The recordings range from 20 to 50 min, totaling 240 h. The subjects' mean age was 31.5 years (SD = 12.3, range 18-63), and the recordings took place in quiet settings. The videos are labelled with a BDI-II score ranging from 0 to 63. Detailed information about the dataset can be found in [14]. In particular, it is worth noting that the classes are not ideally balanced in data, with only a few to no videos for higher depressive scores. The descriptors used were designed to estimate the quantity of movement on the face (micro-movements) and of the face itself (head gesture). For a given video, a set of 68 facial two-dimensional landmarks is extracted from each image (see Figure 5. The faces are aligned and their sizes are standardized by rotating the shapes by an angle \(\theta\) with a scale factor \(S\). This similarity transform can be represented by the matrix in Equation (6) and was estimated using OpenCV [31]. \[M= \begin{bmatrix}s_{x}cos(\theta)&sin(\theta)\\ -sin(\theta)&s_{y}cos(\theta)\end{bmatrix}  \tag{6}\]  
The final descriptor is composed of a rolling standard deviation \(\sigma\) of facial points of interest over an overlapping window of time of size \(\mathsf{T}_{\text{win}}\) with step \(\mathsf{T}_{\text{step}}\). The resulting features are aimed at capturing slow- and long-changing patterns occurring in the data, as well as their dynamics. This manner of processing data is not novel and was applied by [14], still in the context of affective computing. Let \(x_{jt}\) be the \(j\)th component of an observation at time \(t\). Then, the resulting descriptor component \(x^{\prime}_{jt}\) can be expressed as in Equation (7). \[ \begin{split} x^{\prime}_{jt}&=\sigma\Big{(}x_{jt},x_{j(t+ 1)},\dots,x_{j(t+\mathsf{T}_{\text{win}})}\Big{)}\\ x^{\prime}_{j(t+1)}&=\sigma\Big{(}x_{j(t+\mathsf{T}_{ \text{step}})},x_{j(t+\mathsf{T}_{\text{step}}+1)},\dots,x_{j(t+\mathsf{T}_{ \text{step}}+\mathsf{T}_{\text{win}})}\Big{)}\end{split}  \tag{7}\]  
Figure 4: Compared plot of classification accuracies on the iris dataset for ten common classifiers (color dash-dotted lines) plus the INN (plain black line). The ratio of prototypes is also plotted for the INN (blue dotted line). As the accuracy increases with the training set size to stabilize over 0.9, the ratio of prototypes progressively drops to less than 30%. The INN achieved state-of-the-art performance while requiring very few data during the training phase. An INN has been trained with a leave-one-subject-out cross-validation strategy. For each round of validation, one subject is left out from the training set and used as the validation set. This process is repeated for all subjects in the dataset, and the results are averaged to obtain an estimate of the model's performance. This strategy ensures that the model does not learn to associate a specific subject with a depressive state, which could bias the results. The classification errors of the cross-validation are provided in Table 1. As the depressive scores range from 0 to 63, an error around 10 could be regarded as \"fair\" given the complexity of the data, the validation strategy, and the nature of the classifier. It is also worth noting that the classifier struggled more with severe depressive states, while achieving smaller errors for other states.