# Explaining Deep Learning-Based Driver Models
## 4 Experimental Setup and Results
### Results and Discussion
#### 4.2.2 Activity Model's Results
All of this proves that, to build an accurate model, one pair of subject and activity is not enough for each subject. It is necessary to record the same person doing the same activity in different poses, and with different clothes, illuminations, and backgrounds, so that the model does not learn the specific characteristics of the situation recorded. On the other hand, it is also important to train the model only with input data that is representative of its class. However, XRAI proved to be an excellent method to validate the model. By analyzing the decisions taken, we can understand the scope and utility of the model. In this case, using only training data, we found that, even though accuracy was high, it is still necessary that the model learns to recognize generic elements instead of characteristics specific to some drivers. Moving on to the other data, which we recreated by ourselves by imitating the _3 MDAD_ dataset, we expected the results to be worse, because this dataset was not used for training and it changed the subjects, the environment, and even the car. However, since the angle and position of the camera were similar, if the model was trained with enough data to generalize its deductions, then it should be able to classify correctly those new instances. Figure 23 shows an example of a photograph recreated by us, next to an original one from 3 MDAD. The GPS is absent in the photo, since we did not have the resources to attach it in all cars used for testing. This limited the number of photographs depicting the action \"GPS\" that we could take. Figure 21: Frames 15, 30, 45, and 60 of Subject 27 from 3 MDAD performing activity “GPS”. Figure 15:0 XRAI visualization of Frame 60 of Subject 27 from 3 MDAD performing activity “GPS” that was classified as “GPS”. After processing all 200 photographs, we found that only 78 of the images (39%) were correctly classified. This result by itself reveals that the model is not capable of generalization by using only the 3 MDAD dataset at training, and this becomes even more evident when one looks at the misclassifications and realizes that most of the false positives are of instances classified as REACH BEH (39 out of 122, i.e., 31%), which shows that something is not working correctly. Following the methodology presented in this paper, we use the XRAI visualizations to understand why the model underperforms with the new data. We start by searching for an explanation as to why the model gives so many false positives of the label \"REACH BEH\". To do this, we first examine two photographs: one that is correctly classified and one that it is not. These two examples can be seen in Figure 15:1  
As we can see, the main difference between the two images is that in the second example the hand of the woman is sightly lowered, as is her head. This simple difference makes the model stop focusing on the hand and instead tries to classify the photograph based mostly on the woman's head position and on the elements around her. Figures 25 and 26 show the corresponding XRAI explanations. Figure 15:2 (**left**) Frame 60 of Subject 43 from 3 MDAD performing activity “MESSAGE R”. (**right**) Photograph taken by us of a subject performing activity “MESSAGE R”. Figure 15:3 (**left**) Woman performing activity “GPS” that was classified as “GPS”. (**right**) Woman performing activity “GPS” that was classified as “REACH BEH”. While performing activity \"REACH BEH\", where the driver tries to reach something in the backseat, it is usual that he or she tries to keep their gaze on the road. Because of this, it would be reasonable that the model detects this action analyzing the driver's posture: one hand on the wheel, a turned head, body leaning to the side, etc.