# Experimenting the Automatic Recognition of Non-Conventionalized Units in Sign Language
## 3 Results and Discussion
### Qualitative Analysis on Test Set  
Although performance metrics provide interesting insights on the results of the proposed model, a more qualitative analysis is needed. In this section, we analyze the prediction results of the proposed model, on two test sequences of Dicta-Sign-LSF-v2 (Video clips are visible at [[https://github.com/vbelissen/cslr_limsi/blob/master/Clips.md](https://github.com/vbelissen/cslr_limsi/blob/master/Clips.md)]([https://github.com/vbelissen/cslr_limsi/blob/master/Clips.md](https://github.com/vbelissen/cslr_limsi/blob/master/Clips.md))). The signer representation is decided from the optimization  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & & & \multicolumn{3}{c}{**Frame-Wise**} & \multicolumn{3}{c}{**Unit-Wise**} \\ \cline{4-9}  & **SI** & **TI** & **Acc** & **F1** (_n_f  
Tables 8 and 9. The chosen setup is signer-independent and task-dependent (SI-TD). The test signers are then unknown both from the training and validation sets. These results complement preliminary analyses focused on DSs and developed in [67]. In this analysis, we have trained four binary descriptors, corresponding to FLSs, DSs, PTSs and FBuoys. In Figures 9 and 10 we show, from top to bottom: a few key thumbnails, a proposed English translation, expert annotations for FLSs and PLSs--each on three tracks, corresponding to right-handed, two-handed or left-handed units--and model predictions. Because all descriptors are binary, a positive prediction is equivalent to a probability greater than 0.5. Video S7_T2_A10, Frames 660-790 (Figure 9)  
This is a longer and much more complex sequence with all four types of annotations--ten FLSs, two DSs, three PTSs and one FBuoy. We propose the following translation: _\"For you to decide between those two [touristic] options, I will present them one after the other, then we will also discuss prices.\"_. This sequence makes extensive use of space at the syntactic level. Indeed, the lexical sign HESITATE is used in context in quite an iconic fashion, with one hand corresponding to an option A and the other hand to another option B. Using pointing signs and a visible tilt in the upper body, as well as localized signs like EXPLAIN, the two options are sequentially referred to in a very spatial and visual way. FLSs are detected quite correctly, with an \(I_{pr}\) of 0.60. Two pointing signs are detected, while one is missed. The two successive DSs are correctly detected, even though they are not segmented like the annotations. In the end of the sequence--and to a lesser extent the beginning--DSs are predicted by the model although they are not annotated. However, they do include a form of iconicity--as mentioned earlier, it is spatial iconicity used at the syntactic level. The unique FBuoy is not detected, resulting in \(I_{pr}=0\). Video S7_T2_A10, Frames 885-990 (Figure 10)  
This sequence is rather sequential and includes an illustrative structure around frame 920, with the left hand of the lexical sign PARIS iconically reactivated into a FBuoy, while the right hand performs a DS-Size&Shape (DS-SS). We propose the simple translation _\"You will need some time to explore Paris!\"_. All FLSs are detected correctly, but two _false positives_ are observed in the vicinity of the illustrative structure. The unique PTS is perfectly recognized. The DS unit is very well detected too, while the simultaneous FBuoy is detected but much shorter than it is annotated. Interestingly, the FLS VISIT is also detected as a DS. This makes some sense as it is produced in quite an iconic way, in a form of Transfer of Persons (T-P), emphasized by the gaze moving away from the addressee and the crinkled eyes. In conclusion to this qualitative assessment, it seems that the predictions of the four descriptors are generally well in line with the annotations, and could be used to describe a much broader part of SL discourse than the pure CLexSR approach.