# Experimenting the Automatic Recognition of Non-Conventionalized Units in Sign Language
## Appendix A Performance Metrics for Temporal Data: Details and Illustration
### Unit-Wise Metrics  
For these metrics, the time gap between the middle of \(u_{d}\) and the middle of the closest unit of the same class in \(U_{G}\) is compared to \(t_{w}\), in order to decide whether \(u_{d}\) is a correct detection-respectively, the time gap between the middle of \(u_{g}\) and the middle of the closest unit of the same class in \(U_{D}\) is compared to \(t_{w}\), in order to decide whether \(u_{g}\) is correctly detected. Let us first note that:  
* The closest unit from \(u_{1}^{D}\) is unit \(u_{1}^{G}\), with 4 frames of shift between their respective centers. * The closest unit from \(u_{2}^{D}\) is unit \(u_{1}^{G}\), with 14 frames of shift between their respective centers. * The closest unit from \(u_{3}^{D}\) is unit \(u_{2}^{G}\), with 0.5 frame of shift between their respective centers. Also:  
* The closest unit from \(u_{1}^{G}\) is unit \(u_{1}^{D}\), with 4 frames of shift between their respective centers. * The closest unit from \(u_{1}^{G}\) is unit \(u_{3}^{D}\), with 0.5 frame of shift between their respective centers. * The closest unit from \(u_{3}^{G}\) is unit \(u_{3}^{D}\), with 21 frames of shift between their respective centers. * The closest unit from \(u_{4}^{G}\) is unit \(u_{3}^{D}\), with 32.5 frames of shift between their respective centers. From Equations (14) and (15), unit-wise precision and recall as a function of a margin \(t_{w}\) can be written as:  
\[P_{w}^{\star}(t_{w}) =\frac{1}{3}\left(\mathds{1}_{t_{w}>4}+\mathds{1}_{t_{w}>14}+ \mathds{1}_{t_{w}>0.5}\right)\] \[R_{w}^{\star}(t_{w}) =\frac{1}{4}\left(\mathds{1}_{t_{w}>4}+\mathds{1}_{t_{w}>0.5}+ \mathds{1}_{t_{w}>21}+\mathds{1}_{t_{w}>32.5}\right).\]  
With margins of half a second (12 frames) or one second (25 frames), numerical values are:  
\[P_{w}^{\star}(12)\simeq 0.67\ ;\ \mathds{R}_{w}^{\star}(12)=0.5\ ;\ \mathds{1}_{w}^{ \star}(12)\simeq 0.57\]\[P_{w}^{\star}(25)=1\ ;\ R_{w}^{\star}(25)=0.75\ ;\ \text{F1}_{w}^{\star}(25)\simeq 0.86.\]  
Appendix A.2.2. \(P_{\text{pr}}^{\star}\), \(R_{\text{pr}}^{\star}\), \(F1_{\text{pr}}^{\star}\)  
\(P_{pr}^{\star}(\bar{t}_{p},\bar{t}_{r})\) and \(R_{pr}^{\star}(\bar{t}_{p},\bar{t}_{r})\) can be expressed as:  
\[P_{pr}^{\star}(\bar{t}_{p},\bar{t}_{r})=\frac{1}{|U_{D}|}\sum_{u_{d}\in U_{D} }\text{IsMatched}\left(\text{BestMatch}\left(u_{d},U_{G}\right),u_{d},\bar{t}_{ p},\bar{t}_{r}\right) \tag{1}\]  
\[R_{pr}^{\star}(\bar{t}_{p},\bar{t}_{r})=\frac{1}{|U_{G}|}\sum_{u_{g}\in U_{G}} \text{IsMatched}\left(u_{g},\text{BestMatch}\left(u_{g},U_{D}\right),\bar{t}_{ p},\bar{t}_{r}\right)\text{.} \tag{2}\]  
For each unit \(u_{d}\) in the list \(U_{D}\), one can define the best match unit in \(U_{G}\) as the one maximizing the normalized temporal overlap between units (and a symmetric formula for the best match of a unit \(u_{g}\) in \(U_{D}\)):  
\[\text{BestMatch}\left(u_{d},U_{G}\right)=\underset{u_{g}\in U_{G}}{\text{argmax}} \begin{cases}\frac{2\ \#\ \text{frames}\left(u_{g}\cap u_{d}\right)}{\#\ \text{frames}\left(u_{g}\right)+\#\ \text{frames}\left(u_{d}\right)}&\text{if}\ \text{Class}(u_{g})=\text{Class}(u_{d})\\ 0&\text{otherwise}\end{cases}  \tag{3}\]  
IsMatched decides whether two units are sufficiently similar, which can be written down as follows:  
\[\text{IsMatched}\left(u_{g},u_{d},\bar{t}_{p},\bar{t}_{r}\right)=\begin{cases} 1&\text{if}\ \begin{cases}\#\ \text{frames}\left(u_{g}\cap u_{d}\right) \\ \#\ \text{frames}\left(u_{d}\right)\\ \#\ \text{frames}\left(u_{g}\right)\\ \text{Class}(u_{g})=\text{Class}(u_{d})\\ 0&\text{otherwise.}\end{cases}  \tag{4}\]  
From Equations (3) and its symmetric, one can note that:  
* The best match for unit \(u_{1}^{D}\) is unit \(u_{1}^{G}\), with 7 intersecting frames over the 7 frames of \(u_{1}^{D}\). * The best match for unit \(u_{2}^{D}\) is unit \(u_{1}^{G}\), with 3 intersecting frames over the 11 frames of \(u_{2}^{D}\). * The best match for unit \(u_{3}^{D}\) is unit \(u_{2}^{G}\), with 5 intersecting frames over the 8 frames of \(u_{3}^{D}\).