# Experimenting the Automatic Recognition of Non-Conventionalized Units in Sign Language
## 1 Introduction
### Automatic Continuous Sign Language Recognition: State of the Art
#### 1.2.3 Experiments Outside the Field of Continuous Lexical Sign Recognition  
As exemplified by Table 1, competition in the field of SLR is highly focused on the task of CLexSR, especially on the RWTH-PW corpus. However, it is notable that a few works have tried to explore the task of end-to-end Sign Language Translation, using the Neural Machine Translation (NMT) encoder-decoder architecture. However, the associated models are trained with a full [34] or partial [33] gloss supervision, on the quite linguistically restricted corpus RWTH-PW (see Section 1.2.1). Such models are thus ill-equipped to tackle natural SL utterances that include non-conventionalized illustrative structures.  
While acceptable SLT performance is not nearly achieved, other SLR works have opted for more realistic goals and tackling some complex linguistics processes of SL. Very early on, [16] made use of data gloves to build a SLR system for the recognition of standard lexical signs, proforms and directional verbs. This HMM-based model was tested on a small self-made corpus, with encouraging results, although scaling  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline \multirow{2}{*}{**Paper**} & \multicolumn{2}{c}{**RWTH-Phonix-Weather**} & \multicolumn{2}{c}{**SIGNUM**} & \multicolumn{2}{c}{**Continuous SLR100 \({}^{\ddagger}\)**} \\ \cline{2-6}  & **SD** & **SI** & **SD** & **SI** & **SD** & **SI \({}^{\star}\)** \\ \hline [PERSON] et al. [31] & - & - & 12.7 & **34.9** & - & - \\ [PERSON] et al. [13] & 53.0 & - & 10.0 & - & - & - \\ [PERSON] et al. [18] & 45.1 & - & 7.6 & - & - & - \\ [PERSON] et al. [19] & 38.8 & - & 7.4 & - & - & - \\ [PERSON] et al. [20] & 40.7 & - & - & - & - & - \\ [PERSON] et al. [21] & 38.7 & - & - & - & - & - \\ [PERSON] et al. [22] & 26.8 & **44.1** & **4.8** & - & - & - \\ [PERSON] et al. [32] & 32.5 & - & 7.4 & - & - & - \\ [PERSON] et al. [15] & 38.3 & - & - & - & - & 17.3 \\ [PERSON] et al. [24] & - & - & - & - & 63.0 & 10.2 \\ [PERSON] et al. [25] & 36.7 & - & - & - & **32.7** & - \\ [PERSON] et al. [26] & 38.7 & - & - & - & 61.9 & - \\ [PERSON] et al. [27] & 36.5 & - & - & - & 44.7 & 14.3 \\ [PERSON] et al. [29] & 34.5 & - & - & - & - & 4.5 \\ [PERSON] et al. [28] & 34.9 & - & - & - & - & **3.8** \\ [PERSON] et al. [30] & 26.0 & - & - & - & - & - \\ [PERSON] et al. [33] & **24.5** & - & - & - & - & - \\ \hline \hline \end{tabular}
\end{table}
Table 1: Reported Word Error Rate (WER) (%) of methods detailed in Section 1.2.2 applied to Continuous Lexical Sign Recognition on the corpora presented in Section 1.2.1. SD and SI stand for Signer-Dependent and Signer-Independent. \({}^{\star}\) The exact same annotated _sentencs_ are present in training and test sets. \({}^{\ddagger}\) It is unclear whether the training/test splits of the different papers are comparable.  
up to bigger corpora with coarser annotation schemes is not straightforward. More recently, on the NCSLGR corpus (see below) [35] trained a HMM-SVM model to recognize five non-manual markers--in this case, face expressions--on a subset of the NCSLGR corpus, which are relevant at the syntactic level, namely: Negation, Wh-questions, Yes/no questions, Topic or focus and Conditional or 'when' clauses. Related to this, [36] trained and tested a _sign type_ classifier. Their model, based on optical flow and a Conditional Random Field (CRF) architecture, classifies any frame into one of three main sign types: Lexical sign, Fingerspelled sign and Classifier sign. The advertised accuracy is high (91.3% at the frame level), but it is computed on frames that belong to the three categories only.