# Grammar-Supervised End-to-End Speech Recognition with Part-of-Speech Tagging and Dependency Parsing
## 4 Experiments
### Grammar-Based Mean Opinion Score of Speech Recognition Hypothesis  
As an objective statistical indicator based on the LTP toolkit [34], GDD is defined to evaluate the reasonable degree of the speech recognition hypothesis. In order to reflect the proper degree of intelligibility for all the hypotheses of the different methods, we introduce a new method to listen to the natural feeling of the users and pay more attention to their satisfaction. Inspired by the application of the mean opinion score (MOS) in text-to-speech [35; 36], the grammar-based mean opinion score (GMOS) is analyzed based on a subjective evaluation experiment. Three volunteers with linguistic ability are chosen to rate the candidate hypothesis on factors including reading fluency, grammar changes, and semantic deviation. After obtaining the GMOS of the three volunteers, the mean value is calculated. From Table 6, it can be observed that the multiple evaluation methodology of grammar has a significant improvement. Compared with the baseline system, the MEMG increased the GMOS from 4.11 to 4.29 on AISHELL-1, while it increased the GMOS from 3.02 to 3.20 on TRANS-M. These results indicate that the proposed method is conducive to improving the subjective experience.  
In Table 7, we also list some examples from the test sets. Our proposed methods improve the readability of the speech recognition results compared with the baseline model, which proves the improvement of the grammatical level again.  
\begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**GMOS**} \\ \cline{2-3}  & **AISHELL-1** & **TRANS-M** \\ \hline Baseline & 4.11 \(\pm\) 0.06 & 3.02 \(\pm\) 0.08 \\ MEMG & 4.29 \(\pm\) 0.06 & 3.20 \(\pm\) 0.08 \\ \hline \hline \end{tabular}
\end{table}
Table 6: The GMOS of the proposed system on two test sets.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**No LM (\%)**} & \multicolumn{2}{c}{**4-gram LM (\%)**} \\ \cline{2-5}  & **CER** & **GDD** & **CER** & **GDD** \\ \hline Baseline & 4.87 & 6.16 & 4.59 & 5.79 \\ MEMG & 4.71 & 5.87 & 4.47 & 5.51 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Model comparison with language model on AISHELL-1.  
Both the objective statistical indicators and subjective experience indicators above show that the grammar-supervised end-to-end speech recognition system may be more suitable for improving the readability and accuracy of the speech recognition hypothesis. By introducing the grammar knowledge such as part-of-speech tagging and dependency parsing, the information representation of the attention-based encoder-decoder ASR system can correlate well with an ability to understand the syntactic structure. Compared with the implicit learning of grammatical knowledge in the decoder module, the explicit use of grammar knowledge is more conducive to correlating with practical value. The expandable usage paradigm with grammar knowledge can be used in other intelligent speech systems, especially those involved in speech and text transition such as text-to-speech systems [35, 36].