## 5 Experiment
### Experimental Performance Evaluation of Language Frameworks Based on Dynamic Tuple Extraction
#### 5.3.2 Overall Synthetic AOM Performance Evaluation  
The experimental results are shown in Table 9.  
The following points can be obtained from the above table.  
* In the table of synthetic AOM performance evaluation values, the method Seq2 Seq with BERT in this paper achieves 78.71% accuracy, 79.21% precision, 78.98% recall, and 79.09% F-score values on the NL2 AOM task. Its accuracy gains are 46.54%, 45.33%, 37.44%, and 37.88% for Seq2 Seq(LSTM), Seq2 Seq(GRU), Seq2 Seq(BiLSTM), and Seq2 Seq(BiGRU), respectively. The main reason is that LSTM, BiLSTM, GRU, and BiGRU are all variants of RNN, which can solve the recursive timing problem well but cannot sense the information globally. The recursive timing feature is implemented. This results in a higher accuracy rate on the sequence-to-sequence task from natural language commands to AOM statements.
* The experimental results also show that the dynamic tuple extraction-based language framework proposed in this paper can collect high-quality training datasets efficiently,  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Fields** & **Model Name** & **Accuracy(Train)** & **Accuracy(Test)** \\ \hline \multirow{6}{*}{Action} & Seq2 Seq(LSTM) & 45.12\%(758/1680) & 42.62\%(179/420) \\  & Seq2 Seq(GRU) & 41.43\%(696/1680) & 39.52\%(166/420) \\  & Seq2 Seq(BiLSTM) & 48.87\%(821/1680) & 45.48\%(191/420) \\  & Seq2 Seq(BiGRU) & 52.02\%(874/1680) & 44.52\%(187/420) \\  & Seq2 Seq(BERT)(Our method) & 86.43\%(1452/1680) & 85.24\%(358/420) \\ \hline \multirow{6}{*}{Object} & Seq2 Seq(LSTM) & 27.20\%(457/1680) & 26.90\%(113/420) \\  & Seq2 Seq(GRU) & 31.01\%(521/1680) & 29.05\%(122/420) \\ \cline{1-1}  & Seq2 Seq(BiLSTM) & 36.37\%(611/1680) & 34.76\%(146/420) \\ \cline{1-1}  & Seq2 Seq(BiGRU) & 35.77\%(601/1680) & 33.57\%(141/420) \\ \cline{1-1}  & Seq2 Seq(BERT)(Our method) & 61.13\%(1027/1680) & 60.24\%(253/420) \\ \hline \multirow{6}{*}{Member} & Seq2 Seq(LSTM) & 45.12\%(758/1680) & 42.62\%(179/420) \\ \cline{1-1}  & Seq2 Seq(GRU) & 41.43\%(696/1680) & 39.52\%(166/420) \\ \cline{1-1}  & Seq2 Seq(BiLSTM) & 48.87\%(821/1680) & 45.48\%(191/420) \\ \cline{1-1}  & Seq2 Seq(BiGRU) & 52.02\%(874/1680) & 44.52\%(187/420) \\ \cline{1-1}  & Seq2 Seq(BERT)(Our method) & 86.43\%(1452/1680) & 85.24\%(358/420) \\ \hline \hline \end{tabular}
\end{table}
Table 8: Segments Generation Accuracy.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**No.** & **Model Name** & **Accuracy** & **Precision** & **Recall** & **F-Score** \\ \hline
1 & Seq2 Seq(LSTM) & 32.17\% & 38.19\% & 37.21\% & 37.69\% \\
2 & Seq2 Seq(GRU) & 33.38\% & 34.57\% & 33.27\% & 33.91\% \\
3 & Seq2 Seq(BiLSTM) & 41.27\% & 47.89\% & 43.57\% & 45.53\% \\
4 & Seq2 Seq(BiGRU) & 40.83\% & 47.24\% & 46.87\% & 47.05\% \\
5 & Seq2 Seq(BERT)(Our method) & 78.71\% & 79.21\% & 78.98\% & 79.09\% \\ \hline \hline \end{tabular}
\end{table}
Table 9: Experimental comparison results of the language models.  
quickly, and at a low cost. Further, the proposed BERT-based Seq2 Seq model, based on the use of self-attention masks, can both globally acquire natural language instruction information and implement the sequence-to-sequence concept, thus enabling the better implementation of the NL2 AOM task and providing a more robust baseline.