# Order-Based Schedule of Dynamic Topology for Recurrent Neural Network
## 5 Discussion
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**MSE** & **Phase 0** & **Phase 1** & **Phase 2** & **Phase 3** & **Phase 4** & **Phase 5** & **Phase 6** & **Phase 7** & **Phase 8** & **AVG** \\ \hline M1 & **0.00005** & 0.00001 & **0.00023** & 0.00015 & 0.00002 & 0.00001 & 0.00001 & 0.00001 & **0.00006** \\ MD1 & 0.00006 & 0.00001 & 0.00035 & **0.00013** & 0.00002 & 0.00001 & 0.00001 & 0.00001 & 0.00001 & 0.00007 \\ TR1 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 1 & 2 & \\ RATE 1 & 1 & 0.85 & 0.85 & 0.85 & 0.85 & 0.85 & 0.85 & 0.7 & 0.85 & \\ \hline M2 & 0.00014 & 0.00001 & 0.00104 & 0.00014 & 0.00002 & 0.00001 & 0.00002 & 0.00001 & 0.00002 & 0.00015 \\ MD2 & **0.00008** & 0.00001 & **0.00029** & **0.00012** & 0.00002 & 0.00001 & **0.00001** & 0.00001 & **0.00001** & **0.00006** \\ TR2 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & \\ RATE 2 & 1 & 0.7 & 0.7 & 0.7 & 0.7 & 0.7 & 0.7 & 0.55 & 0.55 & \\ \hline M3 & 0.00009 & 0.00000 & **0.00020** & 0.00029 & 0.00002 & 0.00001 & 0.00001 & 0.00001 & 0.00001 & 0.00007 \\ MD3 & 0.00009 & 0.00000 & 0.00035 & **0.00014** & 0.00002 & 0.00001 & 0.00001 & 0.00001 & 0.00001 & 0.00007 \\ TR3 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & \\ RATE 3 & 1 & 0.55 & 0.55 & 0.55 & 0.55 & 0.55 & 0.4 & 0.4 & \\ \hline M4 & **0.00014** & 0.00001 & **0.00126** & 0.00015 & **0.00001** & 0.00001 & 0.00001 & 0.00001 & **0.00018** \\ MD4 & 0.00020 & 0.00001 & 0.00141 & 0.00015 & 0.00002 & 0.00001 & 0.00001 & 0.00001 & 0.00001 & 0.00020 \\ TR4 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & \\ RATE 4 & 1 & 0.7 & 0.7 & 0.7 & 0.7 & 0.7 & 0.7 & 0.55 & 0.55 & \\ \hline M5 & 0.00013 & 0.00000 & **0.00169** & 0.00020 & 0.00002 & 0.00001 & 0.00003 & 0.00001 & 0.00001 & 0.00023 \\ MD5 & **0.00008** & 0.00001 & 0.00184 & **0.00013** & 0.00002 & 0.00001 & **0.00001** & 0.00002 & 0.00001 & 0.00023 \\ TR5 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & \\ RATE 5 & 1 & 0.55 & 0.55 & 0.55 & 0.55 & 0.55 & 0.4 & 0.4 & \\ \hline
**CAO** & 5 & 4 & 4 & 4 & 4 & 4 & 5 & 4 & 5 & \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Series 11 (MSE), order [Cao], and comparison between the 5 models [M] and their respective dynamic versions [MD], rate, and the decision made [TR] (Growth = 2, Pruning = 1 Normal = 0). \begin{table}
\begin{tabular}{c c c} \hline
**Model** & **OWA** & **RATE** & **NEW Units** \\ \hline M1/MD1 & 1.11 & 0.85 & 0 \\ M2/MD2 & **0.54** & 0.55 & 0 \\ M3/MD3 & **0.93** & 0.4 & 0 \\ M4/MD4 & 1.03 & 0.55 & 0 \\ M5/MD5 & **0.87** & 0.4 & 0 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Metric (OWA). Final percentage (RATE) of the topology with respect to the initial model. (New units) Number of extra units added, test eleven. This process also helped in the situation described before, when a phase after a change of topology doesn't get a good result. Being this training a way to reorganize the knowledge of the model and give better results in the next phases. This case is presented in Table 11, that is one of the results of series 4, which has a reduction of order in phase 1 and after this, the next 3 phases have an increase of order. As can be seen in Table 11 model 3 failed in phases 2 and 3, but even though this occurred the model achieved at the end a 9% improvement, this result can be seen in Table 12.  
Figure 10: Graph of the Topology Rate (Dotted Lines, measured by left axis), Order evolution (CAO), and the decisions made by the algorithm (TR-Bars, the last two metrics measured by the right axis) in test five. Figure 9: Graph of the MSE metric (Solid and Dotted Lines, measured by left axis), Order evolution (CAO), and the decisions made by the algorithm (TR-Bars, the last two metrics measured by the right axis) in test five. Unifying the three methods of training (pruning, growing, and normal) of the proposed system, makes the models able to identify a global trend of the data and get better results in later phases. Figure 11 shows the evolution of models 1 (Vanilla/DyLSTM) over phases 1 to 8 of series 4.