# A Topological Approach to Enhancing Consistency in Machine Learning via Recurrent Neural Networks
## 4 Experiments
### Analysis in 1D-Space  
In 1D-Space, we examine the cases where \(m=n=1\), according to the consistency discussion in Section 3.1. To generate a synthetic dataset in 1D-Space, the cubic continuous function in Equation (6) is used, where the nonlinear transformation is applied to the input values within the domain limited to the interval \((0,0.5)\). This nonlinearity introduces a specific curvature to the data distribution, enriching the dataset with a more complex structure for analysis. Another important reason for selecting the function in Equation (6) is that its domain, \((0,0.5)\), and range, \((0,1)\), naturally alleviate the need for normalization in the training of NN architectures.  
\[F:(0,0.5)\rightarrow(0,1),\quad F(x)=x^{3}+1.5x^{2}+x \tag{6}\]  
By executing the randomization algorithm described in [40] one thousand times, we generated twenty real number values within the range of \((0,0.5)\), using a uniform distribution. The labels for each data point were calculated according to Equation (6). These data points were then divided into training and test sets with dimensions of \((800,20,1)\) and \((200,20,1)\), respectively; here, the first number indicates the number of samples, the second represents the time steps, and the third denotes the number of output channels. Furthermore, using the same randomization algorithm, we performed one hundred iterations to generate datasets with varying numbers of real number values, specifically with \(10^{2}\), \(10^{3}\), \(10^{4}\), \(10^{5}\), and \(10^{6}\) data points, all within the domain of \((0,0.5)\). These data points, with an incrementally increased number of values, are employed to quantitatively observe the consistency of the predictions.  
Two different NN architectures are built using a GRU and a ConsRNN layer. In both architectures, the recurrent layers have 128 hidden neurons with a sigmoid activation function, followed by a fully connected layer with a sigmoid activation function. The optimizer is chosen as Adam with a learning rate of 0.001. The built architectures are trained for 1000 epochs using a mean squared error (MSE) loss function on the training set. While the GRU-built architecture gave \(4.0\times 10^{-5}\) MSE, the ConsRNN-built architecture gave \(2.0\times 10^{-4}\) MSE on the test set. The trained NN architectures are utilized to estimate the objective function presented in Equation (6), based on data points that include increments of \(10^{2}\), \(10^{3}\), \(10^{4}\), \(10^{5}\), and \(10^{6}\), as shown in Figure 3.  
When examining the test results at different increment numbers, as shown in Figure 3, we observe that the architecture built from the ConsRNN layer yields a slightly worse MSE score over 20 increments. This initial observation might suggest a marginal reduction in learning capacity. However, upon examining larger increment numbers--\(10^{2}\), \(10^{3}\), \(10^{4}\)  
Figure 3: Consistency results in 1D-Space: As seen in the figures, while the number of increments increases, the ConsRNN-based NN architecture provides consistent estimates, whereas the GRU-based NN architecture loses accuracy and deviates from the ground truth.  
\(10^{5}\), \(10^{6}\)--it becomes evident that the MSE results of the NN architecture built from the GRU layer deteriorate significantly. Meanwhile, the MSE results of the NN architecture built from the ConsRNN layer approximately maintain the MSE results, showcasing a stable performance. This pattern indicates that while the transition function, introduced in Equation (4), may slightly diminish the overall learning capacity in initial stages, the prediction results tend to converge and become more consistent with the increase in the number of increments. These results in 1D space validate the theoretical construction on the basis of the consistency of the transition formula in Equation (4).