# Using Neural Networks for Bicycle Route Planning
## 3 Materials and Methods
### Modelling Bicycle Paths Using Neural Networks
We noticed that, with the neural number increasing (e.g., a number of learning parameters increase), the training time also increased, but the model's accuracy has not significantly improved. The Levenberg-Marquardt backpropagation [26] was used as a training function; it uses a Levenberg-Marquardt optimisation for weight and bias weight update. It requires more memory than the other training functions available in nntool, but it is recommended since it is the fastest. The Levenberg-Marquardt algorithm does not use the Hessian matrix, which can be approximated as:  
\[H=J^{T}J \tag{2}\]  
and the gradient can be calculated as:  
\[g=J^{T}e \tag{3}\]  
\(J\) is the Jacobian matrix containing network errors first derivatives concerning the weights and biases, while \(e\) is a vector of network errors. \(T\) stands for transposition, \(\mu\) is the damping factor that is adjusted at each iteration; it is decreased after each successful step and decreased if that would increase the performance function. \(I\) is the identity matrix. The Levenberg-Marquardt algorithm can calculate the Jacobian matrix using the approximation that is much less complex than Hessian matrix computation:  
\[x_{k+1}=x_{k}-\big{[}J^{T}J+\mu I\big{]}^{-1}J^{T}e \tag{4}\] \begin{table}
\begin{tabular}{c c} \hline
**Town** & **No. of Segments** \\ \hline Vrgorac & 168 \\ Knin & 215 \\ Imotski & 268 \\ \hline \end{tabular}
\end{table} 
Table 1: Number of road segments for each town. The gradient descent with momentum weight and bias learning function was used in our training process as an adaption learning function. The choice of learning rate has a great impact on NN performance. It is a number that tells how the error is going to affect model weight parameters. Adaptive learning functions can adapt the learning rate for all weights in NN, where momentum will consider the weight delta from the previous iteration to calculate weight in the current iteration. MSE (mean-squared error) was used as a performance function. In this work, we used hyperbolic tangent sigmoid transfer function as the activation function of each neuron as:  
\[a=\frac{2}{(1+e^{-2a})}-1 \tag{5}\]  
This activation function is bounded and continuous, which makes it compatible with the backprop NN. Training parameters are shown in Table 2.  
NNs with information passing across time steps are recurrent neural networks (RNN) [27]. Many applications of RNN are involved in text processing, image recognition, face detection, speech-to-text systems... At time \(t\), nodes receive input information from the present node \(x(t)\) and information from hidden nodes \(h(t-1)\) from NN's previous state. The output, marked with \(y^{\prime}(t)\), will be produced using that step's hidden state, marked with \(h(t)\). Using recurrent connections, the input \(x(t-1)\) at time \(t-1\) can affect the output \(y(t)\) at time \(t\). An example of a recurrent NN unfolded across time steps is shown in Figure 3.  
Figure 2: Diagram with route calculation procedure explained. Recurrent network training is considered to be difficult since its problems of exploding gradients. It occurs when errors propagate across time steps. Considering the input passed to the network at time \(t1\) and an error occurring at time \(t\), the input's impact at time \(t1\) on the output at time \(t\) will explode as \(t-t1\) grows. The NN's performance was evaluated using mean square error, which is the mean of the squared difference between the actual and the estimated value. The MSE was calculated using cross-validation in which 70% of the data were used for training, and 15% for validation and testing. The MSE for each NN model is shown in Table 3.