## 3 Process for Training a Neural Network
### User Data Collection and Problem Reformulation  
In the identical scenario to the navigation problem in Definition 1, human users were asked to make a decision based on the observed scenes at each occasion. First of all, the users were only able to observe the space within the detection range of the sensor, where the distance between the robot and obstacles as well as the direction to the destination point were provided. Then the users judged how the robot should move based on the known information and controlled its movement to the next space through the keyboard. Note that to reduce the correlation of each datum in time and space, the position and orientation of the robot were randomly placed after performing an action. The decisions made by these users were recorded for a total number of \(N\) occasions. On the \(j\)-th occasion, a set of labeled data was generated and stored in a database \(\Lambda\) as  
\[\lambda_{j}=[d_{1}^{j}\ d_{2}^{j}\ldots\ d_{m}^{j}\ \phi^{j}\ l^{j}],\ j=1, \ldots,N, \tag{5}\]  
where \(j\) is the occasion index number, \(d_{i^{\prime}}^{j}\ i=1,\ldots,m\), are the obstacle distances detected by the laser distance sensors, \(\phi^{j}\) is the bias of the heading direction measured by the location sensor, and \(l^{j}\) is the label of human made decision \(\mathcal{M}_{i}\), i.e.,  
\[l^{j}\in\{1\ 2...\ n\}. \tag{6}\]  
To ensure the collected data were meaningful, an assumption was made as follows:  
**Assumption 3**: _The human users tried their best to achieve the given navigation task, and they were capable of successively completing this task, i.e., they did not aim to make the mobile robot collide with an obstacle or move toward the wrong destination._  
Based on the above assumption, it is acknowledged that the navigation problem in Definition 1 can be solved by a mechanism, which replicates the human decision-making process. Therefore, using the stored labeled data, the navigation problem can be transform into a classification task. As explained in [34], after obtaining the data set \(\Lambda\), \(\eta\) elements were randomly selected to form the training set \(\Lambda_{train}\), and the rest were regarded as the test set \(\Lambda_{test}\). An \((m+1)\)-input and \(n\)-output neural network model was employed to obtain the probability of each navigation action, whose input was \([d_{1}^{i}\ d_{2}^{i}\ldots\ d_{m}^{i}\ \phi^{j}]\in\mathbb{R}^{m+1}\) and output was \(z_{i}\in\mathbb{R}^{n}\). As a result, the robot navigation task was reformulated as a supervised learning problem defined below.  
**Definition 2**: _The **Navigation Decision-Making Problem** is to iteratively update the parameters of network model using the elements cut into the training set \(\Lambda_{train}\) from collected data, such that the cross-entropy loss function_  
\[\mathcal{L}(z_{i},\ \hat{z}_{i})=-\sum_{j=1}^{n}\hat{z}_{i}\log z_{i}(j),\ i=1, \ldots,N \tag{7}\]  
_is minimized for all \(N\) labeled datasets, where \(\hat{z}_{i}\in\mathbb{R}^{n}\) is the expected output defined by \(l^{i}\) as_  
\[\hat{z}_{i}(j)=\begin{cases}1,\ j=l^{i},\\ 0,\ \text{else}.\end{cases} \tag{8}\]  
The cross-entropy loss function (7) was utilized to evaluate the loss between the output of the neural network and the human behavior label. The weights of the neural network were continuously adjusted to reduce the loss value through the error back-propagation algorithm. When the cross entropy loss became small enough, the neural network model was considered to completely fit the mapping relationship between input and output.