## 5 Experimental Analysis
### Embedded State Dimension  
_How to choose the embedded dimension of the SRL model?_  
To assess how the choice of embedded dimension can affect the RL algorithm performance, we use our aforementioned default set of hyperparameters except of the embedded dimension across an extended suite of discrete robotic tasks and study how well the RL algorithms perform across the different embedded states.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Task**} & \multirow{2}{*}{\begin{tabular}{c} **Network** \\ **Configuration** \\ \end{tabular} } & \multicolumn{4}{c}{**Correlation**} & \multirow{2}{*}{\begin{tabular}{c} **KNN-MSE** \\ **Max Correlation Matrix** \\ \end{tabular} } & \multirow{2}{*}{\
\begin{tabular}{c} **Mean** \\ **Mean** \\ \end{tabular} } & \multirow{2}{*}{
\begin{tabular}{c} **Average Total Reward** \\ **PPO (1e6 Timesteps)** \\ \end{tabular} } \\ \hline \multirow{3}{*}{1D-Nav.} & Config 1 (1.8 k*) & 0.7821 & 2.9816e-15 & None & None & 0.3910 & 0 & 238.41 \(\pm\) 3.72 \\  & Config 2 (87.9 k) & 0.8563 & 3.2195e-15 & None & None & 0.4282 & 0 & 238.35 \(\pm\) 4.68 \\  & Config 3 (789 k) & 0.5612 & 2.3240e-15 & None & None & 0.2806 & 0 & 238.21 \(\pm\) 3.76 \\ \hline \multirow{3}{*}{2D-Nav.} & Config 1 (1.8 k*) & 0.4797 & 0.5046 & 0.9999 & 0.9999 & 0.7461 & 0.00063 & 229.04 \(\pm\) 4.90 \\  & Config 2 (87.9 k) & 0.3303 & 0.4587 & 0.9999 & 0.9999 & 0.6973 & 0.00066 & 227.30 \(\pm\) 5.38 \\  & Config 3 (789 k) & 0.2176 & 0.3834 & 0.9999 & 0.9999 & 0.6503 & 0.00064 & 227.05 \(\pm\) 6.09 \\ \hline \hline \end{tabular}
\end{table}
Table 3: The influence of the network configuration in mobile robot navigation task.  
Figure 6: Performance (mean and standard error) on mobile robot 2D navigation task with different autoencoder methods using PPO algorithm.  
The following two tasks with OpenAI Gym interface are utilized for the experiments: mobile robot 1D navigation task and mobile robot 2D navigation with random-target task. The dimensions of the ground-truth state space (i.e., robot and target positions) in these two tasks are 2 and 4 separately. The dimensions of the action space are 2 and 4, respectively. The complexity of the image observations also increases, which makes the state representation learning process gradually difficult.  
**Results:** Table 4 shows the \(GTC\), \(GTC_{mean}\), \(KNN\)-\(MSE\) and associate mean reward performance of RL methods (DQN and PPO) per episode (average on 100 episodes) after \(1e6\) steps for the mobile robot navigation tasks. When the environment and the task are simple, as shown in Figures 7 and 8, a small state dimension could help the RL agent learn faster generally; however, when the agent interacts with a complicated environment in which the task is relatively difficult, the state dimension for the SRL model must be large enough to solve the task efficiently. Increasing the state dimension above a certain threshold, on the other hand, has no effect (positively  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Task**} & \multirow{2}{*}{
\begin{tabular}{c} **Embedded** \\ **Dimension** \\ \end{tabular} } & \multicolumn{4}{c}{**Ground Truth Correlation (GTC)**} & \multirow{2}{*}{**KNN-MSE**} & \multicolumn{2}{c}{**Average Total Reward**} \\ \cline{3-9}  & & \multicolumn{2}{c}{**Max Correlation Matrix**} & \multicolumn{2}{c}{**Mean**} & \multicolumn{2}{c}{**DQN**}