## Appendix A Details for Training Via Soft Actor-Critic
```
Distributed Soft Actor-Critic Hyperparameters  Parameter  Value  Discount factor \(\gamma\) 0.999  Target smoothing coefficient \(\tau\) 1 (hard update)  Target network update interval \(\eta\) 1000  Initial temperature coefficient \(\alpha_{0}\) 0.2  Learning rates for network optimizer \(\lambda_{\mathcal{Q}}\), \(\lambda_{\alpha}\), \(\lambda_{\pi}\) \(2\times 10^{-4}\)  Optimizer  Adam  Replay buffer capacity \(2^{20}\) (Binary Tree) (PER) prioritization parameter \(c\) 0.6 (PER) initial prioritization weight \(b_{0}\) 0.4 (PER) final prioritization weight \(b_{1}\) 0.6
```
input :\(\theta_{1},\theta_{2},\phi,Env,b,m\); \(\triangleright\) List of environments and the batch sizes for the SAC and the NavACL updates
1\(\overline{\theta_{1}}\leftarrow\theta_{1},\overline{\theta_{2}}\leftarrow\theta_{2}\); \(\triangleright\) Initialize target network weights \(\mathcal{D}\leftarrow\emptyset\); \(\triangleright\) Initialize an empty PER replay buffer \(\mathcal{E}_{s}\leftarrow\emptyset\); \(\triangleright\) Initialize an empty, shared episode replay queue \(\mathcal{L}_{m}\leftarrow\emptyset\); \(\triangleright\) Initialize an empty task result set \(init(f_{\pi})\); \(\triangleright\) Initialize the NavACL-Q network weights \(n\_updates\gets 0\); \(\triangleright\) Number of SAC updates foragent_index \(\leftarrow\) 0 to num_agentsdo
2 Spawn Process AsynchronousExperienceGathering(\(\phi,\overline{\theta},f_{\pi},\mathcal{E}_{s},env=Env[agent\_index]\))
3 end for
4
5 end for
6 whileTrue do
7 foragent_index \(\leftarrow\) 0 to num_agents do
8 if\(\mathcal{E}_{s}\
eq\emptyset\)then
9\(Ep\leftarrow\mathcal{E}_{s}.pop()\); \(\triangleright\) Pop one episode from \(\mathcal{E}_{s}\)
10\(\mathcal{D}\leftarrow\mathcal{D}\cup Ep\); \(\triangleright\) Append the episode to the PER buffer \(\mathcal{L}_{m}\leftarrow\mathcal{T}_{m}\cup Ep_{l}\); \(\triangleright\) Append the task of \(Ep\) and the result of \(Ep\) to \(\mathcal{L}_{m}\)
11 end if
12 if\(\mathcal{L}_{m}\) contains \(m\) tasks and task-results then
13\(f_{\pi}\leftarrow Train(f_{\pi},\mathcal{L}_{m})\); \(\triangleright\) Train \(f_{\pi}\)
14\(\mathcal{L}_{m}\leftarrow\emptyset\)
15 end if
16 end for
17\(\mathcal{B},w_{i}\gets sample(\mathcal{D},b)\); \(\triangleright\) Sample a batch of interactions from the PER replay buffer
18 for each iteration in \(\mathcal{B}\)do
19 foreach gradient stepdo
20\(\theta_{i}\leftarrow\theta_{i}-\lambda_{\mathcal{Q}}\hat{\
abla}_{\theta_{i}}w_{ i}I_{Q}(\theta_{i})\) for \(i\in\{1,2\}\); \(\triangleright\) Update the Q-function parameters
21\(\phi\leftarrow\phi-\lambda_{\pi}\hat{\
abla}_{\phi}f_{\pi}(\phi)\); \(\triangleright\) Update policy weights \(\alpha\leftarrow\alpha-\lambda\hat{\
abla}_{\alpha}J(\alpha)\); \(\triangleright\) Adjust Temperature
22 end for
23
24 end for
25\(n\_updates\gets n\_updates+1\);
26 if\(n\_updates\)\(\%\)\(\eta=0\)then
27\(\overline{\theta_{i}}\leftarrow\theta_{i}\) for \(\{1,2\}\); \(\triangleright\) Hard Update since \(\tau=1\)
28 end if
29
30 end while
```

**Algorithm 2**Distributed Soft Actor-Critic--Master Process. ## Appendix B Details for Training the _NavACL-Q_ Algorithm

Here, we show the hyper-parameters of _NavACL-Q_ algorithm. The success prediction network \(f_{\pi}\) consists of two dense hidden layers with 32 nodes each and the ReLU ([PERSON] et al., 2016) as non-linear activation function. We use a sigmoid function for the output layer to limit the output-range to \([0,1]\) together with binary entropy loss. Relevant details are listed in Table 1.

## Appendix C Arena Randomization

In this part, we show the randomization for each training arena cells including the initial pose of mobile robot and the target dolly in Table 14. For instance, the initial Yaw-Rotation (either robot or dolly) of \(0^{\circ}\) corresponds to alignment with the y-axis illustrated by Figure 6 (frontal robot camera heads towards the dolly), and \(-90^{\circ}\) corresponds to alignment with the x-axis (frontal camera points towards the right wall).