## 3 Results
### Impact of Identical Network Initialization  
To test whether performance variability stems solely from different random initializations or from the training process itself, we ran experiments where all five agents in an environment started with the same network weights. The results show that performance variance, while sometimes reduced, remains substantial in high-variance environments (Table 11). In Boxing, scores still ranged from 100 down to 90.97. In Qbert, the gap was even larger, from 4050 down to 800. In contrast, Freeway remained highly consistent. Crucially, the observation space and network divergence metrics for these agents mirrored the results from the randomly initialized experiments. For instance, in high-variance environments like Boxing and Qbert, the single-agent dispersion statistics showed considerable spread between agents, unlike in Freeway where all agents explored consistently (Table 12). This divergence in experience led to functionally different networks; the cross-agent absolute TD-error was substantial in high-variance games but negligible in Freeway (Table 13). This pattern of divergence is also confirmed by the high pairwise dissimilarity (high TV and MMD values, see Tables 14 and 15) and network error metrics (high RMSE and KLD values, see Tables 16 and 17). The key finding is that the stochasticity of the agent-environment interaction loop itself is a powerful driver of divergence. Even from an identical starting point, chance exploration events can set agents on different learning pathways from which they do not recover, leading to the same pattern of specialized, non-transferable policies and significant performance gaps. \begin{table}
\begin{tabular}{c c c c c c} \hline
**Task** & **\#1** & **\#2** & **\#3** & **\#4** & **\#5** \\ \hline Alien & 1456.00 (1065.92) & 1079.33 (211.80) & 1075.00 (425.80) & 1070.33 (325.79) & 916.33 (631.83) \\ Boxing & 100.00 (0.00) & 93.77 (2.50) & 93.77 (2.60) & 93.03 (4.53) & 90.97 (3.43) \\ Freeway & 21.43 (1.31) & 21.33 (1.35) & 21.30 (1.37) & 21.27 (1.29) & 21.13 (1.12) \\ Qbert & 4050.00 (0.00) & 4010.00 (48.99) & 958.33 (221.11) & 873.33 (44.22) & 800.00 (0.00) \\ \hline \end{tabular}
\end{table} 
Table 11: Performance of trained agents with identically initialized networks (mean and standard deviation of rewards). \begin{table}
\begin{tabular}{c c c c c} \hline
**Agent** & **\(H\)** & **N\({}_{eff}\)** & **Cov** & **\(G\)** \\ \hline Alien \#1 & 3.84 & 46.55 & 0.66 & 0.97 \\ Alien \#2 & 3.46 & 31.88 & 0.77 & 0.93 \\ Alien \#3 & 3.42 & 30.48 & 0.60 & 0.95 \\ Alien \#4 & 3.71 & 40.69 & 0.76 & 0.97 \\ Alien \#5 & 3.72 & 41.14 & 0.77 & 0.96 \\ \hline Boxing \#1 & 2.77 & 15.89 & 0.28 & 0.92 \\ Boxing \#2 & 3.35 & 28.45 & 0.49 & 0.96 \\ Boxing \#3 & 3.83 & 46.28 & 0.65 & 0.98 \\ Boxing \#4 & 3.04 & 20.84 & 0.40 & 0.95 \\ Boxing \#5 & 3.82 & 45.46 & 0.67 & 0.97 \\ \hline Freeway \#1 & 4.55 & 94.53 & 0.99 & 0.99 \\ Freeway \#2 & 4.54 & 93.97 & 1.00 & 0.99 \\ Freeway \#3 & 4.54 & 93.89 & 1.00 & 0.99 \\ Freeway \#4 & 4.54 & 93.80 & 1.00 & 0.99 \\ Freeway \#5 & 4.55 & 94.28 & 1.00 & 0.99 \\ \hline Qbert \#1 & 3.08 & 21.82 & 0.43 & 0.93 \\ Qbert \#2 & 3.29 & 26.77 & 0.47 & 0.94 \\ Qbert \#3 & 2.92 & 18.46 & 0.31 & 0.93 \\ Qbert \#4 & 2.93 & 18.74 & 0.38 & 0.92 \\ Qbert \#5 & 3.04 & 20.90 & 0.37 & 0.93 \\ \hline \end{tabular}
\end{table} 
Table 12: Single agent dispersion statistics of trained agents with identically initialized networks.