## 3 Results
### Analysis of Actor and Value Network Differences
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & Alien \#1 & Alien \#2 & Alien \#3 & Alien \#4 & Alien \#5 \\ \hline Alien \#1 & 0.000 & 3.640 (2.879) & 2.441 (3.047) & 3.451 (2.731) & 4.730 (4.081) \\ Alien \#2 & 3.251 (3.140) & 0.000 & 2.253 (2.433) & 2.831 (2.740) & 2.213 (2.185) \\ Alien \#3 & 2.105 (2.308) & 2.308 (2.391) & 0.000 & 3.250 (2.959) & 4.702 (4.060) \\ Alien \#4 & 2.916 (2.809) & 2.705 (3.039) & 2.899 (3.103) & 0.000 & 3.208 (3.014) \\ Alien \#5 & 3.262 (3.063) & 2.092 (2.352) & 3.758 (4.141) & 2.801 (2.476) & 0.000 \\ \hline  & Boxing \#1 & Boxing \#2 & Boxing \#3 & Boxing \#4 & Boxing \#5 \\ \hline Boxing \#1 & 0.000 & 9.609 (4.661) & 6.428 (4.556) & 6.798 (3.950) & 4.989 (3.495) \\ Boxing \#2 & 10.185 (7.840) & 0.000 & 6.217 (5.017) & 4.573 (5.023) & 4.514 (5.009) \\ Boxing \#3 & 12.033 (8.884) & 7.685 (5.039) & 0.000 & 6.610 (3.730) & 6.582 (4.077) \\ Boxing \#4 & 10.883 (7.330) & 5.016 (4.726) & 5.804 (3.565) & 0.000 & 2.552 (3.602) \\ Boxing \#5 & 10.436 (7.049) & 4.731 (4.573) & 5.519 (3.862) & 1.836 (2.045) & 0.000 \\ \hline  & Breakout \#1 & Breakout \#2 & Breakout \#3 & Breakout \#4 & Breakout \#5 \\ \hline Breakout \#1 & 0.000 & 0.496 (0.975) & 1.798 (2.181) & 0.602 (1.265) & 1.297 (1.968) \\ Breakout \#2 & 1.065 (1.537) & 0.000 & 0.782 (0.929) & 0.469 (0.818) & 1.656 (1.960) \\ Breakout \#3 & 0.423 (0.759) & 0.457 (0.577) & 0.000 & 0.243 (0.917) & 0.363 (0.681) \\ Breakout \#4 & 0.968 (1.415) & 0.631 (1.055) & 0.667 (0.937) & 0.000 & 0.991 (1.439) \\ Breakout \#5 & 0.367 (0.828) & 0.426 (0.576) & 0.163 (0.644) & 0.187 (0.803) & 0.000 \\ \hline  & Enduro \#1 & Enduro \#2 & Enduro \#3 & Enduro \#4 & Enduro \#5 \\ \hline Enduro \#1 & 0.000 & 0.709 (0.862) & 0.688 (0.775) & 0.668 (0.863) & 0.790 (0.881) \\ Enduro \#2 & 0.737 (0.840) & 0.000 & 0.732 (0.854) & 0.685 (0.822) & 0.760 (0.902) \\ Enduro \#3 & 0.679 (0.810) & 0.752 (0.998) & 0.000 & 0.642 (1.018) & 0.646 (0.953) \\ Enduro \#4 & 0.657 (0.828) & 0.667 (0.834) & 0.649 (0.807) & 0.000 & 0.682 (0.968) \\ Enduro \#5 & 0.782 (0.839) & 0.735 (0.883) & 0.630 (0.686) & 0.676 (0.806) & 0.000 \\ \hline  & Freeway \#1 & Freeway \#2 & Freeway \#3 & Freeway \#4 & Freeway \#5 \\ \hline Freeway \#1 & 0.000 & 0.002 (0.000) & 0.000 (0.000) & 0.003 (0.000) & 0.124 (0.005) \\ Freeway \#2 & 0.001 (0.000) & 0.000 & 0.001 (0.000) & 0.003 (0.000) & 0.109 (0.004) \\ Freeway \#3 & 0.000 (0.000) & 0.001 (0.000) & 0.000 & 0.002 (0.001) & 0.108 (0.013) \\ Freeway \#4 & 0.002 (0.000) & 0.002 (0.000) & 0.002 (0.000) & 0.000 & 0.040 (0.001) \\ Freeway \#5 & 0.008 (0.000) & 0.007 (0.000) & 0.007 (0.001) & 0.003 (0.000) & 0.000 \\ \hline  & KFM \#1 & KFM \#2 & KFM \#3 & KFM \#4 & KFM \#5 \\ \hline KFM \#1 & 0.000 & 0.042 (0.022) & 0.035 (0.036) & 0.051 (0.073) & 0.042 (0.025) \\ KFM \#2 & 0.043 (0.030) & 0.000 & 0.039 (0.021) & 0.022 (0.029) & 0.088 (0.032) \\ KFM \#3 & 0.039 (0.038) & 0.045 (0.027) & 0.000 & 0.039 (0.037) & 0.037 (0.030) \\ KFM \#4 & 0.060 (0.111) & 0.024 (0.037) & 0.046 (0.052) & 0.000 & 0.093 (0.103) \\ KFM \#5 & 0.080 (0.031) & 0.142 (0.055) & 0.034 (0.027) & 0.064 (0.050) & 0.000 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Mean and standard deviation of KL divergence of action logits (actor network cross-evaluation).