## 3 Results
### Analysis of Actor and Value Network Differences
\begin{table}
\begin{tabular}{c c c c c c} \hline  & Alien \#1 & Alien \#2 & Alien \#3 & Alien \#4 & Alien \#5 \\ \hline Alien \#1 & 5.062 (13.643) & 4.566 (20.929) & 4.126 (21.149) & 3.962 (21.101) & 4.096 (21.242) \\ Alien \#2 & 4.426 (19.533) & 4.855 (18.012) & 3.631 (19.554) & 3.677 (19.447) & 3.610 (19.559) \\ Alien \#3 & 5.524 (15.813) & 4.156 (14.448) & 3.414 (14.492) & 3.787 (14.483) & 3.670 (14.567) \\ Alien \#4 & 4.288 (11.267) & 4.177 (11.085) & 3.442 (10.821) & 3.929 (8.413) & 3.775 (10.457) \\ Alien \#5 & 3.610 (11.117) & 4.304 (11.979) & 2.804 (10.900) & 3.355 (11.073) & 3.093 (10.593) \\ \hline  & Boxing \#1 & Boxing \#2 & Boxing \#3 & Boxing \#4 & Boxing \#5 \\ \hline Boxing \#1 & 0.302 (0.318) & 0.596 (0.555) & 0.492 (0.563) & 0.482 (0.543) & 0.389 (0.516) \\ Boxing \#2 & 0.806 (0.756) & 0.278 (0.281) & 0.398 (0.519) & 0.282 (0.443) & 0.239 (0.415) \\ Boxing \#3 & 0.887 (0.785) & 0.937 (0.802) & 0.211 (0.220) & 0.578 (0.571) & 0.425 (0.512) \\ Boxing \#4 & 0.780 (0.640) & 0.375 (0.342) & 0.339 (0.327) & 0.165 (0.211) & 0.181 (0.227) \\ Boxing \#5 & 0.777 (0.683) & 0.395 (0.384) & 0.352 (0.376) & 0.202 (0.274) & 0.178 (0.245) \\ \hline  & Breakout \#1 & Breakout \#2 & Breakout \#3 & Breakout \#4 & Breakout \#5 \\ \hline Breakout \#1 & 0.492 (4.030) & 0.496 (4.823) & 0.360 (1.212) & 0.510 (1.231) & 0.510 (2.710) \\ Breakout \#2 & 0.299 (3.824) & 0.331 (4.800) & 0.173 (0.858) & 0.165 (0.635) & 0.224 (2.645) \\ Breakout \#3 & 0.273 (4.810) & 0.283 (5.625) & 0.126 (0.699) & 0.098 (0.758) & 0.110 (2.346) \\ Breakout \#4 & 0.406 (4.518) & 0.527 (5.628) & 0.488 (0.842) & 0.213 (0.868) & 0.313 (2.500) \\ Breakout \#5 & 0.275 (4.896) & 0.324 (5.904) & 0.125 (0.808) & 0.108 (0.819) & 0.117 (2.402) \\ \hline  & Enduro \#1 & Enduro \#2 & Enduro \#3 & Enduro \#4 & Enduro \#5 \\ \hline Enduro \#1 & 0.365 (0.556) & 0.375 (0.543) & 0.362 (0.535) & 0.384 (0.544) & 0.369 (0.516) \\ Enduro \#2 & 0.362 (0.556) & 0.368 (0.541) & 0.356 (0.534) & 0.383 (0.548) & 0.365 (0.513) \\ Enduro \#3 & 0.346 (0.524) & 0.358 (0.520) & 0.341 (0.504) & 0.366 (0.518) & 0.354 (0.496) \\ Enduro \#4 & 0.365 (0.569) & 0.377 (0.559) & 0.362 (0.545) & 0.380 (0.551) & 0.369 (0.530) \\ Enduro \#5 & 0.358 (0.530) & 0.366 (0.521) & 0.351 (0.509) & 0.376 (0.527) & 0.357 (0.493) \\ \hline  & Freeway \#1 & Freeway \#2 & Freeway \#3 & Freeway \#4 & Freeway \#5 \\ \hline Freeway \#1 & 0.014 (0.018) & 0.015 (0.019) & 0.014 (0.018) & 0.014 (0.017) & 0.011 (0.018) \\ Freeway \#2 & 0.014 (0.019) & 0.015 (0.019) & 0.014 (0.018) & 0.014 (0.018) & 0.011 (0.019) \\ Freeway \#3 & 0.015 (0.019) & 0.015 (0.019) & 0.014 (0.019) & 0.014 (0.018) & 0.011 (0.018) \\ Freeway \#4 & 0.014 (0.018) & 0.015 (0.018) & 0.014 (0.018) & 0.014 (0.017) & 0.011 (0.017) \\ Freeway \#5 & 0.015 (0.019) & 0.015 (0.019) & 0.014 (0.018) & 0.015 (0.018) & 0.012 (0.019) \\ \hline  & KFM \#1 & KFM \#2 & KFM \#3 & KFM \#4 & KFM \#5 \\ \hline KFM \#1 & 21.380 (40.486) & 21.578 (39.371) & 21.223 (38.835) & 20.854 (37.581) & 22.033 (40.885) \\ KFM \#2 & 21.788 (41.577) & 21.903 (40.504) & 21.573 (40.012) & 21.547 (38.781) & 22.244 (41.771) \\ KFM \#3 & 20.385 (37.923) & 20.541 (37.465) & 20.302 (35.906) & 19.908 (34.719) & 20.512 (37.669) \\ KFM \#4 & 21.799 (41.559) & 21.873 (40.700) & 21.747 (40.639) & 21.488 (39.138) & 22.458 (42.314) \\ KFM \#5 & 14.926 (28.450) & 15.255 (32.852) & 15.377 (28.981) & 14.786 (28.030) & 15.571 (30.718) \\ \hline  & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\ \end{tabular}
\end{table} 
Table 6: Mean and standard deviation of absolute TD-error (value network cross-evaluation).