Table 5: Raw scores across all 55 games, starting with 30 no-op actions. We report the best scores for DQN, QR-DQN, IQN and Rainbow on 50M frames, averaged by 5 seeds. Reference values were provided by DQN Zoo framework [29]. **Bold are wins against DQN, QR-DQN and IQN, and *asterisk are wins over Rainbow.
GAMES	RANDOM	HUMAN	DQN(50M)	QR-DQN(50M)	IQN(50M)	RAINBOW(50M)	PQR(50M)
Alien	227.8	7127.7	1541.5	1645.7	1769.2	4356.9	2455.8
Amidar	5.8	1719.5	324.2	683.4	799.2	2549.2	938.4
Assault	222.4	742.0	2387.8	11684.2	15152.4	9737.0	10759.2
Asterix	210.0	8503.3	5249.5	18373.4	32598.2	33378.6	10490.5
Asteroids	719.1	47388.7	1106.3	1503.9	1972.6	1825.4	1662.0
Atlantis	12850.0	29028.1	283392.2	937275.0	865360.0	941740.0	897640.0
BankHeist	14.2	753.1	389.0	1223.9	1266.8	1081.7	1038.8
BattleZone	2360.0	37187.5	19092.4	26325.0	30253.9	35467.1	28470.5
BeamRider	363.9	16926.5	7133.1	12912.0	19251.4	15421.9	10224.9
Berzerk	123.7	2630.4	577.4	826.5	918.9	2061.6	*137873.1
Bowling	23.1	160.7	34.4	45.4	41.5	54.7	*86.9
Boxing	0.1	12.1	87.2	99.6	99.2	99.8	97.1
Breakout	1.7	30.5	316.8	426.5	468.0	335.3	380.3
Centipede	2090.9	12017.0	4935.7	7124.0	7008.3	5691.4	*7291.2
ChopperCommand	811.0	7387.8	974.2	1187.8	1549.0	5525.1	1300.0
CrazyClimber	10780.5	35829.4	96939.0	93499.1	127156.5	160757.7	84390.9
DemonAttack	152.1	1971.0	8325.6	106401.8	110773.1	85776.5	73794.0
DoubleDunk	-18.6	-16.4	-15.7	-10.5	-12.1	-0.3	-7.5
Enduro	0.0	860.5	750.6	2105.7	2280.6	2318.3	*2341.2
FishingDerby	-91.7	-38.7	8.2	25.7	23.4	35.5	31.7
Freeway	0.0	29.6	24.4	33.3	33.7	34.0	34.0
Frostbite	65.2	4334.7	408.2	3859.2	5650.8	9672.6	4148.2
Gopher	257.6	2412.5	3439.4	6561.9	26768.9	32081.3	*47054.5
Gravitar	173.0	3351.4	180.9	548.1	470.2	2236.8	635.8
Hero	1027.0	30826.4	9948.3	9909.8	12491.1	38017.9	12579.2
IceHockey	-11.2	0.9	-11.4	-2.1	-4.2	1.9	-1.4
Jamesbond	29.0	302.8	486.4	1163.8	1058.0	14415.5	2121.8
Kangaroo	52.0	3035.0	6720.7	14558.2	14256.0	14383.6	*14617.1
Krull	1598.0	2665.5	7130.5	9612.5	9616.7	8328.5	*9746.1
KungFuMaster	258.5	22736.3	21330.9	27764.3	39450.1	30506.9	*43258.6
MontezumaRevenge	0.0	4753.3	0.3	0.0	0.2	80.0	0.0
MsPacman	307.3	6951.6	2362.9	2877.5	2737.4	3703.4	2928.9
NameThisGame	2292.3	8049.0	6328.0	11843.3	11582.2	11341.5	10298.2
Phoenix	761.4	7242.6	10153.6	35128.6	29138.9	49138.8	20453.8
Pitfall	-229.4	6463.7	-9.5	0.0	0.0	0.0	0.0
Pong	-20.7	14.6	18.7	20.9	20.9	21.0	21.0
PrivateEye	24.9	69571.3	266.6	100.0	100.0	160.0	*372.4
Qbert	163.9	13455.0	5567.9	12808.4	15101.8	24484.9	15267.4
Riverraid	1338.5	17118.0	6782.8	9721.9	13555.9	17522.9	11175.3
RoadRunner	11.5	7845.0	29137.5	54276.3	53850.9	52222.6	50854.7
Robotank	2.2	11.9	31.4	54.5	53.8	64.5	60.3
Seaquest	68.4	42054.7	2525.8	7608.2	17085.6	3048.9	*19652.5
Skiing	-17098.1	-4336.9	-13930.8	-14589.7	-19191.1	-15232.3	*-9299.3
Solaris	1236.3	12326.7	2031.5	1857.3	1301.5	2522.6	*2640.0
SpaceInvaders	148.0	1668.7	1179.1	1753.2	2906.7	2715.3	1749.4
StarGunner	664.0	10250.0	24532.5	63717.3	78503.4	107177.8	62920.6
Tennis	-23.8	-8.3	-0.9	0.0	0.0	0.0	-1.0
TimePilot	3568.0	5229.2	2091.8	6266.8	6379.1	12082.1	6506.4
Tutankham	11.4	167.6	138.7	210.2	204.4	194.3	*231.3
UpNDown	533.4	11693.2	6724.5	27311.3	35797.6	65174.2	36008.1
Venture	0.0	1187.5	53.3	12.5	17.4	1.1	*993.3
VideoPinball	16256.9	17667.9	140528.4	104405.8	341767.5	465636.5	465578.3
WizardOfWor	563.5	4756.5	3459.9	14370.2	10612.1	12056.1	6132.8
YarsRevenge	3092.9	54576.9	16433.7	21641.4	21645.0	67893.3	27674.4
Zaxxon	32.5	9173.3	3244.9	9172.1	8205.2	22045.8	10806.6