Table 7: Continuous control with default pessimism hyperparameters score after 1M timesteps.
Suite/Task	Score at 1M timesteps (IQM ± standard deviation where appropriate)
DoubleGum	MoG-DDPG	DDPG	TD3	Twin-SAC	Twin-XQL	QR-DDPG	FinerTD3
acrobot-swingup	330.2 ± 79.44	364.9 ± 71.75	334.7 ± 96.92	7.315 ± 26.53	6.568 ± 6.164	38.84 ± 91.41	356.7 ± 88.26	8.114 ± 15.96
reach-hard	979.4 ± 54.28	976.9 ± 4.985	975.7 ± 26.78	975.9 ± 31.13	976.2 ± 27.21	975.1 ± 4.506	972.9 ± 14.86	972.4 ± 25.15
finger-turn-hard	931.8 ± 98.91	935.4 ± 77.4	909.1 ± 50.68	969.7 ± 28.51	951.5 ± 63.21	967.2 ± 59.48	924.3 ± 73.62	974.1 ± 24.9
hopper-hop	306.7 ± 101	313.3 ± 99.74	305.7 ± 80.12	123.4 ± 69.37	0.007008 ± 0.03587	135.9 ± 65.21	304.3 ± 69.23	116 ± 51.71
fish-swim	675.7 ± 58.5	758.1 ± 62.68	710.2 ± 94.54	517.9 ± 111.2	344.9 ± 251.5	632.7 ± 99.64	695.2 ± 54.08	675 ± 118.8
cheetah-run	883.1 ± 22.51	844.9 ± 53.54	804.5 ± 64.31	745 ± 52.78	708.8 ± 44.35	761.3 ± 56.24	785.9 ± 63.43	741.7 ± 43.59
walker-run	783.6 ± 26.97	778.7 ± 21.86	755.6 ± 28.29	696 ± 39.53	538.6 ± 311.1	705.5 ± 114.6	743.9 ± 29.61	731.4 ± 109.2
quadruped-run	835.1 ± 65.11	818.2 ± 76.37	736 ± 87.58	677.2 ± 169.6	677 ± 181.7	743.9 ± 100.6	772.4 ± 80.69	739.8 ± 117.1
swimmer-swimmer15	608.9 ± 137.8	623.4 ± 81.98	531.5 ± 104.1	477.8 ± 139.2	225.4 ± 130.5	435.5 ± 148.5	499.8 ± 64.12	612.5 ± 82.18
humanoid-run	142.5 ± 12.6	96.97 ± 12.58	119.6 ± 25.09	39.51 ± 62.78	0.8364 ± 0.234	119.2 ± 62.17	117.8 ± 17.68	1.452 ± 47.32
dog-run	187.1 ± 13.14	138.8 ± 20.58	177.4 ± 12.17	209.2 ± 25.22	5.226 ± 0.7772	229.5 ± 20.56	155.5 ± 22.55	234.4 ± 29.06
Hopper-v4	1399 ± 668.1	1211 ± 319.2	1348 ± 740.2	2589 ± 939.2	942 ± 135.3	94.89 ± 122.7	1641 ± 554.7	2183 ± 890.6
HalfChetah-v4	10710 ± 608.3	9398 ± 1044	9552 ± 1557	10020 ± 1390	7171 ± 1000	9694 ± 1571	9148 ± 1476	10420 ± 328.8
Walker2d-v4	4148 ± 1439	2196 ± 1127	1466 ± 470.3	3868 ± 593.1	2894 ± 1109	152.7 ± 79.47	2205 ± 890.8	4276 ± 357.1
Ant-v4	6046 ± 552.5	4644 ± 966.2	976.9 ± 302.6	5645 ± 914	5908 ± 811.3	3652 ± 1685	2245 ± 761.4	6048 ± 464.4
Humanoid-v4	5645 ± 904.1	1715 ± 783.4	2023 ± 604.9	5241 ± 302.4	5286 ± 703.2	160.2 ± 32.99	1814 ± 465.6	5668 ± 339.5
button-press-v2	1436 ± 1241	919.6 ± 1066	1093 ± 1016	635.5 ± 566	1366 ± 1133	1128 ± 886.1	1424 ± 898.3	835 ± 1190
door-open-v2	3671 ± 1606	4288 ± 1431	2691 ± 1415	3784 ± 809.1	4232 ± 454.2	3956 ± 941	3818 ± 1258	4083 ± 1059
drawing-close-v2	4839 ± 1726	4706 ± 1578	3880 ± 1916	4743 ± 1718	3039 ± 2206	4750 ± 1560	4178 ± 1788	4808 ± 1324
drawer-open-v2	2762 ± 1212	4074 ± 1421	2510 ± 1467	2951 ± 1511	2617 ± 1642	2256 ± 1819	2820 ± 1463	2048 ± 1187
peg-insert-side-v2	1226 ± 1769	431.4 ± 1417	402.4 ± 1855	432 ± 1191	7.378 ± 2.552	317.8 ± 579.6	339.8 ± 1211	2146 ± 1752
pick-place-v2	12.49 ± 110.5	212.5 ± 355	509.3 ± 672.7	7.501 ± 478.7	5.553 ± 2.761	16.98 ± 378	451.2 ± 847.2	6.942 ± 732.7
push-v2	191.1 ± 1282	310.9 ± 1005	777.9 ± 928.9	42.5 ± 930.5	25.99 ± 76.78	415.2 ± 971.5	74.32 ± 847.7	729.7 ± 1571
reach-v2	1746 ± 1127	2889 ± 909.7	2992 ± 1466	2092 ± 1304	3390 ± 1243	2552 ± 1066	3068 ± 1231	2735 ± 1113
window-open-v2	2668 ± 1515	2501 ± 1764	3029 ± 1476	3452 ± 1150	491.3 ± 1326	3916 ± 1023	2294 ± 1551	2786 ± 1488
window-close-v2	4404 ± 712	4520 ± 214.5	4352 ± 342.3	4202 ± 1145	4372 ± 704.6	4574 ± 866.3	4022 ± 1191	4291 ± 556
basketball-v2	780.9 ± 1238	602.4 ± 1106	809.1 ± 901.4	627.9 ± 648.4	9.246 ± 197.2	1126 ± 1670	1972 ± 1197	524.4 ± 702.6
dial-turn-v2	1214 ± 952.6	1552 ± 1250	2385 ± 1486	1228 ± 1206	2192 ± 1593	1446 ± 1020	2157 ± 1465	1258 ± 1034
sweep-into-v2	4237 ± 1348	1417 ± 1448	3007 ± 1677	1977 ± 1907	530.2 ± 1712	3650 ± 1591	2203 ± 2026	1496 ± 2042
hammer-v2	4308 ± 1215	3500 ± 1344	2664 ± 1477	1340 ± 1099	487.5 ± 945	2812 ± 1869	3620 ± 1311	2866 ± 1886
assembly-v2	2606 ± 660.4	1941 ± 1241	2265 ± 861.8	2085 ± 801.6	947.2 ± 983.3	439.6 ± 1429	2376 ± 1032	1528 ± 921.1
BipedalWalker-v3	315.2 ± 12.26	325.4 ± 8.892	218.6 ± 115.9	321.2 ± 2.343	321.1 ± 21.67	331.9 ± 6.717	247.4 ± 72.5	322.1 ± 6.48
BipedalWalkerHardcore-v3	189 ± 86.95	152.6 ± 52.39	-71.08 ± 30.12	18.27 ± 45.01	-32.19 ± 56.1	114.1 ± 65.9	-84.9 ± 17.95	53.3 ± 40.58
DeepMind Control Aggregate	0.6191	0.6326	0.5823	0.4588	0.293	0.5003	0.5787	0.5352
MuJoCo Aggregate	0.818	0.4663	0.3077	0.7716	0.6304	0.1423	0.4079	0.8305
MetaWorld Aggregate	0.209	0.193	0.1904	0.149	0.09027	0.1881	0.2017	0.1867
Box2D Aggregate	0.9391	0.8911	0.319	0.7204	0.6658	0.8659	0.3764	0.7679
All Aggregate	0.428	0.3709	0.3026	0.3358	0.226	0.2802	0.3255	0.39