Table 8: Continuous control with the best pessimism hyperparameters adjusted per suite after 1M timesteps
Suite/Task	Score at 1M timesteps (IQM ± standard deviation where appropriate) with best pessimism
DoubleGum	MoG-DDPG	DDPG/TD3	SAC	XQL	QR-DDPG	FinerTD3
acrobot-swingup	330.2 ± 79.44	364.9 ± 71.75	334.7 ± 96.92	226.6 ± 134.5	356.7 ± 88.26	225.7 ± 131.1	283.3 ± 89.79
reach-hard	979.4 ± 54.28	976.9 ± 4.985	975.7 ± 26.78	976.2 ± 56.58	972.9 ± 14.86	975.4 ± 13.21	976.3 ± 27.78
finger-turn-hard	931.8 ± 98.91	935.4 ± 77.4	909.1 ± 50.68	972.4 ± 25.53	924.3 ± 73.62	952.6 ± 65.92	956.4 ± 74.46
hopper-hop	306.7 ± 101	313.3 ± 99.74	305.7 ± 80.12	0.007651 ± 56.18	304.3 ± 69.23	346.8 ± 105	519.5 ± 47.02
fish-swim	675.7 ± 58.5	758.1 ± 62.68	710.2 ± 94.54	616.4 ± 226.4	695.2 ± 54.08	708.8 ± 38.73	761.6 ± 41.15
cheetah-run	883.1 ± 22.51	844.9 ± 53.54	804.5 ± 64.31	764.5 ± 74.55	785.9 ± 63.43	791.1 ± 69.23	699 ± 84.17
walker-run	783.6 ± 26.97	778.7 ± 21.86	755.6 ± 28.29	732 ± 199.4	743.9 ± 29.61	759.8 ± 24.46	777 ± 12.7
quadruped-run	835.1 ± 65.11	818.2 ± 76.37	736 ± 87.58	774 ± 62.96	772.4 ± 80.69	775 ± 54.76	724.4 ± 92.74
swimmer-swimmer15	608.9 ± 137.8	623.4 ± 81.98	531.5 ± 104.1	437.8 ± 192.5	499.8 ± 64.12	552.7 ± 140.2	560.2 ± 106.4
humanoid-run	142.5 ± 12.6	96.97 ± 12.58	119.6 ± 25.09	0.9647 ± 0.1496	117.8 ± 17.68	122.4 ± 9.644	85.08 ± 6.825
dog-run	187.1 ± 13.14	138.8 ± 20.58	177.4 ± 12.17	22.63 ± 56.75	155.5 ± 22.55	145.3 ± 28.37	87.51 ± 32.47
Hopper-v4	3290 ± 829.1	1211 ± 319.2	2589 ± 939.2	942 ± 135.3	2398 ± 946.4	94.89 ± 122.7	2183 ± 890.6
HalfCheetah-v4	9874 ± 997.4	9398 ± 1044	10020 ± 1390	7171 ± 1000	9855 ± 1323	9694 ± 1571	10420 ± 328.8
Walker2d-v4	4871 ± 403.7	2196 ± 1127	3868 ± 593.1	2894 ± 1109	4027 ± 962	152.7 ± 79.47	4276 ± 357.1
Ant-v4	5681 ± 416.8	4644 ± 966.2	5645 ± 914	5908 ± 811.3	6185 ± 168.3	3652 ± 1685	6048 ± 464.4
Humanoid-v4	5565 ± 160.7	1715 ± 783.4	5241 ± 302.4	5286 ± 703.2	5452 ± 243.1	160.2 ± 32.99	5668 ± 339.5
button-press-v2	1436 ± 1241	919.6 ± 1066	1093 ± 1016	645.5 ± 728.3	1424 ± 898.3	1506 ± 1047	1086 ± 758.2
door-open-v2	3671 ± 1606	4288 ± 1431	2691 ± 1415	3114 ± 1166	3818 ± 1258	3512 ± 1387	4268 ± 953.8
drawer-close-v2	4839 ± 1726	4706 ± 1578	3880 ± 1916	4749 ± 1339	4178 ± 1788	4588 ± 2042	3741 ± 2037
drawer-open-v2	2762 ± 1212	4074 ± 1421	2510 ± 1467	1710 ± 1514	2820 ± 1463	2972 ± 1382	1608 ± 1333
peg-insert-side-v2	1226 ± 1769	431.4 ± 1417	402.4 ± 1855	9.693 ± 232.9	339.8 ± 1211	100.8 ± 1462	1092 ± 1699
pick-place-v2	12.49 ± 110.5	212.5 ± 355	509.3 ± 672.7	4.821 ± 1.873	451.2 ± 847.2	26.68 ± 47.73	501.4 ± 963.9
push-v2	191.1 ± 1282	310.9 ± 1005	777.9 ± 928.9	90.81 ± 745	74.32 ± 847.7	76.38 ± 95.72	256.8 ± 453.4
reach-v2	1746 ± 1127	2889 ± 909.7	2992 ± 1466	2974 ± 1393	3068 ± 1231	2583 ± 1345	2402 ± 922.9
window-open-v2	2668 ± 1515	2501 ± 1764	3029 ± 1476	1957 ± 1832	2294 ± 1551	1672 ± 1813	2964 ± 1365
window-close-v2	4404 ± 712	4520 ± 214.5	4352 ± 342.3	4000 ± 1045	4022 ± 1191	4528 ± 232.3	4272 ± 260
basketball-v2	780.9 ± 1238	602.4 ± 1106	809.1 ± 901.4	9.932 ± 11.24	1972 ± 1197	99.83 ± 745.6	1868 ± 959.1
dial-turn-v2	1214 ± 952.6	1552 ± 1250	2385 ± 1486	1472 ± 1178	2157 ± 1465	1690 ± 1408	1101 ± 1255
sweep-into-v2	4237 ± 1348	1417 ± 1448	3007 ± 1677	247.9 ± 1718	2203 ± 2026	1008 ± 1782	2844 ± 1934
hammer-v2	4308 ± 1215	3500 ± 1344	2664 ± 1477	1180 ± 1650	3620 ± 1311	1892 ± 1387	4151 ± 1115
assembly-v2	2606 ± 660.4	1941 ± 1241	2265 ± 861.8	218.1 ± 492.3	2376 ± 1032	164 ± 401.5	2067 ± 1507
BipedalWalker-v3	315.2 ± 12.26	325.4 ± 8.892	321.2 ± 2.343	321.1 ± 21.67	311.3 ± 18.27	331.9 ± 6.717	322.1 ± 6.48
BipedalWalkerHardcore-v3	189 ± 86.95	152.6 ± 52.39	18.27 ± 45.01	-32.19 ± 56.1	-31.74 ± 18.9	114.1 ± 65.9	53.3 ± 40.58
DMC Aggregate	0.6191	0.6326	0.5823	0.4762	0.5918	0.5787	0.6121
MuJoCo Aggregate	0.8367	0.4663	0.7716	0.6304	0.1423	0.8061	0.8305
MetaWorld Aggregate	0.2505	0.193	0.1904	0.08515	0.1204	0.2017	0.2042
Box2D Aggregate	0.9391	0.8911	0.7204	0.6658	0.8659	0.601	0.7679
All Aggregate	0.4707	0.3709	0.3979	0.2719	0.2786	0.3924	0.4239