SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 12: Reproductions of TD3+BC baseline for 8 MuJoCo tasks across 5 seeds for 1M gradient steps, showing that all our reported results (blue, dotted line) lie within the reproduced confidence intervals and most of the results reported in Liu and Abbeel [2023] (red, dotted line) do not.
Moreover, to attempt to reduce the possible effect of hyperparameters on the performance difference, we sweep 16 combinations of the number of actor and critic layers based on the original codebase used by the TD3+BC authors here: https://github.com/sfujim/TD3_BC. Note that their original implementation features a hardcoded 2 layer neural network for actor and critic. In 9, we show that no combination of hyperparameters for the number of actor and critic layers is sufficient to reach the performance reported by Liu and Abbeel [2023] and that as the number of layers increases beyond 3 or decreases beyond 2, the performance decreases on hopper-medium-replay-v2.
Table 9: Average normalized score of TD3+BC baseline across 5 seeds by varying number of actor  $(L_{A})$  and critic network hidden layers  $(L_{C})$  on hopper-medium-replay-v2.
LC
LA	1	2	3	4
1	48.7	80.1	73.1	73.9
2	55.2	64.4	50.2	61.3
3	59.8	59.1	69.6	64.6
4	42.9	60.5	63.6	54.4