SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 6: Hyper-parameters for DQN / SUGARL-DQN (on Atari)
Total steps	1,000,000 or 5,000,000
Replay buffer size	100,000
ε start	1.0
ε end	0.01
min ε step	100,000
γ	0.99
Learning start	80,000
Q network train frequency	4
Target network update frequency	1,000
Learning rate	10-4
Batch size	32
Self-understanding module train frequency	4
Self-understanding module learning rate	10-4
Table 7: Hyper-parameters for SAC (on Atari)
Total steps	1,000,000
Replay buffer size	100,000
γ	0.99
Learning start	80,000
Actor train frequency	4
Critic train frequency	4
Target network update frequency	8,000
Actor Learning rate	3 × 10-4
Critic Learning rate	3 × 10-4
Batch size	64
Self-understanding module train frequency	4
Self-understanding module learning rate	3 × 10-4
Visual policy alpha	0.2
Physical policy alpha	autotune
Physical policy target entropy scale	0.2
Table 8: Hyper-parameters for DrQv2 (on DMC)
Total steps	100,000
Replay buffer size	100,000
γ	0.99
Standard deviation start	1.0
Standard deviation end	0.1
Standard deviation end step	50,000
Standard deviation clip	0.3
Learning start	2,000
Actor train frequency	2
Critic train frequency	2
Target network update frequency	2
Target network exponential moving average weight	0.01
Actor Learning rate	10-4
Critic Learning rate	10-4
Batch size	256
Self-understanding module train frequency	2
Self-understanding module learning rate	10-4
Multiple-step reward	3
Table 9: Hyper-parameters for DrQv2 (on Robosuite)
Total steps	100,000
Replay buffer size	100,000
γ	0.99
Standard deviation start	1.0
Standard deviation end	0.1
Standard deviation end step	50,000
Standard deviation clip	0.3
Learning start	8,000
Actor train frequency	2
Critic train frequency	2
Target network update frequency	2
Target network exponential moving average weight	0.01
Actor Learning rate	10-4
Critic Learning rate	10-4
Batch size	256
Self-understanding module train frequency	2
Self-understanding module learning rate	10-4
Multiple-step reward	3
Table 10: Environment Settings
Atari
Gray-scale	True
Full observation size	84x84
Frame stacking	4
Action repeat (frame skipping)	4
Observable area initial location	(0,0)
Sensory action options	4 × 4 grid
Sensory action space size	16 (abs) or 5 (rel)
PVM number of steps	3
DMC
Gray-scale	True
Full observation size	84x84
Frame stacking	3
Action repeat (frame skipping)	2
Observable area initial location	(0,0)
Sensory action options	4 × 4 grid
Sensory action space size	5 (rel)
PVM number of steps	3
Robosuite
Gray-scale	False
Full observation size	84x84
Frame stacking	3
Action repeat (frame skipping)	2
Observable area initial location	Side View
Sensory action space	continuous relative 5-DoF control
PVM number of steps	3