SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We add two baselines for performance comparison, the RMF (Recurrent Model-free) is a recent SOTA method on POMDP problems and the TD3-FPOW (TD3 with fixed previous observation window) is a modified version of TD3, which takes 3 concatenation frames as the input.
Table 6: Performance comparison
Environments	SLAC	VRM	SAC-LSTM	RMF	TD3-FPOW	Ours
Ant-P	950±129	1040±75	946±47	1048±74	974±43	1243±43
Ant-V	663±87	981±63	690±34	1021±165	903±54	998±93
Walker-P	277±121	1121±167	971±103	1123±176	1043±103	1487±81
Walker-V	138±25	551±30	491±38	586±52	482±32	612±27
Hopper-P	222±21	1851±61	1236±39	2133±326	1654±217	2455±87
Hopper-V	310±41	1652±67	890±43	1495 ± 38	1312±87	1545±104
The result shows that our method achieved the best performance in 4 out of 6 environments.
13 Additional ablation study
We study the influence of the shared/separate encoders, and different concatenations of inputs as the ablation study. Due to the time limitation, we conducted these experiments on the Walker-P and Walker-V environments.
Table 7: Performance comparison about shared/separate encoders
Environments	shared	separate
Walker-P	1336	1487
Walker-V	584	612
The result shows that separate context encoders for the policy network and value function improve the performance.
Table 8: Performance comparison different kinds of inputs
Environments	o	oa	or	oar
Walker-P	1276	1065	1396	1487
The result shows that taking the input  $x_{t}$  as the concatenation of  $o_{t}$ ,  $r_{t}$  and  $a_{t-1}$  for the context encoder (GRU-ODE) improves the performance.