SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 10: GROOVE Min-Atar performance after Grid-World meta-training, using random, expert, A2C and PPO agents as the algorithmic regret antagonist—mean return over 10 random seeds is marked, with standard error shaded.
To further investigate this result, we evaluated PPO and A2C on both random and difficult, handcrafted Grid-World levels. PPO achieves lower performance than A2C on Grid-World, with a larger gap on difficult, handcrafted Grid-World levels. This explains the previous results, as PPO will be inferior at identifying difficult levels when used as the AR antagonist. Furthermore, the update parameterized by LPG is capable of representing A2C, but not PPO. This implies that levels solvable by A2C should also be solvable by LPG, making them useful for training. In contrast, PPO may identify levels that cannot be solved without components found in PPO (clipping, mini-batch iterations) but not LPG.
Figure 11: A2C and PPO training curves on random and handcrafted Grid-World levels—we observe a larger performance gap on harder, handcrafted levels (10 seeds).