SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We also conduct an experiment using the grid-world problem as in Wachi and Sui [41]. Experimental settings are based on their original implementation (https://github.com/akifumi-wachi-4/safe_near_optimal_mdp). We consider a  $20 \times 20$  square grid in which reward and safety functions are randomly generated. There are two types regarding the safety threshold: one is time-invariant as in [41] and the other is time-variant as in the GSE problem.
We run SNO-MDP [41] and MASE in 100 randomly generated environments, and we compute the reward collected by the algorithms and count the number of episodes in which the safety constraint is violated at least once. The reward is normalized with respect to that by SNO-MDP.
Table 3: Experimental results for grid-world experiments.
TIME-INVARIANT SAFETY THRESHOLD	TIME-INVARIANT SAFETY THRESHOLD
REWARD	SAFETY VIOLATION	REWARD	SAFETY VIOLATION
SNO-MDP [41]	1.0 ± 0.0	0	1.0 ± 0.0	87
MASE (OURS)	1.0 ± 0.0	0	2.4 ± 1.0	0
The experimental results are shown in Table 3. When the safety threshold is time-invariant, MASE behaves identically with SNO-MDP; thus, the performance of the MASE is comparable with that of SNO-MDP. When the safety threshold is time-variant, SNO-MDP cannot deal with it by nature; hence, the safety constraint is not satisfied in most of the episodes. In contrast, our MASE satisfies the safety constraint in every episode, which also contributes to the larger reward.