SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The results of OPE on three tasks are in Tab. 8. Firstly, we can see that for all recorded rank-correlation scores, GALILEO is significantly better than all of the baseline methods, with at least  $25\%$  improvements. As for regret@1, although GALILEO cannot reach the best performance among all of the tasks, it is always one of the top-2 methods among the three tasks, which demonstrates the stability of GALILEO. Finally, GALILEO has the smallest value gaps except for Halfcheetah. However, for all of the top-4 methods, the value gaps in halfcheetah are around 1,200. Thus we believe that their performances on value gaps are roughly at the same level.
Table 8: Results of OPE on DOPE benchmark.  $\pm$  is the standard deviation. We bold the top-2 scores for each metric. The results are tested on 3 random seeds. The results of the baselines are from [18]. Note that the rank correlation is "NAN" for HalfCheetah because the scores are not given in [18].
TASK	HalfCheetah
METRIC	Absolute value gap	Rank correlation	Regret@1
GALILEO	1280 ± 83	0.313 ± 0.09	0.12 ± 0.02
Best DICE	1382 ± 130	NAN	0.82 ± 0.29
VPM	1374 ± 153	NAN	0.33 ± 0.19
FQE (L2)	1211 ± 130	NAN	0.38 ± 0.13
IS	1217 ± 123	NAN	0.05 ± 0.05
Doubly Rubost	1222 ± 134	NAN	0.37 ± 0.13
TASK	Walker2d
METRIC	Absolute value gap	Rank correlation	Regret@1
GALILEO	176 ± 52	0.57 ± 0.08	0.08 ± 0.06
Best DICE	273 ± 31	0.12 ± 0.38	0.27 ± 0.43
VPM	426 ± 60	0.44 ± 0.21	0.08 ± 0.06
FQE (L2)	350 ± 79	-0.09 ± 0.36	0.31 ± 0.10
IS	428 ± 60	-0.25 ± 0.35	0.70 ± 0.39
Doubly Rubost	368 ± 74	0.02 ± 0.37	0.25 ± 0.09
TASK	Hopper
METRIC	Absolute value gap	Rank correlation	Regret@1
GALILEO	156 ± 23	0.45 ± 0.1	0.08 ± 0.08
Best DICE	215 ± 41	0.19 ± 0.33	0.18 ± 0.19
VPM	433 ± 44	0.13 ± 0.37	0.10 ± 0.14
FQE (L2)	283 ± 73	-0.29 ± 0.33	0.32 ± 0.32
IS	405 ± 48	-0.55 ± 0.26	0.32 ± 0.32
Doubly Rubost	307 ± 73	-0.31 ± 0.34	0.38 ± 0.28
H.7 Detailed Results in the BAT Task
Since there is no oracle environment model for querying, we have to describe the results with other metrics.
(a) City-A
(b) City-B
(c) City-C
(d) City-D
(e) City-E
(f) City-F
Figure 18: Illustration of the response curves in the 6 cities. Although the ground-truth curves are unknown, through human expert knowledge, we know that it is expected to be monotonically increasing.
First, we review whether the tendency of the response curve is consistent. In this application, with a larger budget of allowance, the supply will not be decreased. As can be seen in Fig. 18, the tendency of GALILEO's response is valid in 6 cities but almost all of the models of SL give opposite directions to the response. If we learn a policy through the model of SL, the optimal solution is canceling all of the allowances, which is obviously incorrect in practice.
Second, we conduct randomized controlled trials (RCT) in one of the testing cities. Using the RCT samples, we can evaluate the correctness of the sort order of the model predictions via Area Under the Uplift Curve (AUUC) [6]. To plot AUUC, we first sort the RCT samples based on the predicted treatment effects. Then the cumulative treatment effects are computed by scanning the sorted sample list. If the sort order of the model predictions is better, the sample with larger treatment effects will be computed early. Then
the area of AUUC will be larger than the one via a random sorting strategy. The result of AUUC show GALILEO gives a reasonable sorting to the RCT samples (see Fig. 19).
Finally, we search for the optimal policy via the cross-entropy method planner [21] based on the learned model. We test the online supply improvement in 6 cities. The algorithm compared is a human-expert policy, which is also the behavior policy of the offline datasets. We conduct online
Figure 19: Illustration of the AUCC result for BAT. The model with larger areas above the "random" line makes better predictions in randomized-controlled-trials data [61].
A/B tests for each of the cities. For each test, we randomly split a city into two partitions, one is for deploying the optimal policy learned from the GALILEO model, and the other is as a control group, which keeps the human-expert policy as before. Before the intervention, we collect 10 days' observation data and compute the averaged five-minute order-taken rates as the baselines of the treatment and control group, named  $b^{t}$  and  $b^{c}$  respectively. Then we start intervention and observe the five-minute order-taken rate in the following 14 days for the two groups. The results of the treatment and control groups are  $y_{i}^{t}$  and  $y_{i}^{c}$  respectively, where  $i$  denotes the  $i$ -th day of the deployment. The percentage points of the supply improvement are computed via difference-in-difference (DID):
$$
\frac {\sum_ {i} ^ {T} \left(y _ {i} ^ {t} - b ^ {t}\right) - \left(y _ {i} ^ {c} - b ^ {c}\right)}{T} \times 1 0 0,
$$
where  $T$  is the total days of the intervention and  $T = 14$  in our experiments.
Table 9: Results on BAT. We use City-X to denote the experiments on different cities. "pp" is an abbreviation of percentage points on the supply improvement.
target city	City-A	City-B	City-C
supply improvement	+1.63pp	+0.79pp	+0.27pp
target city	City-D	City-E	City-F
supply improvement	+0.2pp	+0.14pp	+0.41pp
The results are summarized in Tab. 9. The online experiment is conducted in 14 days and the results show that the policy learned with GALILEO can make better (the supply improvements are from 0.14 to 1.63 percentage points) budget allocation than the behavior policies in all the testing cities. We give detailed results which record the supply difference between the treatment group and the control group in Fig. 20.
(f) City-F
Figure 20: Illustration of the daily responses in the A/B test in the 6 cities.
Table 10: The root mean square errors on MuJoCo tasks. We bold the lowest error for each task. "medium" dataset is used for training, while "expert" and "medium-replay" datasets are just used for testing.  $\pm$  follows the standard deviation of three seeds.
TASK	HalfCheetah
DATASET	medium (train)	expert (test)	medium-replay (test)
GALILEO	0.378 ± 0.003	2.287± 0.005	1.411 ± 0.037
OFF-SL	0.404 ± 0.001	3.311 ± 0.055	2.246 ± 0.016
IPW	0.513 ± 0.033	2.892 ± 0.050	2.058 ± 0.021
SCIGAN	0.309 ± 0.002	3.813 ± 0.133	2.484 ± 0.040
TASK	Walker2d
DATASET	medium (train)	expert (test)	medium-replay (test)
GALILEO	0.49 ± 0.001	1.514 ± 0.002	0.968 ± 0.004
OFF-SL	0.467 ± 0.004	1.825 ± 0.061	1.239 ± 0.004
IPW	0.564 ± 0.001	1.826 ± 0.025	1.282 ± 0.007
SCIGAN	0.438 ± 0.001	1.825 ± 0.031	1.196 ± 0.005
TASK	Hopper
DATASET	medium (train)	expert (test)	medium-replay (test)
GALILEO	0.037 ± 0.002	0.322 ± 0.036	0.408 ± 0.003
OFF-SL	0.034 ± 0.001	0.464 ± 0.021	0.574 ± 0.008
IPW	0.039 ± 0.001	0.533 ± 0.00	0.671 ± 0.001
SCIGAN	0.039 ± 0.002	0.628 ± 0.050	0.742 ± 0.019
Table 11:  $\sqrt{MISE}$  results on GNFC. We bold the lowest error for each task.  $\pm$  is the standard deviation of three random seeds.
e1_p1	e0.2_p1	e0.05_p1
GALILEO	5.17 ± 0.06	4.73 ± 0.13	4.70 ± 0.02
SL	5.15 ± 0.23	4.73 ± 0.31	23.64 ± 4.86
IPW	5.22 ± 0.09	5.50 ± 0.01	5.02 ± 0.07
SCIGAN	7.05 ± 0.52	6.58 ± 0.58	18.55 ± 3.50
e1_p0.2	e0.2_p0.2	e0.05_p0.2
GALILEO	5.03 ± 0.09	4.72 ± 0.05	4.87 ± 0.15
SL	5.21 ± 0.63	6.74 ± 0.15	33.52 ± 1.32
IPW	5.27 ± 0.05	5.69 ± 0.00	20.23 ± 0.45
SCIGAN	16.07 ± 0.27	12.07 ± 1.93	19.27 ± 10.72
e1_p0.05	e0.2_p0.05	e0.05_p0.05
GALILEO	5.23 ± 0.41	5.01 ± 0.08	6.17 ± 0.33
SL	5.89 ± 0.88	14.25 ± 3.48	37.50 ± 2.29
IPW	5.21 ± 0.01	5.52 ± 0.44	31.95 ± 0.05
SCIGAN	11.50 ± 7.76	13.05 ± 4.19	25.74 ± 8.30
Table 12:  $\sqrt{MISE}$  results on TCGA. We bold the lowest error for each task.  $\pm$  is the standard deviation of three random seeds.
t0.bias_2.0	t0.bias_20.0	t0.bias_50.0
GALILEO	0.34 ± 0.05	0.67 ± 0.13	2.04 ± 0.12
SL	0.38 ± 0.13	1.50 ± 0.31	3.06 ± 0.65
IPW	6.57 ± 1.16	6.88 ± 0.30	5.84 ± 0.71
SCIGAN	0.74 ± 0.05	2.74 ± 0.35	3.19 ± 0.09
t1.bias_2.0	t1.bias_6.0	t1.bias_8.0
GALILEO	0.43 ± 0.05	0.25 ± 0.02	0.21 ± 0.04
SL	0.47 ± 0.05	1.33 ± 0.97	1.18 ± 0.73
IPW	3.67 ± 2.37	0.54 ± 0.13	2.69 ± 1.17
SCIGAN	0.45 ± 0.25	1.08 ± 1.04	1.01 ± 0.77
t2.bias_2.0	t2.bias_6.0	t2.bias_8.0
GALILEO	1.46 ± 0.09	0.85 ± 0.04	0.46 ± 0.01
SL	0.81 ± 0.14	3.74 ± 2.04	3.59 ± 0.14
IPW	2.94 ± 1.59	1.24 ± 0.01	0.99 ± 0.06
SCIGAN	0.73 ± 0.15	1.20 ± 0.53	2.13 ± 1.75
Table 13:  $\sqrt{MMSE}$  results on GNFC. We bold the lowest error for each task.  $\pm$  is the standard deviation of three random seeds.
e1_p1	e0.2_p1	e0.05_p1
GALILEO	3.86 ± 0.03	3.99 ± 0.01	4.07 ± 0.03
SL	5.73 ± 0.33	5.80 ± 0.28	18.78 ± 3.13
IPW	4.02 ± 0.05	4.15 ± 0.12	22.66 ± 0.33
SCIGAN	8.84 ± 0.54	12.62 ± 2.17	24.21 ± 5.20
e1_p0.2	e0.2_p0.2	e0.05_p0.2
GALILEO	4.13 ± 0.10	4.11 ± 0.15	4.21 ± 0.15
SL	5.87 ± 0.43	7.44 ± 1.13	29.13 ± 3.44
IPW	4.12 ± 0.02	6.12 ± 0.48	30.96 ± 0.17
SCIGAN	12.87 ± 3.02	14.59 ± 2.13	24.57 ± 3.00
e1_p0.05	e0.2_p0.05	e0.05_p0.05
GALILEO	4.39 ± 0.20	4.34 ± 0.20	5.26 ± 0.29
SL	6.12 ± 0.43	14.88 ± 4.41	30.81 ± 1.69
IPW	13.60 ± 7.83	26.27 ± 2.67	32.55 ± 0.12
SCIGAN	9.19 ± 1.04	15.08 ± 1.26	17.52 ± 0.02
Table 14:  $\sqrt{MMSE}$  results on TCGA. We bold the lowest error for each task.  $\pm$  is the standard deviation of three random seeds.
t0.bias_2.0	t0.bias_20.0	t0.bias_50.0
GALILEO	1.56 ± 0.04	1.96 ± 0.53	3.16 ± 0.13
SL	1.92 ± 0.67	2.31 ± 0.19	5.11 ± 0.66
IPW	7.42 ± 0.46	5.36 ± 0.96	5.38 ± 1.24
SCIGAN	2.11 ± 0.47	5.23 ± 0.27	5.59 ± 1.02
t1.bias_2.0	t1.bias_6.0	t1.bias_8.0
GALILEO	1.43 ± 0.06	1.09 ± 0.05	1.36 ± 0.36
SL	1.12 ± 0.15	3.65 ± 1.91	3.96 ± 1.81
IPW	1.14 ± 0.11	0.90 ± 0.09	2.04 ± 0.99
SCIGAN	3.32 ± 0.88	4.74 ± 2.12	5.17 ± 2.42
t2.bias_2.0	t2.bias_6.0	t2.bias_8.0
GALILEO	3.77 ± 0.35	3.99 ± 0.40	2.08 ± 0.60
SL	2.70 ± 0.67	8.33 ± 5.05	9.70 ± 3.12
IPW	2.92 ± 0.15	3.90 ± 0.17	4.47 ± 2.16
SCIGAN	3.82 ± 2.12	1.83 ± 1.49	3.62 ± 4.9
(a) t0.bias2
(b) t0.bias20
(c) t0.bias50
(d) t1.bias2
(e) t1.bias6
(t) t1.bias8
(g) t2.bias2
(h) t2.bias6
(i) t2.bias8
Figure 21: Illustration of the averaged response curves of Supervised Learning (SL) in TCGA.
(a) e1_p1
(b) e0.2_p1
(c) e0.05_p1
(d) e1_p0.2
(e) e0.2_p0.2
(f) e0.05_p0.2
(g) e1_p0.05
(h) e0.2_p0.05
(i) e0.05_p0.05
Figure 22: Illustration of the averaged response curves of Supervised Learning (SL) in GNFC.
(i) t2.bias8
Figure 23: Illustration of the averaged response curves of Inverse Propensity Weighting (IPW) in TCGA.
(i) e0.05_p0.05
Figure 24: Illustration of the averaged response curves of Inverse Propensity Weighting (IPW) in GNFC.
(i) t2.bias8
Figure 25: Illustration of the averaged response curves of SCIGAN in TCGA.
(i) e0.05_p0.05
Figure 26: Illustration of the averaged response curves of SCIGAN in GNFC.
(i) t2.bias8
Figure 27: Illustration of the averaged response curves of GALILEO in TCGA.
(i) e0.05_p0.05
Figure 28: Illustration of the averaged response curves of GALILEO in GNFC.