SlowGuess/NeurIPS_2023_OCR huggingface dataset:
John picked up the football.
Bob went to the kitchen.
Where is the football? A:playground
Task 3: Three Supporting Facts
John picked up the apple.
John went to the office.
John went to the kitchen.
John dropped the apple.
Where was the apple before the kitchen? A:office
Task 4: Counting
Daniel picked up the football.
Daniel dropped the football.
Daniel got the milk.
Daniel took the apple. A: office
How many objects is Daniel holding? A: two
Task 5: Lists/Sets
Daniel picks up the football.
Daniel drops the newspaper.
Daniel picks up the milk.
What is Daniel holding? milk, football
Task 6: Simple Negation
Sandra travelled to the office.
Fred is no longer in the office.
Is Fred in the office? A: no
Is Sandra in the office? A:yes
Task 7: Time Reasoning
In the afternoon Julie went to the park.
Yesterday Julie was at school.
Julie went to the cinema this evening.
Where did Julie go after the park? A:cinema
Where was Julie before the park? A:school
Task 8: Positional Reasoning
The triangle is to the right of the blue square.
The red square is on top of the blue square.
The red sphere is to the right of the blue square.
Is the red sphere to the right of the blue square? A:yes
Is the red square to the left of the triangle? A:yes
Table 5: Examples for bAbI tasks.
A.6 Results of Main Experiments and Ablation Studies
Experimental results on our constructed downstream tasks are shown in Table 6, and the results on bAbI are shown in Table 7. We also show the results of ablation studies in Table 8.
A.7 Human Evaluations
We conduct human evaluations on plan generation for GPT-J model. Following Huang et al. [18] we asked 3 people to annotate whether each task can be completed using a generated plan. We randomly sampled 150 tasks and asked each person to annotate 50 of them. The Results show that the base GPT-J model can only achieve  $24.0\%$  accuracy, while the finetuned model can achieve  $62.4\%$ . The higher planning accuracy demonstrates the superior task planning ability of our model.
A.8 SuperGLUE Results
We evaluate the base GPT-J model and our finetuned model on appropriate SuperGLUE tasks, e.g., that can be formulated as a multi-choice QA task without prompt engineering. Our model's performance matches and even outperforms the baseline, showing our model retains the general language capability.
Task	Metric	GPT-Neo	GPT-J	OPT-13B	LLaMA-13B	ChatGPT (GPT3.5-turbo)
Base	Ours	Base	FT	Ours	Base	Ours	Base	Ours
Plan Generation
- Vanilla Seen	Rouge-L	21.25	49.70	34.31	47.98	51.23	36.00	50.15	41.77	52.05	40.57
- Vanilla UnSeen	Rouge-L	17.64	49.27	34.22	47.86	49.58	29.34	45.11	38.78	47.44	41.01
- Confusing Seen	Rouge-L	16.86	46.88	34.81	47.59	48.94	31.92	49.87	40.33	51.00	40.41
- Confusing Unseen	Rouge-L	17.05	42.34	32.98	44.43	45.60	36.98	47.93	41.73	50.49	40.97
Housework QA	Accuracy	70.11	72.41	77.78	51.34	85.44	81.61	84.29	81.99	86.59	83.91
Negation Housework QA	Accuracy	38.27	41.98	35.19	33.33	39.51	43.21	40.21	43.21	30.25	87.65
Activity Recognition	Accuracy	69.22	85.43	87.98	71.41	88.52	89.07	91.44	90.53	91.80	95.05
Activity Inference	Accuracy	56.49	66.03	69.08	70.99	74.43	67.94	70.61	74.05	68.32	83.59
Counting	Accuracy	22.68	28.87	30.41	16.49	67.01	20.01	62.37	29.38	79.38	66.49
Object Path Tracking	LCS	30.80	85.91	33.86	46.25	98.67	33.49	96.28	38.82	96.99	59.53
Object Location QA	Accuracy	22.50	33.50	30.00	22.50	34.50	37.00	33.00	28.50	79.00	67.50
Table 6: Experimental results on various downstream evaluation tasks. The best result among baselines and our method is shown in bold, and the best result among all the models is underlined.
Task	GPT-Neo	GPT-J	ChatGPT
Base	Ours	Base	Ours
Single Supporting Fact	51.86	56.29	65.16	68.98	96.27
Two Supporting Fact	33.43	30.82	40.48	26.08	47.33
Three Supporting Fact	7.85	13.49	22.46	30.41	16.82
Counting	34.04	48.84	41.39	69.08	93.96
Lists/Sets	14.80	51.76	34.74	84.99	76.84
Simple Negation	36.05	65.56	42.80	63.95	93.66
Time Reasoning	21.45	23.46	36.96	59.42	61.63
Positional Reasoning	50.51	53.64	49.70	53.23	58.38
Table 7: Experimental results on bAbI test sets.
Base	Ours	-GPT-Neo
-w/o Plan Gen	-w/o Act Recog	-w/o Count	-w/o Obj PT
Plan Gen
-Vanilla / Seen	21.25	49.70	14.48	49.38	49.85	50.06
-Vanilla / Unseen	17.64	49.27	14.28	48.96	51.16	49.02
-Confusing / Seen	16.86	46.88	13.63	46.37	48.30	49.14
-Confusing / Unseen	17.05	42.34	9.86	43.79	46.28	44.64
QA	70.11	72.41	73.18	71.26	74.71	70.11
Neg QA	38.27	41.98	32.72	35.80	36.42	38.89
Act Recog	69.22	85.43	85.97	48.63	85.25	84.34
Act Infer	56.49	66.03	66.03	58.40	64.89	62.21
Count	22.68	28.87	18.56	25.26	35.05	32.99
Obj PT	30.80	85.91	92.13	84.17	86.46	29.90
Obj QA	22.50	33.50	35.00	49.00	43.50	22.00
Perplexity	4.120*	4.193	4.171	4.151	4.162	4.164
Table 8: Ablation experimental results on training tasks.
Model	BoolQ	CB	RTE	AX-g	AX-b	COPA
GPT-J
- Base	45.20	41.07	47.29	50.00	57.50	59.00
- Ours	66.00	41.07	58.84	53.37	54.00	62.00
Table 9: Results on SuperGLUE subset.
A.9 Broader Impact
Like other generation systems, the language model trained by our approach is susceptible to producing unintended output when confronted with harmful input, such as unethical text or input intended for adversarial attacks. Therefore, we strongly advise against utilizing our approach outside of controlled research environments until these risks have been mitigated. It is important to note that a thoughtless deployment of our method could potentially enable malicious exploitation of the underlying language models. Thus, precautions, such as implementing a filtering mechanism, must be taken.