Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning
Authors: Pan Lu
Liang Qiu
Kai-Wei Chang
Ying Nian Wu
Song-Chun Zhu
Tanmay Rajpurohit
Peter Clark
Ashwin Kalyan
Venue: ICLR 2023 poster (2023)
Keywords: Mathematical Reasoning
Tabular Math Word Problems
Prompt Learning
Policy Gradient

Abstract: Mathematical reasoning, a core ability of human intelligence, presents unique challenges for machines in abstract thinking and logical reasoning. Recent large pre-trained language models such as GPT-3 have achieved remarkable progress on mathematical reasoning tasks written in text form, such as math word problems (MWP). However, it is unknown if the models can handle more complex problems that involve math reasoning over heterogeneous information, such as tabular data. To fill the gap, we present Tabular Math Word Problems (TabMWP), a new dataset containing 38,431 open-domain grade-level problems that require mathematical reasoning on both textual and tabular data. Each question in TabMWP is aligned with a tabular context, which is presented as an image, semi-structured text, and a structured table. There are two types of questions: free-text and multi-choice, and each problem is annotated with gold solutions to reveal the multi-step reasoning process. We evaluate different pre-trained models on TabMWP, including the GPT-3 model in a few-shot setting. As earlier studies suggest, since few-shot GPT-3 relies on the selection of in-context examples, its performance is unstable and can degrade to near chance. The unstable issue is more severe when handling complex problems like TabMWP. To mitigate this, we further propose a novel approach, PromptPG, which utilizes policy gradient to learn to select in-context examples from a small amount of training data and then constructs the corresponding prompt for the test example. Experimental results show that our method outperforms the best baseline by 5.31% on the accuracy metric and reduces the prediction variance significantly compared to random selection, which verifies its effectiveness in selecting in-context examples. The data and code are available at https://promptpg.github.io.

Published as a conference paper at ICLR 2023
DYNAMIC PROMPT LEARNING VIA POLICY GRADIENT FOR SEMI-STRUCTURED MATHEMATICAL REASONING
Pan Lu 1.3 L.3 Liang Qiu' Kai-Wei Chang', Ying Nian Wu', WW'',nnn-CCinl Zhu',
Tanmay Rajpurohit', Peter Clark", Ashwin Ashwin Kalyan² UUniversity of California, Los Angeles, ?Georgia Institute of Technology, 3 AAlen Institute for Al
ABSTRACT
Mathematical reasoning. core ability of human intelligence, presents unique challenges for machines in abstract thinking and logical reasoning. Recent large pre-trained language models such as GPT-3 have achieved remarkable progress on mathematical reasoning tasks written in text form, such math word problems (MWP). However, it is unknown if the models can handle more complex prob- lems that involve math reasoning over heterogeneous information, such as tabular data. To fill the gap, we present Tabular Math Word Problems (TABMWP), a new dataset containing 38,431 open-domain grade-level problems that require mathe- matical reasoning on both textual and tabular data, Each question in TABMWT WP is aligned with tabular context, which is presented as an image, semi-structured text, and structured table. There are two types of questions: free-text and multi- choice, and each problem annotated with gold solutions reveal the multi-step reasoning process. We evaluate different pre-trained models on TABMWP, in- cluding the GPT-3 model in a few-shot setting. As earlier studies suggest, since few-shot GPT-3 relies on the selection of in-context examples, its performance is unstable and can degrade to near chance. The unstable issue is more severe when handling complex problems like TABMWP. To mitigate this, we further propose a novel approach, PROMPTPG, which utilizes policy gradient to learn to select in-context examples from small amount of training data and then constructs the corresponding prompt for the test example. Experimental results show that our method outperforms the best baseline by 5.31% on the accuracy metric and re- duces the prediction variance significantly compared to random selection, which verifies its effectiveness in the selection of in-context examples.'
INTRODUCTION
Developing machines equipped with mathematical reasoning capabilities is one of the stangsinn goals of artificial intelligence. Solving math word problems (MWPs) a well-defined task di- agnose the ability of intelligent systems to perform numerical reasoning and problem-solving as humans. surge of datasets has been proposed to facilitate the research in this domain (Upadhyay & Chang, 2017; Amini et al., 2019; Miao et al., 2020; Cobbe et al., 2021). However, most existing MWP datasets focus on textual math word problems only. Tables, widely distributed in different documents such invoices, health records, and financial reports, contain rich structured informa- tion different from unstructured text. Solving math word problems in such a tabular context is much more challenging than existing MWP benchmarks since the system needs to make cell selections and align heterogeneous information before performing further numerical reasoning.
To fill this gap, we propose Tabular Math Word Problems (TABMWP), new large-scale dataset that contains 38,431 math word problems with tabular context, taken from grade-level math curricula. There are two question types: free-text questions in which the answer an integer or decimal number, and multi-choice questions where the answer is a text span chosen from option candidates. Different from existing MWP datasets, each problem TABMWP is accompanied by tabular context, which is represented in three formats: an image, semi-structured text, and a structured
'The data and code are available at https: / /ttp/pptppppptp////// pg .ii git....... c.. Work was partially done while Pan Lu was an intern at Allen Institute for AT (AI2).

---

Published as a conference paper at ICLR 2023
square beads
$2.97 kilogram
Question: If Tracy buys 5 kilograms of spherical beads, kilograms of star-shaped beads, and killorams of flower-shaped beads, how much will she spend? (unit: S)
oval bends
$3.41 per kilogrum
flower-shaped beads
$2.18 per kilogram
Answer: 31.44
Solution:
star-shaped beads
$1.95 per kilogram
Find the cost of the spherical beads. Multiply: $3.42 x 5 $17.10.
heart-shiped beads
$1.52 per kilogrum
Find the cost
star-shaped beads. Multiply: $1.95 4 4$$75800
spherical beads
$3.42 kilogram
Find the cost of the flower-shaped beuds. Multiply: S2.18 $6.54.
Now find the total cost by adding: $17.10 $7.80 $6.54 $31.44.
rectangular beads
$1.97 kilogram
She will spend $31.44.
Sandwich sales
Question:
As part of project health class, Cara surveyed local delis about the kinds of sandwiches sold. Which shop sold fewer sandwiches, Sandwich City Express Sandwiches?
Shop
Tuna
Egg salad
City Cafe
Options: (A) Sandwich City Express Sandwiches
Answer: (A) Sandwich City
Sandwich City
Solution:
Express Sandwiches
Add the numbers in the Sandwich City row. Then, add the numbers the Express Sandwiches row.
Sam's Sandwich Shop
Sandwich City: 12 15. Express Sandwiches: 7 17 24. 15 is less than 24. Sandwich City sold fewer sandwiches.
Kelly's Subs
Figure 1: Two examples from the TABMWP dataset. The example above is a free-text problem with numerical answer; the example below is multi-choice problem with a textual answer.
table. Each problem is also annotated with detailed solution that reveals the multi-step reasoning steps to ensure full explainability. To solve problems in TABMWP, a system requires multi-hop mathematical reasoning over heterogeneous information by looking up table cells given textual clues and conducting multi-step operations to predict the final answer. Take the problem above in Figure 1 example. To answer the question "how much will she spend (if Tracy buys three kinds of beads)?", we first need to look up the corresponding three rows in the given table, calculate the individual cost for each kind of bead, and finally sum three costs up get the answer of 31.44.
Inspired the success of the large pre-trained language model GPT-3 (Brown et al., 2020) in solving math word problems (Wei et al., 2022: Wang al., 2022), we first build a strong baseline using few-shot GPT-3 on TABMWP. A few in-context examples are randomly selected from the training set, along with the test example, and are constructed as prompt for GPT-3 to predict the answer. However, recent studies have shown that this type of few-shot learning can be highly unstable across different selections of in-context examples (Zhao et al., 2021; Liu et al., 2022a; Lu et al., 2022c). could be worse on TABMWP since its problems are distributed across multiple question types and diverse table layouts. Liu et al. (2022a) try to address this issue by retrieving semantically similar examples. However, this method might not work well sometimes TABMWP because it is not capable of measuring the similarity of structured information, such as the number of cells in tables.
To alleviate this challenge, we further propose novel approach that can learn to select in-context examples from a small amount of training data via policy gradient for prompt learning, termed PROMPTPG. As illustrated in Figure 2, an agent learns to find optimal in-context examples from a candidate pool, with the goal of maximizing the prediction rewards on given training examples when interacting with the GPT-3 environment. A policy network defines the strategy of how to select the in-context examples given the current training example. The policy network is built on top of the language model BERT (Devlin al., 2018) with fixed parameters, followed by one-layer linear neural network with learnable parameters. The learnable parameters are updated following the policy gradient strategy (Sutton et al., 1998). Unlike random selection (Wei et al., 2022; Wang et al., 2022), brute-foree force search, or retrieval- based selection (Liu et al., 2022a), PROMPTPG learns to construct the prompt dynamically given the candidate pool when interacting with the GPT-3 API.
We implement two state-of-the-arr methods as baselines, i.e., UnifiedQA (Khashabi et al., 2020) on general question answering and TAPEX (Liu et al., 2022b) on tabular question answering. Both implemented in pre-trained and fine-tuned settings. Experimental results show that our model PROMPTPG can achieve an overall accuracy of 68.23% on TABMWP, which greatly surpasses previous methods by large margin of up to 5.31%. Further analysis demonstrates that PROMPTPG can select better in-context examples compared with a wide range of existing selection strategies and reduce the prediction variance significantly compared to random selection.
The main contributions of our work are as follows: (a) We present a new large-scale dataset, TAPMWP, the first dataset for math word problems with tabular context; (b) We propose novel
2

---

tabular math word problem p represented as a pair (1, ạ), where table context and y is question. The table / could be represented in visual format or a structured database. In this work, we focus on the semi-structured format as the table context for simplicity. The table features complicated layouts and formats: it contains multiple rows and columns, and each cell can be string of text, a string of a number, or a mix of them. Depending on the question and answer types, the question may be accompanied by multiple-choice options c = {ci, . Cn) or a unit u. Given a semi-structured tabular context and an unstructured question text q, the task is generate the answer a, which is either numerical only text for free- Published as a conference paper at ICLR 2023
Agent
in-canlext
examples d;
sampling
Action (e..
Environment
ni (e-lps)
Prompt Creator
One-layer NN #
GPT-3 Engine
Pre-trained LM
prediction d:
Reward 1%
2
3
candidate exampies Ecand training example
Our proposed PROMPTPG is able to learn to select performing in-context examples via policy gradient when interacting with the GPT-3 API without any manually designed heuristics.
Figure 2:
approach, PROMPTPG, which learns the prompt dynamically via policy gradient to select in-context examples for few-shot GPT-3. To the best of our knowledge, it is the first work that applies rein- forcement learning to select in-context examples for the few-shot GPT-3 model; (c) Experimental results show that PROMPTPG achieves improvement of up to 5.31% TABMWP over existing methods, with reduced selection instability compared to random selection.
2
THE TABMWP DATASET
2.1 TASK FORMULATION
image, semi-structured text,
text question, or text span from given options for a multiple-choice question.
2.2 DATASET CONSTRUCTION
Data collection. We construct TABMWP based on openly available content and more details are provided in Appendix A.1. Only math word problems that are accompanied by tabular context and a detailed solution are collected. We develop a script to extract the tabular context, the question, options that apply, the correct answer, and the solution for each problem. These elements can be precisely identified using HTML tags, For each table, we take screenshot and store its raw text.
Data preprocessing. To make TABMWP compatible with various baselines, we represent the tab- ular context as three formats: an image, semi-structured text, and structured spreadsheet. The semi-structured format is created by converting the raw table text into a flattened token sequence, with each row separated by newline character "(n' and each column separated by 'I'. The semi- structured text further transformed executed by SQL-based methods (Liu et al., 2022b) using packages like pandas. For clarity, the table title separated from the raw table. Examples of three formats are shown Appendix A.
the structured format, which can be easily retrieved and
For better quantitative evaluation, we formalize the TAEMWP problems as two question types: (a) free-text questions, where the answer is numerical text only and the unit text is separately extracted; and (b) multi-choice questions, the answer of which is the text span from choice options. The order of choice options is shuffled to alleviate distribution bias. Redundant information in solutions is removed, and some solutions manually rewritten to be more human-readable. Finally, problems with the same table, question, and answer text are regarded as redundant and thus removed. We further conduct quality control to ensure data quality, which is discussed in Appendix A.I.
3

---

Published as a conference paper at ICLR 2023
2.3 DATASET STATISTICS
Key statistics. The TABMWP dataset contains 38,431 tabular math word problems, which are partitioned with 6:2:2 into the training. development, and test splits, cor- responding 23,059, 7.686, and 7.686 problems. Their main statistics are shown in Table 1. 74.7% of the ques- tions in TA belong to free-text questions, while 25.3% multi-choice questions. There are 28,876 dif- ferent questions, 6,153 different answers, and 35,442 different solutions, indicating that TABMWP has rich diversity in the problem distribution. The questions have an average of 22.1 words in length and solutions of 49.5, showing that they have lexical richness.
Statistic
Number
Total questions
38,431
free-text questions multi-choice questions
28,719
9,712
# different questions # of different answers
28,876
6,153
# of different solutions
35,442
# of different tables
37,644
# of tables with title
23,259
# of table cells (Average/Max)
12.9
# of table rows (Average/Max)
5.9/11
One distinct characteristic of TABMWP is that each problem is accompanied by tabular context, without which the problem would be unsolvable. There are 37,644 different tables in total, and 60.5% of the tables have a title. The table has an average of 5.9 rows and 2.2 columns, which results in an average of 12.9 cells and a maximum of 54 cells. These statistics suggest that tables in TABMWP distribute diversely across semantics and layouts.
# of table columns (Average/Max)
2
Question length (Average/Max)
29.1/92
Answer length (Average/Max) Solution length (Average/Max)
1.1
49.5 /350
Table I: Key statistics for TABMWP.
Comparison existing datasets. As shown in Table 2, TABMWP differs from related datasets in various aspects: (1) TABMWP is the first dataset to study math word problems over tabular context on open domains and the largest in terms data size; (2) Problems in TABMWP are annotated with the tabular context, unlike previous MWP datasets in the first segment: (3) Different from Table QA datasets like FinQA, TAT-QA, and MultiHiertt, a lack either mathematical reasoming or the tabular context renders the problems in TABMWP unanswerable; (4) There are two question types in TABMWE and the answer could be a text span, an integer number, or a decimal numher; (5) Each problem is annotated with natural language solutions to reveal multi-hop reasoning steps.
Table Type
Question Type
Answer Type
Need Math?
Need Table?
Solution Type
Dataset
Size
Domain
Format
Free-text
MC
Text Integer
Decimal
Dolphin18K (2016)
831
X
X
formula
DRAW-IK(2011
1,000
formula
Math23K (2017)
23,162
X
V
X
K
X
/
X
V
formula
MathQA (2019)
37,297
X
X
formula
ASDiv (2020)
2,305
V
formula
SVAMP (2021)
1,000
X
X
X
V
formula
GSMBK (2021)
8,792
X
X
text
IconQA (2021b)
107,439
FinQA (2021)
8,281
2,766
/
76.6%
finance
text
program
(2021) (2021)
16,552
2,747
50.0%
/
finance
text
x
X
MultiHiertt (2022)
10,440
9,843
89.8%
finance
text
TABMWP (ours)
38,431
37,644
open
text
text
Table 2: A comparison of MWP and Table QA datasets that require numerical reasoning. text*: each table in TABMWP is accompanied by image format.
3 METHODS
3.1 FEW-SHOT GPT-3 FOR TABMWP
Provided with a few in-context examples of math word problems as the context, GPT-3 can generate the answer for test problem, and shows impressive performance across different MWP datasets (Wei et al., 2022; Wang et al., 2022). Inspired by its success, we first build a strong baseline using few-shot GPT-3 on our TABMWP dataset. Specifically, few training examples, along with the test example p.. are provided to GPT-3 for the answer prediction. Each training example consists of table context i, question q, options that apply, and an answer a. To make the few-shot
4

---

Published as a conference paper at ICLR 2023
GPT-3 model workable on TABMWP, utilize the stri-strectured format the tabular context. Following Wei et al. (2022), a solution s can be augmented in front of the answer a to reveal the multi-step reasoning process, which is able to boost the prediction performance.
3.2 DYNAMIC PROMPTING VIA POLICY GRADIENT
The in-context examples can be randomly (Wei et al., 2022; Wang et al., 2022) or retrieval- selected (Liu et al., 2022a) from the training set. Recent research, however, has shown that few-shot GPT-3 can be highly unstable across different selections of in-context examples and permutations of those examples (Zhao et al., 2021; Liu et al., 2022a; Lu et al., 2022c). This instability may be more severe on TABMWP, where examples are more distinct because they include both unstructured questions of various types and
tables in various layouts. To alleviate this issue, we aim to propose novel approach that can learn to select performing in-context examples using policy gradient strategy, without brute-force searching or manually designed heuristics.
Formally, given TABMWP problem p;, want the agent find K in-context examples e; {c), ee.....e" from a candidate pool Ecand, and generate the answer å;, maximizing a reward ri Rp). The in-context examples are selected according policy
<m((ppp),, ex E Leand, e, are independent for k = 2I,
(I)
where 0 are the policy's parameters. The answer is generated through: à; GPT-3 Pt) using the selected examples and the given problem as the input prompt. The reward is then computed by evaluating the generated answer à with respect to the ground truth answer aş:
Ti R(å+lp;) EVAL a;), E (-1,1}.
The function EVAL.)) returns a reward of 1 if the generated answer aligned with the label and -1 otherwise. Our goal is to maximize the expected reward of the generated answer under the policy (e (p.) (PP-33( p.))]. We optimize the reward with respect to the parameters of the policy network using the Policy Gradient method (Sutton et al., 1998). The expected reward cannot be computed in closed form, compute unbiased estimation with Monte Carlo Sampling,
Be p.) (e)
Σ R(GPT-3(e)P))), et (e(e)p))) N
1
where N is the size of each batch yielded from training problem set Priin- In this work, experiment using the REINFORCE policy gradient algorithm (Williams, 1992):
VE, VEe.sT((e))))) (R(GPT-3(e,,p))) Be-roo(s Vo
es ( mo(espp)). (4)
Vo
Intuitively, if the predicted answer is correct, we update the policy so that the probability of selecting the same prompts gets higher. Otherwise, we update the policy to reduce the probability of selecting such less matched examples. The learning process is summarized in Algorithm I in the appendix.
To get the contextualized representation of the given problem and candidate examples, BERT (Devlin et al., 2018) [CLS] token representation as the problem encoding. We add a small linear layer on top of the BERT final pooling layer. That allows our model to learn both the semantic similarity that the pre- pre-trained BERT model provides and the hidden logical similarity shared among the math problems. During training, the parameters of BERT are fixed and only the appended linear layer is updated, i.e., 0 composed the learnable parameters W and b:
use the
h(e;) ((BERT(<)) + b,
h(p:) = W(BERT(p:)) + b. exp [h(e)) ) h(p.)] To(c))p:) = EefEEon exp [h(e/) hh(p))
(5)
5

---

Published as a conference paper at ICLR 2023
4 EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
Baselines. We first develop two large language models, UnifiedQA (Khashabi et al., 2020) and TAPEX (Liu et al., 2022b). in both pre-trained and fine-tuned settings, as strong baselines on TABMWP. Different model sizes are included to examine the performance across different model capacities. We further implement the zero-shot GPT-3 model, the few-shot GPT-3 model, and their chain-of-thought (Co'T) reasoning variants (Wei et al., 2022). We also study the heuristic guess base- line and human performance to analyze the lower and upper bounds on TABMWP, respectively.
Evaluation metric. The answer part is extracted from the GPT-3 generation using manually de- signed regular expressions. To evaluate the baselines and our method, we utilize the accuracy metric to determine if the generated answer is correct given the ground truth answer. For free-text problems where the answer is set as a number, we normalize the prediction and the label to decimal numbers with two-digit precision and check if their values are equivalent. For multi-choice problems, choose the most similar one from options to the generated answer following Khashabi et al. (2020).
Implementation details. Fine-tuned UnifiedQA and TAPEX baselines are trained on the train split and evaluated on the test split. Few-shot GPT-3 and few-shot-CoT GPT-3 randomly select two in- context examples from the training data build the prompt. Our PROMPTPG is built on top of few-shot GPT-3 with a different selection strategy: (a) in the training stage, the agent learns to select two examples from 20 candidates and is evaluated on 160 training examples to calculate the reward; (b) in the test stage, the agent with an optimal policy chooses two examples from 20 candidates for each test example. The candidates are randomly selected from the training set. Experiments for two few-shot GPT-3 baselines and our PROMPTPG are repeated three times, and the average accuracy is reported in Table More implementation details can be found in Appendix A.4.
4.2 EXPERIMENTAL RESULTS
Table 3 demonstrates the results of different baselines and our method on the TABMWP dataset. Benefiting from pre-training on the tabular corpus, the TAPEX baseline performs better on average than UnifiedQA with a similar model size, which is only pre-trained on unstructured textual data. Increasing the model size can improve the prediction accuracy for both UnifiedQA and TAPEX. Fine-tuned on TABMWP, the baseline models can significantly improve the prediction performance on the average and all aggregated accuracy metries.
Without any examples provided to GPT-3, zero-shot GPT-3 achieves a comparable accuracy to the best fine-tuned baselines, Unified(AALARGE and TAPEXLARGB- showing its surprisingly good gener- alization ability on TABMWP. Provided with two randomly sampled in-context examples as the prompt, few-shot GPT-3 gets an improvement of 0.17%. Generating the multi-step solution be- fore the answer, the few-shot-CoT GPT-3 model reports the best performance among all of these baseline models, with an accuracy of 62.92%. Unlike few-shot-CoTT GPT-3 randomly selecting the in-context examples, our proposed PROMPTPG learns to select performing examples with the help of policy gradient. PROMPTPG establishes a state-of-ther tht-arrtttttttttt-t-ore----- performance on the TABMWP dataset: surpasses the best baseline few-shot-Cot GPT-3 by 5.31% on average. PROMPTPG shows its consistent advantages on two question types, two grade groups, and most of the answer types.
Heuristic guess and human performance. The accuracy of multi-choice questions by heuristic guess is 39.81%, 9,, which aligns with the fact that there are 2.88 options on average. The accuracy for free-text questions is considerably low since the inputs of TABMWP problems do not have direct clues for the answers. Humans outperform all benchmarks consistently across question types, answer types, and grade groups, with a 21. 21.99% average accuracy advantage over our best performing PROMPTPG. This gap is to be filled by future research on semi-structured mathematical reasoning.
Problem types and difficulty. Among all the baselines, we find is easier for models to an- swer multi-choice questions than free-text questions. Questions with the boolean (BOOL) and other (OTH) answer types tend to have lower accuracy scores than the extractive (EXTR) answer type, because the former ones need the abilities of fact verification and language understanding on diverse options, respectively. It is also not surprising for us to find that all the models perform worse on problems in grades 7-8 than in lower-level group of 1-6.

---

Published as a conference paper at ICLR 2023
Question Types
Answer Types
Grades
Training Data
Selection Strategy
Method
FREE
MC
INI
DEC
EXTR
BOOL
OTH
1-6
7-8
Avg.
Heuristic Baselines
Heuristic guess
6,71
39.81
8.37
0.26
30,80
51.22
26.67
17.55
12.27
15.29
Human performance
84.61
93.32
84,95
83,29
97. 18
88.69
96.20
94.27
81.28
90.22
pre-trained Baselines
UnitiedQAsuan SMALL
1.18
43.62
1.37
0.43
38.70
49.78
37.14
15.57
7.65
12.18
UnifiedQAn.ss
4.60
43.02
1.97
37.08
50.11
38.10
17.14
11.11
14.56
UniliedQALaKue
4.48
48.80
1.72
48.33
50.33
40.00
19.78
15.96
TAPEX
7.32
39.76
8.68
2.06
35,06
47.11
20.95
18.67
15.73
TAPEX
8.80
46.59
10:62
1.72
46.91
48.11
30.48
22.65
13.18
18.59
fine-tnned Baselines
UnifiedQA
23,059
22.27
51.31
27.27
2.83
52.28
48,
69.52
35.85
21.71
29.79
UnifiedQAn.st
23,059
34.02
70.68
40.74
7.90
84,09
55.67
73.33
53.31
30.46
43.52
UnifiedQALasce
23,059
48.67
82.18
55.97
20.26
94.63
68.89
79.05
65.92
45.92
57.35
TAPEX
23,059
39.59
73.09
46.85
11.33
84.19
61.33
69.52
56.70
37.02
48.27
TAPEX
23,059
51.00
80.02
95.34
73.33
67.11
58.52
Prompting Baselines GPT-3
Zero-shot
53.57
66.67
55.55
45.84
78.22
55.44
54.29
63.37
48.41
56.96
Zero-sbot-CoT
54,36
66.92
55.82
48.67
78.82
55.67
51.43
63.62
49.59
57.61
shot (2-shot)
2
Random
54.69
64.11
58.36
40.40
75.95
52.41
53.02
63.10
49.16
57.13
Few-shot-CoT (2-shot)
2
Random
60.76
69.09
60.04
63.58
76.49
61.19
67.30
68.62
55.31
62.92
PROMPTPG w/ GPT:3 (Ours)
Few-shot-CoT (2-shot) 160+20
Dynamic
66.17
74.11
64.12
74.16
76.19
72.81
65.71
71.20
64.27
68.23%.31-
Table Evaluation results of various baselines and our method on TABMWP. Training Data: num- ber of used training data; Selection Strategy: strategy of selecting in-context examples for few-shot GPT-3; FREE: fre--exx questions; MC: multi-choice questions; INT: integer answers; DEC: decimal answers; EXTR: extractive text answers; BOOL: Boolean text answers; OTH: other text answers.
4.3 ABLATION STUDY
Here, we will study how different factors have an effect on the performances of baselines and our method on TARMWP. Experiments are conducted on 1,000 development examples.
Blind study of the dataset. We evaluate the information gain of each component of the TABMWP problems by removing from model inputs. To eliminate the impact and variance caused by exam- ple selection, the study is conducted using the zero-shot GPT-3 model. As shown in Table 4, there dramatic decline when either the tabular context (T) or the question text (Q) is missing from the inputs. For example, T-A and Q-+A only attain average accuracy of 6.10% and 7.00%, respectively, and their accuracies are near to zero on the multi-choice questions. Taking both tabu- lar and textual data inputs (TQ-A), the model significantly beats the heuristic guess. With the complete input information (TQ(C)->A) the full model achieves the best performance. The blind study shows that our TABMWP is robust and reliable in distribution, and all input components are indispensable parts that provide necessary information for answering the questions.
Model
Format
FREE
MC
INT
DEC
EXTR
BOOL
OTH
1-6
7-8
Avg.
Heuristic guess
TQ A
7.31
40.36
9.20
0.00
34.44
47.32
50.00
17.99
13.96
16.40
Zero-shot GPT-3
T-+A
8.28
0.36
10.24
0.67
0.66
0.00
0.00
9.41
1.02
6.10
Zero-shot GPT-3
Q-A
9.24
1.09
10.94
2.68
1.32
0.89
0.00
10.23
2.03
7.00
Zero-shot GPT-3
T(C)+A
36.42
$0.89
25.00
23.60
8.12
17.50
Zero-shot GPT-3
Q(C)->A >A
9.10
33.09
10.94
2.01
25.17
44.64
25.00
21.29
7.11
15.70
Zero-shot GPT-3
TQ->A
55.31
68.36
56.60
50.34
79.47
54.46
58.33
66.34
47.46
58.90
Zero-shot GPT-3 (full model)
TQ(C)+A
54.76
72.00
56.42
48.32
76.82
66.07
66.67
67.00
47.97
59,50
Table 4: Blind studies on TABMWP. T: tabular context; Q: question; C: choice options; A: answer.
QCC) means choice options come after the question in the input, while refers the question only.
Number of training examples. We study the effect of different numbers of training examples on our dynamic prompt learning in Figure 3 (a). With more training examples, the prediction accuracy first gradually increases to peak of around 160 training examples. After that, the accuracy goes down with a growing variance. We reckon it is because the policy gradient algorithm can benefit from the scaling-up training data but fails to exploit more examples efficiently.

---

Published as a conference paper at ICLR 2023
80
80
Training examples number-Bo
75
number=160
Accuracy (9g
70
Accuracy (8
70
65
55
55
20
40
80
160
320
5
10
20
40
160
320
Number of training examples
Number candidate examples
(a) Accuracy w.r.t. different numbers of training examples, given 20 candidate examples.
(b) Accuracy w.r.t. different numbers of candi- dates, given 80 and 160 training examples.
Figure 3: Accuracy w.r.t. different numbers of training and candidate examples. Experiments are conducted on 1,000 development instances, and each setting is repeated with four random seeds.
Number of candidate examples. In Figure (b), we investigate how different numbers of candi- date examples can affect policy learning performance. With the increasing candidate number, it is observed that the prediction accuracy will first go up and then go down after a threshold, given 80 or 160 training examples. It is probably because when the candidate pool is too small, the policy gra- dient algorithm has a limited action space to explore enough problem types. In contrast, too many candidates could make the algorithm hard learn optimal policy in a large search space.
Different selection strategies. Table 5, we compare the proposed PROMPTPG with random selection and other heuristic-based example selection strategies for the few-shot-CoT GPT-3 model. Compared random selection, selecting the same question or answer type of examples helps the model take the task-relevant ex- amples as the prompt, thus improving the accuracy and reducing the variance. Choosing the most complex ex- amples does not boost the prediction performance con- sistently. Manual selection selects the two examples from 20 with the highest evaluation accuracy shot-CoT GPT-3 as the fixed set of in-context en examples. Although it achieves the lowest prediction variance of 0, only improves by 1.7% over random selection. The kind of nearest neighbor search of the test example, help
Selection strategy
Acc. (%)
Same question type
66.2 + 0.60
Same answer type
67.9 + 0.38
Same grade level
67.9 + 1.87
Most complex table cells)
64.0 0.42
Most complex ques. words)
68.2 - 0.26
Random selection
65.2 4.01
Manual selection (fixed w/ top 2)
66.9 + 0.00
Nearest neighbor
68.2 : 0.29
PROMPTPG (Ours)
70.9 + 1.27
Table 5: Evaluation results of different se- lection strategies with three trials.
most semantically similar examples, construct the performing and stable prompt for GPT-3. PROMPTPG shows its effectiveness in se- lecting optimal in-context examples over other strategies and largely reduces the instability.
4.4 CASE STUDY
We conduct the case study in Appendix A.7. We visualize the two in-context examples selected by strategies of our PROMPTPG, nearest neighbor search, and random selection, Figure 5, 6, and 7, respectively. The nearest neighbor search strategy selects the "superficially"" similar examples to the test example. Instead, PROMPTPG tends to select examples that have multiple reasoning steps in the solution and similar abilities in mathematical reasoning, which results in higher prediction accuracy. Successful examples in Figure 8 12 show that PROMPTPG is able to generate reasonable reasoning steps to predict correct answers for wide range of TABMWP problems. Failure examples in Figure 13 18 suggest that PROMPTPG has limitations when solving problems provided with complex tabular contexts or requiring high-level ability of mathematical reasoning.
RELATED WORK
5.1
MATH WORD PROBLEMS
The task of solving Math Word Problems (MWPs) is to predict the answer given a natural language description of math problem. There have been great efforts in developing datasets for MWPs,
8

---

Published as a conference paper at ICLR 2023
including Math23K (Wang et al., 2017), MathQA (Amini al., 2019), ASDiv (Miao et al., 2020), SVAMP (Patel et al., 2021), and Lila (Mishra et al., 2022). However, these datasets only involve the textual modality, and most are limited to small data scale. Some recent datasets like DVQA (Kafle et al., 2018), IconQA (Luu et al., 2021b), Geometry3K (Luu et al., 2021a), and UniGeo (Chen et al., 2022) introduce math problems with diagrams as the visual context, where the system needs to perform mathematical reasoning over multi-modal information. To the best of our knowledge, our dataset TABMWP the first dataset that requires mathematical reasoning over heterogencous information from both the textual question and the tabular context. To solve MWPs, one popular line of previous methods is to generate the intermediate expressions and execute them to get the final answers (Huang et al., 2017; Roy & Roth, 2017; Amini et al., 2019). Inspired by the recent progress achieved by GPT-3 in solving MWPs (Wei et al., 2022; Wang et al., 2022; Kojima et al., 2022), we evaluate TABMWP using GPT-3 models in zero-shot and few-shot learning manners.
5.2 TABLE QA DATASETS
Table Question Answering (Table QA) refers to the task of answering questions about tabular data. Numerous datasets have been developed for Table QA. For example, TabMCQ (Jauhar et al., 2016) is an early dataset collected from grade exams. Datasets like WTQ (Pasupat & Liang, 2015), WikiSQL. (Zhong et al., 2017), and SQA (lyyer et al., 2017) contain semi-structured tables from Wikipedia, while Spider (Yu et al., 2018) collects structured tables sourced from databases. Recent work aims at introducing datasets that require multi-hop reasoning between the textual and tabular data: Hy- bridQA (Chen al., 2020b), OTTQA (Chen al., 2020a), MultiModalQA (Talmor et al., 2020), AIT-QA (Katsis et al., 2021), and FeTaQQ (Nan et al., 2022). Datasets most related to our TABMWP dataset are FinQA (Chen et al., 2021), TAT-QA (Zhu et al., 2021), and MultiHiert (Zhao al., 2022) because they need numerical reasoning on financial reports with tabular data. Note that 77.6% of questions in TAT-QA can be solvable without mathematical reasoning and 50.0% of questions in FinQA are not table-must to be answered. In contrast, our proposed TABMWP collects questions where both mathematical reasoning and tabular context are necessary.
5.3 PROMPT LEARNING FOR LANGUAGE MODELS
Large pre-trained language models, such GPT-3 (Brown et al., 2020), have shown their remark- able ability of few-shot learning on a wide range of downstream tasks (Houlsby et al., 2019; Brown al., 2020; Ma et al., 2022; Lu al., 2022a). Given few in-context examples as demonstrations, GPT-3 can generalize to unseen test examples without parameter updating. For example, Wei et al. (2022) randomly select different in-context examples from the training set and formulate their cor- responding prompt with test sample. However, recent studies show that few-shot GPT-3 highly depends on the selection of in-context examples and could be unstable, varying from the near chance to near state-of-the-arr performance (Zhao et al., 2021; Liu et al., 2022a; Lu et al., 2022b). To mit- igate the volatility of selecting in-context examples, Lu et al. (2022c) propose retrieving relevant examples that are semantically similar to the test sample. Other possible strategies could be using brute-force permutation search or relying on manually designed heuristics like choosing the most complex examples. Inspired by reinforcement learning's ability to search for optimal action pol- icy, we propose applying the policy gradient strategy (Sutton et al., 1998) to learn to select in-context examples more efficiently and stably without designing human-designed heuristics.
CONCLUSION
In this paper, we propose TABMWP, the first large-scale dataset for math word problems in tabular contexts. TABMWP contains 38,431 open-domain problems with two question types and three answer types, and each problem is annotated with multi-step solution. We evaluate TABMWP using state-of-the-an -ft QA and TableQA methods in both pre-trained and fine-tuned settings, as well as the large pre-trained language model GPT-3. We further propose a novel approach, PROMPTPG, for few-shot GPT-3, which utilizes policy gradient to learn to select in-context examples from the training data and construct the performing prompt for the test example. Experimental results show that PROMPTPG outperforms existing strong baselines by a large margin of 5.31% and reduces the accuracy volatility compared to random selection. To the best of our knowledge, it is the first work that applies reinforcement learning select in-context examples for the few-shot GPT-3 model.

---

Published as a conference paper at ICLR 2023
7 ACKNOWLEDGMENT
We would like to thank Zhou Yu and Jiuxiang Gu for insightful discussions on dataset collection. We thank Muhao Chen and Yao Fu for constructive suggestions in developing baselines and experi- ments. The work does not relate to Liang Qiu's position at Amazon Alexa.
REFERENCES
Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorske Yejin Choi, and Hannaneh Hajishirzi. Mathqa: Towards interpretable math word problem solving with operation-baseet brased for- malisms. In Proceedings of the 2019 Conference of the North American Chapter of the Associ- ation for Computational Linguistics: Human Language Technologies (NAACL), pp. 2357-2367, 2019.
Gabriel Barth-Maron, Matthew W Hoffmmnn, David Budden, Will Dabney, Dan Horgan, Dhruva Tb, Alistair Muldal, Nicolas Heess, and Timothy Lillicrap. Distributed distributional deterministic policy gradients. arXiv preprint arXiv:
2018.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems (NeurIPS), 13:1877-1901,, 2020.
Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen, and Xiaodan Liang. Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression. The 2022 Conference Empirical Methods in Natural Language Processing (EMNLP), 2022.
Wenhu Chen, Ming-Wei Chang, Eva Schlinger, William Yang Wang, and William W Cohen. Open question answering over tables and text. International Conference on Leurning Representations (ICLR), 2020a.
Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, and William Yang Wang. Hybridqa: A dataset of multi-hop hop question answering over tabular and textual data. In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 1026-1036, 2020b.
Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Har Huang, Bryan Routledge, et al. Finqa: A dataset of numerical reasoning over financial data. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3697-3711, 2021.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reicchrr Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. 10.11668, 2021.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 2018.
Amelia Glaese, Nat McAleese, Maja Trebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Mari- beth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, al. Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209, 14375, 2022.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efffcieet transfer learning for nlp. In International Conference on Machine Learning (ICML), pp. 2790-2799. PMLR, 2019.
Danqing Huang, Shuming Shi, Chin-Yesr Lin, Jian Yin, and Wei- Ying Ma. How well do computers solve math word problems? large-scale dataset construction and evaluation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 887-896, 2016.
Danqing Huang, Shuming Shi, Chin-Yew Lin, and Jian Yin. Learning fine-grainee expressions to solve math word problems. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pp. 805-814, 2017.
10

---

Published as a conference paper at ICLR 2023
Mohit lyyer, Wen-tau Yih, and Ming- Wei Chang. Search-based neural structured learning for quential question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1821-18311 2017.
Sujay Kumar Jauhar, Peter Turney, and Eduard Hovy. Tabmeq: dataset of general knowledge tables and multiple-choice questions. arXiv preprint arXiv: 2016.
Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. Dvqa: Understanding data visual- izations via question answering. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pp. 3648-5636 2018.
Kirthevasan Kandasamy, Yoram Bachrach, Ryota Tomioka, Daniel Tarlow, and David Carter. Batch policy gradient methods for improving neural conversation models. arXiv preprint arXiv: 172.00330 2017.
Yannis Katsis, Saneem Chemmengath, Vishwajeet Kumar, Samarth Bharadwaj, Mustafa Canim, Michael Glass, Alfio Gliozzo, Feifei Pan, Jaydeep Sen, Karthik Sankaranarayanan et al. Ait- qa: Question answering dataset over complex tables the airline industry. arXiv preprint arXiv:2/06. /2944, 2021.
Daniel Khashabi. Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. Unifiedqa: Crossing format boundaries with single qa system. In Findings of the Association for Computational Linguistics (EMNLP), pp. 1896-1907, 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6988 6980, 2014.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916 2022.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pre- training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7871-7880, Online, July 2020. Association for Computational Linguistics (ACL).
Timothy P Lillicrap, Jonathan Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv preprint 1rX9v:1509.00 02097,, 2015.
Jiachang Liu, Dinghan Shen, Yizhe Zhang, William B Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for gpt-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pp. 100-114, 2022a.
Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, and Jian-Guang Lou. Tapex: Table pre-training via learning neural sql executor. In International Conference Learning Representations (ICLR), 2022b.
Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang. Xiaodan Liang, and Song-Chun Zhu. Inter- gps: Interpretable geometry problem solving with formal language and symbolic reasoning. In The 59th Annual Meeting of the Association for Computational Linguistics (ACL), 2021a.
Pan Lu, Liang Qiu, Jiaqi Chen, Tony Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, and Song-Chun Zhu. Iconqa: A new benchmark for abstract diagram understanding and visual lan- guage reasoning. In The 35th Conference on Neural Information Processing Systems (NeurIPS) Track on Datasets and Benchmarks, 2021b.
Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai- Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. In The 36th Conference on Neural Information Processing Systems (NeurIPS 2022), 2022a.
11

---

Published as a conference paper at ICLR 2023
Pan Lu, Liang Qiu, Wenhao Yu, Sean Welleck, and Kai- Wei Chang. A survey of deep learning for mathematical reasoning. arXiv preprint arXiv:2212.10SS3 2022b.
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 8086-8096 2022c.
Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang. Sqa3d: Situated question answering in 3d scenes. arXiv preprint arXiv:2210.07474, 2022.
Shen- Yun Miao, Chao-Chur Liang, and Keh- Yih Su. A diverse corpus for evaluating and developing english math word problem solvers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 975-984, 2020.
Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord, Ashish Sabharwal, Peter Clark, and Ashwin Kalyan. Lila: A unified benchmark for mathematical reasoning. In The 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022.
Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International conference on machine learning, pp. 1928-1937. PMLR, 2016.
Linyong Nan, Chiachun Hsich, Ziming Mao, Xi Lin, Neha Verma, Rui Zhang, Wojeiech Kryšcifsski, Nick Schoelkopf, Riley Kong, Xiangru Tang, et al. Fetaqa: Free-form table question answering. Transactions of the Association for Computational Linguistics (TACL), 10:35 49, 2022.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155. 2022.
Panupong Pasupat and Percy Liang. Compositional semantic parsing semi-structured tables. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJNLP), pp. 1470- 1480, 2015.
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are nlp models really able solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), pp- 2080-2094, 2021.
Jan Peters and Stefan Schaal. Policy gradient methods for robotics. In 2006 JEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2219-2225. IEEE, 2006.
Liang Qiu, Yizhou Zhao, Jinchao Li, Pan Lu, Baolin Peng, Jianfeng Gao, and Song-Chun Zhu. Valuenet: A new dataset for human value driven dialogue system. IEEE Trans. on Patlern Analysis and Machine Intelligence (TPAMI), 44(5):2468-24444 2022.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research (JMLR), 21:1-67, 2020.
Subhro Roy and Dan Roth. Unit dependency graph and its application to arithmetic word problem solving. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2017.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: trXiv:/707.060 1707.06347, 2017.
David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin Riedmiller. Deterministic policy gradient algorithms. In International conference on machine learning, pp- 387-395. PMLR, 2014.
12

---

Published as a conference paper at ICLR 2023
Richard S Sutton, Andrew G Barto, al. Introduction to reinforcement learning. MIT press Cam- bridge, 1998.
Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, and Jonathan Berant. Multimodalqa: complex question answering over text, tables and images. In International Conference on Learning Representations (ICLR), 2020.
Shyam Upadhyay and Ming-
Ming-Wei Wei Chang. Annotating derivations: A new evaluation strategy and dataset for algebra word problems. Proceedings of the 15th Conference of the European Chap- ter of the Association for Computational Linguistics (ACL), pp. 494-504, 2017.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint 1rX1n:2203.11171, 2022.
Yan Wang, Xiaojiang Liu, and Shuming Shi. Deep neural solver for math word problems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 845-854, 2017.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoe Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022.
Ronald J Williams.
Simple statistical following algorithms for connectionist reinforcement learning. Machine learning, (()):229-566, 1992.
Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, al. Spider: A large-scale human labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. In Proceedings of the 2018 Conference on Empirical Methods Natural Language Processing (EMNLP), pp. 3911-3921, 2018.
Yilun Zhao, Yunxiang Li, Chenying Li, and Rui Zhang. Multihiertt: Numerical reasoning over multi hierarchical tabular and textual data. In Proceedings of the 60th Anmual Meeting of the Association for Computational Linguistics (ACL), pp. 5588-6600, 2022.
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. In International Conference on Machine Learning (ICML). pp. 12697-12706. PMLR, 2021.
Victor Zhong, Caiming Xiong, and Richard Socher. Seq2sql: Generating structured queries from natural language using reinforcement learning. arXiv preprint 1rXiv:1709.000 2017.
Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. Tat-qa: A question answering benchmark a hybrid of tabular and textual content in finance. In Proceedings of the 59th Annual Meeting of the Association for Computa- tional Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-JCNLP), pp. 3277-3287, 2021.
13

---

Published as a conference paper at ICLR 2023
A APPENDIX
A.1 DATASET COLLECTION
The raw problems are collected from an online learning wehsite, IXL.2, which hosts a large number of high-quality math problems curated by educational experts.
Quality control. The goal of constructing TABMWP is to collect math word problems that necessi- tate multi-hop mathematical reasoning between the question and the tabular context. Therefore, we ask human experts filter problems that can be solved either without the context of the table or by looking up table cells without numerical reasoning. To further ensure data quality, we ask human experts to perform a final review to re-check the dataset and manually revise incorrect annotations.
Question types
Answer types (%)
Deseriptions
Integer (59))0%%
The answer an integer number, e.g.. "40", "1,207", "-3".
Free-text
Decimal (15.23%)
The answer decimal or fraction number, e.g., "192.80", "68/217",
Extractive (13.01% %)
The answer could extracted from the table context.
Multi-choice
Boolean (10.97%)
The answer Boolean, e-g.. "yes"""no", "true" "''aale", "linear"""oolln"""
Other (1 (9.9%)
The belongs other text types, e.g., statement.
Table 6: Format diversity of questions and answers in TABMWP.
Image format
Semi-structured format
Structured format
Field day schedule
Tabll day schedule
day schedule
Begin
Table text:
Event
Degin
End
water balloon toss 11:30 A.M.
A.:50A.M M.
Event Begin End
water balloon
obstacle course
12:05 P.M.
12:25 P.M.
halloon toss 11:30 A.M. 11:50
obstaclo course
1205 PM.
12:26 RM
per achuue beil loss
12:30 P.M. 2:300
ohstacle course 12:05 P.M. 12:25 P.M.
parachute
1230 RM.
jump rope race
1:401 P.M.
2:05 2:05PPM
parachute ball toss 12:30 P.M. | 1:30 P.M.
Jump rope race
1:40 RM.
2:08 PM
balloon stomp
2:15 P.M.
2:35 P.M.
jump rope race
P.M.
stomp
relicy race
2:50 P.M.
3.40
balloon stomp 2:15 P.M. | 2:35 P.M.
reay rice
2:50 RM.
3:40 PM
fulls hoop contest
3:56 P.M.
P.M.
relay mce 2:50 P.M. 3:40 P.M.
P.M.
potaro sack race
4:40 P.M.
5:15 P.M.
hula hoop contest 3:55 P.M. 4:30
potato sack rase
4:40 PMM.
6:16 PM
Table 7: Three different formats for the tables in the TABMWP dataset.
A.2 HUMAN STUDY
To examine how humans perform on our TABMWP dataset, we released the human evaluation task on Amazon Mechanical Turk (AMT) the test split. We designed two sub-tasks for the human study: answering the free-text questions and answering the multi-choice questions. The user inter- shown in Figure Each human intelligence task (HIT) contains
faces for the two sub-tasks exam questions and 15 test questions. A worker should have HIT Approval Rate of 98% or higher and be approved with 5,000 more HITs. The worker provided with detailed instructions the beginning and needs to pass at least 3 free-text exam questions or 4 multi-choice exam questions to be qualified for the human study. Each HIT assigned two different workers. We assign a reward of $0.80 and $0.60 for one HIT of free-text and multi-choice sub-tasks, respectively.
A.3 THE PROMPTPG ALGORITHM
The pipeline of PROMPTPG to learn to select in-context examples is summarized in Algorithm 1.
A.4 IMPLEMENTATION DETAILS
Heuristics guess. To investigate the lower bound of the accuracy on TABMWP, we design sim- ple heuristics to guess answers for each question type. For multi-choice questions, we randomly
Phttps: w/www///w/w//w///////////....... ix1 www.mmmmmttttm
14

---

Published as a conference paper at ICLR 2023
Overview
Overview
mg
Ihank you for horpaiing us with research
will cneworing up to 20 filling in blank questions from grade school
You si be arswerig
curricuuum You sill have multipe choice questions from grade sohool mach minutes.
math curtouumm, uill have minutes.
question there
table image that required for you to
For nech queetios, trem will be a bable image thee necy aisd for you sove
The quartinn
make
Each quectin will have 2- choices. Only one the cornect anpwes
I canmal suscasollilyy click the [Bubnii Lutton al the und of page, clvvase
-18 decimall (e.. 4.1 fraction 1977 number only
parteu queeraon seems ambiguous line cornect anowesmmors than one browoucto please choose arowwer that makes
you.
go back and check all your filed answers are numbers onty.
Instraction Select the chokse that best anowers the quection.
Instruction: Give your numerkal unowee: The answer must intaaue '9's decima (e 14.72 4.12),o fradton th.. 277%
Look the following schedule. Gina just missed the 10:45 A.. bus downtowe. How long does she have to walt until the next bus?
Hanmais cookies Hanmohs bake Saturday than Sunday?
meny
Hotel anutii bus scheckle
Cookies baked
downtown
A0:
A.M
A.M.
Day
Number of cookies
Fricay
183
sottall hald
12:39 PM
12:30 PM
12.
Saturday
291
Ina station
1:00 PM
1:15
RM
Sunday
263
15 minutes
minutes
Answerr
13 30 minutes
Figure 4: User interfaces of human study for free-text and multi-choice questions.
Algorithm Dynamic Prompt Learning via Policy Gradient (PROMPTPG)
Input: Initial policy Teg training example Paxin. candidate example set Feaná, of training epochs N Output: Learned policy we
I: function REINFORCE(TE, Pinin, Eead, N)
Initialize policy network v with parameter 0o
3: for epoch = 1, 2,.,... N......o. do
4:
for Phanch Pazma do
> get batch from the training set
5:
Liurch
for p. E Purce
7:
Sample oît ~ mo(es (.), of E Eaand, k: {1, ..., K}
b K is of in-context examples b år is the GPT-3 generated answer D 0g is the ground truth answer of p:
8:
à GPT-3(e) . ei,
ri EVAL( a.), (-1.1}
10:
Cranch t- Cranch na In (e:ppp) end for
11:
12:
Optimize Ltucch wrt. end for
13:
14:
end for
15:
return me
16:
end function
select one from the given options with even probabilities. For free-text questions on TABMWP, the answers could only be integral or decimal numbers. Intuitively, we take advantage of regular expressions extract all the numbers from the tabular context and the question text as candidates, and then randomly choose one number as the prediction.
UnifiedQA baselines. UnifiedQA (Khashabi et al., 2020) is a T5-based (Raffel et al., 2020) QA system that was pre-trained on 8 seed QA datasets of multiple formats but with a unified text-to-text paradigm. We load the pre-trained checkpoint as the pre-trained baseline and train it on TABMWP as the fine-tuned baseline. Three different parameter sizes are compared: SMALL (60M), BASE (220M), and LARGE (770M).
TAPEX baselines. TAPEX (Liu et al., 2022b) is a BART-based (Lewis et al., 2020) language model pre-trained on structured tabular data to mimic the behavior of a SQL executor that can answer table-based questions. TAPEX shows ofate-of-the-an performance four table-related datasets. We establish the pre-traines and fine-tuned baselines on top of TAPEX with two model sizes: BASE (140M) and LARGE (400M).
15

---

Published as a conference paper at ICLR 2023
Zero-shot GPT-3 and zero-shot-Cof GPT-3. We establish the zero-shot baseline based GPT-3 (Brown et al., 2020). The zero-shot setup follows the format of TQ(C)~+A where the input is the concatenation of tokens of the tabular context (T), the question text (Q), and choice options (C) that apply while the output is to predict the answer (A). Following Kojima et al. (2022), we further build 7ero-shot-CoT GPT-3, which refers to the GPT-3 model with chain-of-thouugh (CoT) prompt. Specifically, we add the prompt "Ler's 's think step by step" at the end of the input to ask the model generate the multi-step solution (S) to mimic the reasoning process humans. Then the model takes the raw input and the newly generated solution to predict the final answer.
Few-shot GPT-3 and few-shot-CoT GPT-3, In the few-shot setting, we follow the standard prompting (Wei et al., 2022) where in-context examples are randomly selected from the training data as demonstrations for the text example. Similarly, the few-shot-CoT GPT-3 baseline takes the prompt template of -QA to generate the solution before the final answer.
Experimental details. Our experiments for UnifiedQA baselines, TAPEX baselines, and our pro- posed PROMPTPG are conducted using PyTorch on two Nvidia RTX 3090 GPUs. For fine-tuning the UnifiedQA and TAPEX baselines, we use the Adam optimizer (Kingma & Ba, 2014) with an initial learning rate of 5e-5. The training process takes 10 epochs with batch size of 16. The maximum number of input tokens is set as 200 and the maximum output length 100.
In our proposed PROMPTPG, the embedding size of the added linear neural network is 768. To learn the policy network, we use the Adam optimizer with an initial learning rate of 1e-3. The maximum number of training epochs is 30, with batch size of 20. The training process is stopped early if there is any NaN value in the loss for a batch of training data.
For the GPT-3 engine, official documentation. The temperature is set as O and the top probability is set as 1.0 to get the most deterministic prediction. The maximum number of tokens allowed for generating text is 512. Both the frequency penalty and the presence penalty are set as the default value, i.c., O.
TEXT MAXT-DAVINCI-002 the most capable engine recommended by the
A.5
MORE EXPERIMENTAL RESULTS
Method
Selection strategy
training examples
# candidate examples
# few-shot examples
Trial
Trial 2
Trial 3
Average (%)
Few-shot GPT-3
Random selection
2
58.
57.00
56.27
57.13 0.93
Few-shot-CoT GPT-3
Random selection
2
59.85
63.52
65.39
62.92 2.30
Few-shot Cols GPT-3
PROMPTPG (ours)
160
20
2
68.85
65.63
70.22
68.23 1.92
Table 8: Experimental settings and raw accuracy results of random selection and our PROMPTPG for the few-shot GPT-3 model on the TABMWP test split. For each setting, we repeat the experiment with the same set of three different random seeds.
Model
Selection strategy
Shot number
Acc. (%)
Few-shot-CoT GPT-3
Random selection
65.2 + 4.01
Few GPT-3
Random selection
65.7 + 1.16
Few-shot-CoT GPT-3
Random selection
67.7 + 0.78
Few-shot-Col GPT-3
Random selection
67.5 + 0.98
Few-shot-CoT GPT-3
PromptPG (ours)
70.9 1 1.27
Table 9: Results of different numbers of few-shot examples on 1,000 development examples.
Number of few-shot examples. We study the few-shot-CoT GPT-3 model with random selection in terms of the different numbers of in-context shots. For each number of in-context shots, the experiment was conducted on 1,000 development examples and repeated three times. The results are shown in Table 9. When increasing the number of in-context shots from the current 2 to 4, in-
the few-shot-CoT GPT-3 model reduces the prediction variance from the random selection context shots and achieves accuracy improvement of 2.5%. When the number of in-context shots is increased to 5, the model with random selection does not gain further benefits. Our PromptPG displays impressive advantages accuracy. With only two in in-context shots, PromptPG achieves the highest accuracy of 70.9% and a comparable low deviation compared to random selection with more shots.
random selection in terms of data efficiency and prediction
16

---

Published as a conference paper at ICLR 2023
A.6 RELATED WORK OF POLICY GRADIENT
Policy gradient is an approach to solving reinforcement learning problems that target modeling and optimizing the policy directly. Many policy gradient algorithms have been proposed in the past decade (Silver et al., 2014; Lillicrap et al., 2015; Mnih et al., 2016; Schulman et al., 2017; Barth- Maron et al., 2018). They have been proven effective in areas like roboties (Peters & Schaal, 2006) and chatbots (Kandasamy et al.. 2017). In recent work that focuses on aligning language models with human values (Ouyang al., 2022: Qiu al., 2022; Glaese al., 2022), policy gradient has been used to optimize language models with rewards learned from human feedback and preference. To the best of our knowledge, our PromptPG the first work that proposes to select prompts dy dynamically for large pre-trained language models in the mathematical reasoning field.
A.7 CASE STUDY EXAMPLES
17

---

Published as a conference paper at ICLR 2023
In-context example (II): 28463)
Table:
Option
Change phone price
Option Change in phone price Add an upgrade $60
Add upgrade
Buy used phone -S75
Buy used phone
Question: Luna is looking at the price new cell phones online. Her favorite company, Orange OrangeTecch, has special this weekend. Luna add upgrade phone for additional cost, she buy used phone to get discount. The change in price for each option shown the table. Which option results in greater change in price?
Options:
(A) adding an upgrade (B) buying a used phone
Answer:
(Step 1) To find the option that results in greater change in price, use absolute value. Absolute value tells
you how much the price changes.
(Step 2) Add an upgrade: [$60] = $60
(Step 3) Buy used phone: |-S75] $75
(Step 4) Buying used phone results in greater change in price. It reduces the price by S75. The amswer is buying a used phone.
In-context example (ID: 13974)
Table:
heart-shaped beads
$3/kilogram
heart-shaped beads $3/kilogram
rectangular beads
$2/kilogram
rectangular beads $2/Kilogram
spherical beads $2/kilogram oval beads S2/kilogram
sphericul beads
$2 32/koggram
oval beads
$2/kilogram
Question: Rebecca bought 2.5 kilograms of oval beads. How much did she spend? (Unit: $) Answer:
(Step 1) Find the cost the oval beads. Multiply the price per kilogram by the number of kilograms.
(Step 2) $2 x 2.5= $5
(Step She spent $5. The answer is
Test example (ID: 17417)
Table:
Birthday party
[TITLE]: Birthday party
Activity
Parents
Children
Activity Parents Children
Singing
14
20
Singing 14 20
Eating cake
10
Eating cake 5
Jumping rope 16 20
Jumping rope
20
Swimming 16
Swimming
16
19
Playing tag |
Playing tag
Question: At Josie's 's birthday party, children and parents celebrated by participating in some activities. How many more children than parents are playing tag? (Unit: children)
Answer:
(Step 1) To find the difference between the number of children and parents playing tag, subtract the number of parents from the number of children.
(Step 2) - 5
(Step There are 5 more children than parents playing tag. The answer is S.
Output: 5
Figure 5: Two in-context examples selected by PROMPTPG, the prompt, and the correct prediction.
The selected examples require similar abilities mathematical reasoning to the test example.
18

---

Published as a conference paper at ICLR 2023
In-context example (ID: 18429)
Table:
Children's weights (lbs)
[TITLE]: Children's weights (Ibs)
Stem
Leaf
Stem
Leaf
2
2 4
3
5 2, 2,
5
2, 2, 8
6 6
7 1,3
8 7,78
7, 8
9
9
Question: Dr. Thornton, pediatrician, weighed all the children who recently visited her office. How many children weighed exactly 52 pounds? (Unit: children)
Answer:
(Step 1) For the number 52, the stem is 5, and the leaf 2. Find the row where the stem 5, In that row, count all the leaves equal
(Step 2) You counted leaves, which are blue in the stem-and-leaf plot above.
(Step 2 children weighed exactly 52 pounds. The answer is 2.
In-context example (ID: 18/29)
Table:
whole wheal pancake mix
$1 per
whole wheat pancake mix $1/lb buttermilk pancake mix $2//bb
buttermilk pancake mix
$2 per
chocolate chip pancake mix
per
chocolate chip pancake mix $2/Ib blueberry pancake mix S1/Ib
blueberry pancake mix
plain pancake mix S2/lb
plain pancake mix
raspberry pancake mix $2/Ib Question:
raspherry pancake mix
$2 per
If Vince buys 3.4 pounds of buttermilk pancake mix, how much will he spend? (Unit:
Answer:
(Step 1) Find the cost of the buttermilk pancake mix. Multiply the price per pound by the number of pounds.
(Step 2) $2 x 3.4 = $6.80
(Step He will spend $6.80. The answer 6.80.
Test example (ID: 17417)
Table:
Birthday party
[TITLE): Birthday party
Activity
Parents
Children
Activity Parents Children
Singing
Singing
Enting
Eating cake 5 10
Jumping rope 16 20
Jumping rope
16
20
Swimming 16 19
Swimming
16
19
Playing tag |
Playing tag
Question: At Josie's birthday party, children and parents celehrated by participating in some activitics. How many children than parents playing tag? (Unit: children)
Answer:
(Step 1) the activity "Playing tag". the stem is 4, and the leaf Find the row where the stem In that row, count all the leaves equal to 9.
(Step You counted leaves, which blue in the stem-and-leat plot above, children are playing tag. The
answer is 9.
Output: 9
Ground truth: 5
Figure 6:
Two in-context examples selected by nearest neighbor search, the prompt, and the final wrong prediction. The selected examples only semantically similar to the test example.
19

---

Published as a conference paper at ICLR 2023
In-context example (ID: 13033)
Table:
Watermelons harvested
[TITLE]: Watermelons harvested
Day
Number of watermelons
Day Number of watermelons
Wednesday
Wednesday
Thursday
51
Thursday |11
Friday |53
Friday
53
Saturday 52
Saturday
Question: A farmer recalled how many watermelons were harvested in the past days. On which day were the most watermelons harvested?
Options: (A) Wednesday (B) Thursday (C) Friday (D) Saturday
Answer:
(Step 1) Find the greatest number the table. Remember compare the numbers starting with the highest place
value, The greatest number 59,
(Step 2) Now find the corresponding day. Wednesday corresponds 59. The answer is Wednesday.
In-context example (ID: 32386)
Table:
Easketball hoops
[TITLE]: Basketball hoops
Park
Number basketball hoops
Park Numher of haskethall hoops
Heron Park
2
Heron Park 2
Kelly
Kelly Park
Westfield Park
4
Westfield Park
Pinehurst Park 4
Pineburst Park
4
Linden Park 3
Linden Park
Mooreland Park 7
Mooreland Park
Crestview Park 2
Crestview Park
Riverfront Park 4
Rivertront Park
4
Question: The parks department compared how many basketball hoops there are at each park. What is the mode of the numbers?
Answer:
(Step 1) Read the numbers from the table.
(Step 2) First, arrange the numbers from to greatest: 3, 4, 7,
(Step 3) Now count how many times each number appears.
(Step 4) appears 2 times. appears I time. appears times. 7 appears times.
(Step 5) The number that appears most often 4.
(Step 6) The mode is 4. The answer is 4.
b Test example (ID: 17417)
Table:
Birthday party
[TITLE]: Birthday party
Activity
Parents
Children
Activity Parents Children
Singing
14
20
Singing 14 20
Fating cake 5 10
Fating cake
10
Jumping rope
20
Jumping rope
20
Swimming 16 19
Swimmning
Playing tag 4 9
Playing tag
Question: At Josie's birthday party, children and parents celehrated by participating in some activitics. How many more children than parents playing tag? (Unit: children)
Answer:
(Step 1) Find the difference between the number of children and the number of parents for each activity.
(Step 2) For singing, the difference
(Step 3) For eating cake, the difference is 5.
(Step 4) For jumping rope, the difference 4.
(Step 5) For swimming, the difference 3.
(Step 6) For playing tag, the difference is 5.
(Step 7) The greatest difference The answer
Output:
Ground truth: 5
Figure 7: Two randomly selected in-context examples, the prompt, and the final prediction. The selected examples have limited relevance the test example, resulting in the wrong output.
20

---

Published as a conference paper at ICLR 2023
Table:
[TITLE] Math teachers
High school
Number of math teachers
High school Number math teachers
Central High
Central High|
Hillview High
Hillview High 10
Westside High
4
Westside High 4
Moore lligh
2
Moore High 2
River High
River High
6
Northside High |
High
Lincoln High| 8
Lincoln High
Thompson High |
Thompson lligh
5
Question: The school district compared how many math teachers each high school has. What is the mean of the numbers?
Answer:
(Step 1) The mean is the average of the numbers,
(Step 2) To find the mean,
all the numbers and divide by the number items.
(Step 3) 9 10 + + 6 + 4 8 5 48
(Step 4) There are 8 items, so the mean is 48 6.
(Step 5) The mean of the numbers is 6.
Output:
Figure 8:
The correct prediction from our PROMPTPG for free-text question example. This ex-
ample requires taking the mean of eight numbers from the table via addition and division.
Table:
$18.55
topaz $18.55 per lb
amethyst
$19.88 lb
amethyst $19.88 per
tiger's s eye
$10.29 per
tiger's eye $10.29 per lb
fool's gold
$16.00 per lb
fool's gold $16.00 per
quartz $14.63 per lb
quartz
$14.63 per lb
calcite $$5.39 per
calcite
$15.39 per
granite $19.23 per lb
granite
$19.23 per
Question: Austin went to the store and bought 3 pounds of fool's gold, pound of topaz, and 2 pounds spend? (unit: $)
of tiger's eye. How much
Answer:
(Step 1) find the total amount Austin spent, multiply the price per pound each type stone by number of pounds of that stone that he bought, and then add all of the products together.
(Step 2) 3 pounds of fool's gold: $16.00 per pound x 3 pounds $48.00
(Step pound of topaz: $18.55 per pound x pound $18.55
(Step 4) 2 pounds of tiger's eye: $10.29 per pound x pounds $20.58
(Step $48.00 $18.55 + $20.58 $87.13
(Step 6) Austin spent $87.13.
Output: 87.13
Figure 9: The correct prediction from our PROMPTPG for a free-ttex question example. This ex- ample requires looking up three items and their corresponding prices in the table, calculating their costs, and finally summing them up to get the final answer.
21

---

Published as a conference paper at ICLR 2023
Table:
Employee
Pay period
Employee Pay period
Dhruba Khanal
December 9-15
Dhruba Khanal December 9-15
Total carnings
S620.00
Total carnings $620.00
Foderal income tax
S71.14
Federal income tax $71.14
State income tax $48.90
income tax
Other taxes $47.00
Other laxes
$47.00
Total taxes ?
Total taxes
?
Pay after taxes || |
Pay after taxes
Question: Look at Dhruba's pay stub. Dhruba lives state that has state income tax. How much did
Dhruba make after taxes? (unit: $)
Answer:
(Step 1) To find Dhruba's pay after taxes, subtract the total taxes from Dhruba's 's total carnings.
(Step 2) $620.00 $71.14 $48.90 $47.00 $452.96
(Step 3) Dhruba made $452.96 after taxes. The answer 452.96.
Output: 452.96
Figure 10: The correct prediction from our PROMPTPG for free-text question example. In this example, the model is asked to understand a hierarchical tax report and calculate the pay after taxes.
Table:
[TITLE] Bus schedule
the school 8:00 A.M. 9:15 A.M. 9:30 A.M. 10:00 A.M.
11:00 A.M.
the zoo 9:00 A.M. 10:15 A.M. 10:30 A.M. 11:00 A.M. 12:00 P.M.
mall 9:15 A.M. 10:30 A.M. 10:45 A.M. 11:15 A.M. 12:15 P.M.
the grocery store 9:30 A.M. 10:45 A.M. 11:00 A.M. 11:30 A.M. 12:30 P.M.
science museum 10:30 A.M. 11:45 A.M. | 12:00 P.M. 12:30 P.M. 1:30 P.M.
the library 11:15 A.M. 12:30 P.M. 12:45 P.M. 1:15 P.M. 2:15 P.M.
kickball field 11:45 A.M. 1:00 P.M. 1:15 P.M. 1:45 P.M. 2:45 P.M.
the playground 12:45 P.M. 2:00 P.M. 2:15 P.M. 2:45 P.M. 3:45 P.M.
the doctor's office
1:15 P.M. 2:30 P.M. 2:45 P.M. | 3:15 P.M. 4:15 P.M.
Bus schedule
school the 200
8:00 A.M.
9:15 A.M.
A.M.
10:00 A.M.
11:00 A.M.
9:00 A.M.
10:15 A.M.
10:30 A.M.
1.:00A.M. A.M.
12:00 P.M.
the mall
9:15 A.M.
10:30 A.M.
10:45 A.M.
11:15 A.M.
12:15 P.M.
the grocery store
9:30 A.M.
10:45 A.M.
11:00 A.M.
11:30 A.M.
P.M.
the science museum
A.M.
11:45 A.M.
12:00 P.M.
12:30 P.M.
1:30 P.M.
library
11:15
12:45 P.M.
2:15
the kickhall field
11:45 A.M.
1:00 P.M.
1:15 P.M.
1:45 P.M.
2:45 P.M.
playground
12:45
2:00 P.M.
2:15 P.M.
2:45
3:45 P.M.
the doctor's office
1:15 P.M.
2:30 P.M.
2:45 P.M.
3:15 P.M.
4:15 P.M.
Question: Look at the following schedule. Lena is at the mall. If she wants to arrive the library 1.15 P.M., what time should she get on the bus?
Options: (A) 11:15 A.M. (B) 10:30 A.M. (C) 12:30 P.M. (D) 12:15 P.M.
Answer:
(Step 1) Find the mall the schedule.
(Step 2) The mall is on the bus route at 9:15 A.M., 10:30 A.M., 10:45 A.M., 11:15 A.M., and 12:15 P.M.
Find the library the schedule.
(Step 3) The library is on the hus route at 11:15 A.M., 12:30 P.M., 12:45 P.M., 1:15 P.M., and 2:15 P.M.
(Step Lena should get on the 11:15 A.M. to arrive the library at 1:15 P.M. The answer 11:15
A.M.
Output: (A) 11:15 A.M.
Figure 11: The correct prediction from our PROMPTPG for a multi-choice question. There are
rows and 6 columns in the given tabular context. Our model successfully locates the target cells in the table and performs multi-hop reasoning to predict the correct answer.
22

---

Published as a conference paper at ICLR 2023
Table:
science seience-fiction book
$4.31
science-fiction book S4.31
mystery novel
$8.26
mystery novel $8.26
crossword puzzle book
S8.74
crossword puzzle book |88.74
geography book $8.61
geography book
$8.61
coloring book $8.08
coloring book
$8.08
paperback book $8.45
paperback book
$8.45
Question: Ariana has $16.50. Does she have enough to buy paperback book and mystery novel?
Options: (A) yes (B) no
Answer:
(Step 1) To answer this question, look the table to find the price a paperhack book the price of mystery novel.
(Step 2) A paperback book costs $8.45 and a mystery novel costs $8.26. (Step $8.45 $8. $16.71 (Step 4) Since $16.71 is greater than $16.50, Ariana does not have enough money to buy paperback book and mystery novel. The answer no. Output: (B) no
Figure 12: The correct prediction from our PROMPTPG for multi-choice question with Boolean options. needs to compare the budget and the total costs to verify if Ariana has enough money.
Table:
yellow tiger's eye
$0.85
yellow tiger's eye $0.85
piece of green slate
$0.59
piece of green slate $0.59
piece of red sandstone
$0.19
piece sandstone $0.19
piece of rose quartz S0.61
piece of rose quartz
$0.61
smooth piece of marble $0.45
smooth piece of marble
$0.45
Question: How much money does Connor need huy piece of quartz, piece of green slate, and a piece of red sandstone? (unit: $)
Answer:
(Step 1) Connor needs to buy piece of rose quartz, a piece of green slate, and a piece of red sandstone. To find the total amount money Connor needs, add the prices the three items.
(Step 2) $0.85 $0.59 $0.19 $1.63
(Step 3) Counor needs $1. $1.63 buy piece of rose quartz, piece green slate, a piece of sandstone.
The answer is 1.63.
Output: 1.63
Ground truth: 1.39
Figure 13: The wrong prediction from our PROMPTPG for a free-text question example. Our model retrieves the wrong price for the rose quartz, thus calculating the wrong cost sum of three items.
23

---

Published as a conference paper at ICLR 2023
Table:
Apples per tree
(TITLE] Apples per tree
Leaf
Stem
Leaf
1,
I, 3,6
3,
2 2, 3.6
0
4 3
5 2,
2,
6 0,0,00 0,
0,0,
2,
2,
8,4, 6,
4,5, 6,
Question: A farmer counted the number of apples on each tree in her orchard. How many trees have at least 51 apples but fewer than 83 apples? (unit: trees)
Answer:
(Step To solve this problem, look at the table and find all the trees that have least apples but fewer
than 83 apples.
(Step There three trees that have least 51 apples but fewer than 83 apples. The answer is
Output: 3
Ground fruth: 7
The wrong prediction from our PROMPTPG for a free-eext question example. In this example, the problem is provided with an abstract Stem-L table. Our model fails to understand this dopaif-specific table and lacks high-level ability of logical reasoning to get the correct answer.
Figure 14:
Table:
Lawn mowers
[TITLE] Lawn mowers sold
Day
Number lawn
Day Number of lawn mowers
Tuesday
Tuesday 8
Wednesday
7
Wednesday
Thursday
Thursday 5
Friday 7
Friday
Saturday
Saturday
9
Sunday 8
Sunday
8
Monday 8
Monday
Question: A garden supply store kept track how many lawn mowers sold in the past days. What
the median of the numbers?
Answer:
(Step The median the "middle" value. To find the median, put numbers order from least greatest and find the number in the middle.
(Step 7, 7, 9, 8,
(Step 3) The median is 7. The answer is 7.
Output:
Ground truth: 8
Figure 15: The wrong prediction from our PROMPTPG for free-text question example. It seems that our model has limited ability order numbers in the mathematical domain.
24

---

Published as a conference paper at ICLR 2023
Table:
[TITLE] Tour boat schedule
Ocean City 8:15 A.M. 9900 A.M. 19:15 A.M. 19:30 A.M. 10:00 A.M.
Whale Watch Harbor 9:30 A.M. 10:15 A.M. 10:30 A.M. 10:45 A.M. 11:15 A.M.
Oyster Lighthouse 10:15 A.M. 11:00 A.M. 11:15 A.M.
11:30 A.M. 12:00 P.M.
Fisherman's 's Cove 11:15 A.M. 12:00 P.M. 12:15 P.M. 12:30 P.M. 1:00 P.M.
Surfing Beach 12:00 P.M. 12:45 P.M. 1:00 P.M. 1:15 P.M. 1:45 P.M.
Tour boat schedule
Occan City
8:15 A.M.
9:00 A.M.
9:15 A.M.
9:30 A.M.
10:00 A.M.
Whale Watch Harbor
A.
10:15 A.M.
10:30 A.M.
10:45 A.M.
11:15 A.M.
Oyster Lighthouse
10:15 A.M.
11:00 A.M.
11:15 A.M.
11:30 A.M.
12:00 P.M.
Fisherman's Cove
11:15 A.M.
12:00 P.M.
12:15 P.M.
12:30 P.M.
1:00 P.M.
Surfing Beach
12:00
12:45 1:00 P.M.
1:45
Question: Look at the following schedule. Haley is at Ocean City at 9.45 A.M. How soon can she get to Surfing Beach?
Options: (A) 1:45 P.M. (B) 12:00 P.M. (C) 10:30 A.M. (D) 12:15 P.M.
Answer:
(Step 1) Find 9:45 A.M. on the schedule.
(Step The next stop is Whale Watch Harbor at 10:15 A.M.
(Step 3) The carliest she can get Surting Beach is 12:00 P.M. The answer is 12:00 P.M.
Output: (B) 12:00 P.M.
Ground truth: (A) 1:45 P.M.
The wrong prediction from our PROMPTPG for multi-choice question example. In this example, the query of the current time provided in the question text does not hit any time slot exactly in the tabular context. Therefore, the model fails to locate the accurate time for the next stop.
Figure 16:
Table:
Blocks from schuol
[TITLE] Blocks from school
Number blocks
Name Number of blocks
Wendy
14
Wendy
Carson
Carson
Zach
17
Zach 17
Lila 17
Nicole 3
Nicole
Shane 7
Shame
7
Henry 5
Henry
5
Devin 17
Question: Some students compared how many blocks they live from school. What the mean of the num-
bers?
Answer:
(Step 1) The mean is the average of the numbers.
(Step 2) To find the mean, add the numbers and divide by the number of students.
(Step 14 8 + 17 + 17 +33 5 17 = 86
(Step 4) There are 8 students, so divide 86 by 8.
(Step 5) 86 = 10.75
(Step 6) The mean is 10.75. The answer is 10.75.
Output: 10.75
Ground truth: 11
Figure 17: The wrong prediction from our PROMPTPG for a-ree-text question example. The exam- ple poses challenge for the model to perform arithmetic operation long chain of numbers.
25

---

Published as a conference paper at ICLR 2023
Table:
Miniature golf scores
[TITLE] Bus schedule
Name
Score
Name
Score
Anita 128
Anita
Krysta 135
Krysta 135
Hugo 137
Hugo 137
Devin 115
Devin
115
Question: Some friends played miniature golf and wrote down their scores. How many more strokes did Hugo take than Anita? (unit: strokes)
Answer:
(Step 1) Anita's score 128. Hugo's score is 137.
(Step Hugo took 9 more strokes than Anita.
Output: 137
Ground truth: 9
Figure 18: The wrong prediction from our PROMPTPG for a free-texi question example. Although our model includes the correct answer in the generated output, the output does not follow the format that is designed in in-context examples. It makes answer extractor fail to get the target answer. This issue could be alleviated by completing numan-deesigned rules or developing an additional module to extract the answer from the prediction accurately in various cases.
26


Reviews:
[{"review_id": "wZ7NW70tpUJ", "reviewer": "Reviewer_fX43", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a dataset of Tabular Math Word Problems (TabMWP), consisting of Math word problems with associated tabular data.  Additionally, the authors introduce a policy gradient approach (PromptPG) for selecting in-context examples as prompts for a GPT-3 few-shot language model in generating answers to the TabMWP problems.  The dataset is open-source, and contains multiple types of answers (numeric, multiple-choice, text), and the policy gradient method is shown to outperform other approaches.", "summary_of_the_review": "The paper represents a potentially valuable contribution to an interesting problem in natural language processing, showing how a 'high-level' algorithm can be developed by using existing components to achieve SOTA results in a surprisingly straightforward manner.  The paper will be of interest to those working in natural language processing and deep learning.", "strength_and_weaknesses": "Strengths:\n\n- The new dataset is a valuable resource for an interesting problem, and goes beyond similar previous datasets in terms of scale and types of problems as shown in Table 2.\n- The PromptPG method demonstrates impressive performance, and the experimentation is very thorough in testing against SOTA approaches, the influence of hyperparameters, human performance, simpler selection strategies (random / nearest neighbor).\n\nWeaknesses:\n\n- The method is mainly a combination of previous approaches, and there is not substantial theoretical/conceptual novelty\n- It is not clear why the method is framed specifically as a reinforcement leaning method - since the problem instances are all independent, it would seem more accurate to describe it as online learning, and describe the approach as optimizing a supervised model with latent variables.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "hqO0ezVeNxb", "reviewer": "Reviewer_5Mut", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents TABMWP, a dataset of grade-school-level math problems with tabular context and human annotated solutions in text form. Then the author proposes a method called PROMPTPG to mitigate the unstable issue occurred when solving problems above in few-shot setting. The problems and solutions in dataset were originally collected from a math learning website. Then the author manually filtered them to ensure there’s no problem can be solved without the tabular context and numerical reasoning process and checked the solutions’ correctness. The idea of the method proposed is trying to find the most appropriate in-context examples for a given problem in test time. The policy to choose in-context example is learnt by a reinforcement learning policy gradient strategy.", "summary_of_the_review": "The main problem of this paper is that the experiments do not validate the motivation of the paper. \n(1) Is the selection of examples important for the TABMWP dataset dataset? PROMPTPG needs to be compared with manually chosen and fixed in-context example.\n(2) The policy gradient method designed in this paper is independent of the tabular data. Therefore, why this method is proposed for tabular data is not explained. Although the authors mentioned in Section 3.2 that \"may be more severe on TABMWP\", this has not been verified.", "strength_and_weaknesses": "Strength:\na)\tThe idea of using reinforcement learning policy gradient strategy to train an agent to find helpful prompting examples is interesting. It enables “automatic prompt-tuning” to some extent, since directly tuning GPT-3 is hard.\nb)\tThe author provided a math problem dataset annotated with quality text solutions. It is helpful for researches which use general-purposed LM to perform mathematical reasoning tasks (e.g. “chain-of-thought”).\nc)\tThe author conducted a rich set of experiments to test the method and incorporated human performance.\n\nWeakness:\na)\tThe idea of constructing a tabular math problem dataset is not very inspiring.\nb)\tSome statements are contradictory. In the introduction, the author claimed that retrieving semantically similar examples might not work well. But in the ablation study part, the result indicates that selecting similar examples as prompt does make an improvement.\nc)\tPROMPTPG needs to be compared with manually chosen and fixed in-context example. If one randomly chooses example for every test-time problem, obviously it can increase the variance of accuracy.\nd)\tDynamically selecting prompt examples still might not be enough. It relies on quality human annotated solutions (when “chain-of-thought” is needed). Sometimes the effort to obtain such annotations is non-trivial.\ne)\tThough ablation study and case study were performed, there is no in-depth analysis of them.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ZKQLeAF0-V", "reviewer": "Reviewer_G2Gx", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces TabMWP - a dataset containing open-domain grade-level problems that require mathematical reasoning on both textual and tabular data. It evaluates a number of models and methods on this dataset and finds that chain-of-thought prompting on GPT-3 is the strongest baseline. Inspired by the sensitivity of the chain-of-thought performance to the choice of prompt examples, the paper further introduces a method to learn to predict a dynamic set of prompt examples per test-question, where the model is trained through REINFORCE to output prompt examples that would maximize the expected accuracy on the test question.", "summary_of_the_review": "This paper introduces a tabular math word dataset and benchmarks a number of methods on this task. The paper further introduces an RL-based method for dynamically choosing the prompt examples to be used for in-context learning. Both are novel and significant contributions to the best of my knowledge.", "strength_and_weaknesses": "### Strength:\n\n- This paper introduces a well-designed math word dataset that contains tabular information, upon which the answers depend. The dataset contains diversity in terms of difficulty level (from different grades), answer format (free-text or multiple choice) and table format (structured, semi-structured, and image), along with the associated answer rationales. This is extremely valuable to the community for evaluating the performance of models when working with tabular datasets.\n\n- The dynamic prompting method via policy gradient appears novel and interesting, and has been shown to improve performance over a number of baseline strategies.\n\n- The experiments are well-designed and covers a broad range of ablations and a large set of baselines, including human performance. This provides a very valuable starting point for the community to benchmark on this dataset.\n\n### Weakness:\n\nI have several questions that I hope the authors can answer:\n\n- Is promptPG using rationale augmented few shot examples? What happens if it uses simple few-shot without rationales?\n\n- What happens if we increase the number of shots in-context? Would using more than 2 shots reduce the variance that come from choosing different prompt examples?\n\n- Clarify what it means to have or not have C (choice options) in Table 4.\n\n- Can you train for a fixed set of in-context examples? What would be the performance gap? It would be interesting to see how much does the improvement come from dynamic prompt examples rather than an optimized set of static examples.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "WKQi6rX6N2X", "reviewer": "Reviewer_MEHB", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposed a novel dataset, TABMWP, for math word problems with tabular context, which contains over 30,000 open-domain problems that require multiple steps of reasoning. On top of the dataset, it further proposed a new approach to learning to select in-context examples and construct the performing prompt for the test example. It then conducted extensive experiments evaluating the performance of the proposed approach, GPT-3, state-of-the-art QA, and TableQA methods on the new dataset TABMWP. The experiment results demonstrate the better performance of PromptPG in many cases.", "summary_of_the_review": "This work proposed a novel and high-quality dataset TABMWP for math word problems with tabular context, which could be used as a benchmark dataset of tabular reasoning domains. It then proposed a new approach PromptPG which applies reinforcement learning to select in-context examples for the few-shot GPT-3. The work also conducts extensive experiments to evaluate the proposed approach over many other baselines. ", "strength_and_weaknesses": "Strength:\n-\n\n[+] The proposed dataset is novel and of high quality.\n\n[+] The proposed approach, built on top of [1] and [2], applies reinforcement learning to select in-context examples for the few-shot GPT-3 model and has better performance over state-of-the-art QA, and TableQA methods in most settings.\n\n[+] The proposed approach also has better performance than some other selection strategies of in-context examples.\n\nWeakness:\n-\n\n[-] The authors should investigate the performance gain of PromptPG over prompting baselines w/ GPT-3 in relation to the usage of many times more training examples.\n\n[-] The presentation of this work misses some important details of the proposed approach PromptPG (see below).\n\nA few detailed questions related to the proposed approach PromptPG:\n\n1. What is the structure of the prompt creator in Figure 2? Does it contain learnable parameters?\n\n2. How does PromptPG handle the structured tabular data exactly? Are there steps for generating queries to the tabular data? If so, where are the steps in Figure 2?\n\n3. How do you select the candidate examples? Are they randomly drawn from the training data? How would different candidate pools affect the performance of selecting in-context examples?\n\n---\n\n[1] Jiachang Liu, Dinghan Shen, Yizhe Zhang, William B Dolan, Lawrence Carin, and Weizhu Chen.\nWhat makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out\n(DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning\nArchitectures, pp. 100–114, 2022a.\n\n[2] Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, and Jian-Guang Lou.\nTapex: Table pre-training via learning a neural SQL executor. In International Conference on\nLearning Representations (ICLR), 2022b.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "M-STnt66zo", "reviewer": "Reviewer_MYrE", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "1. This paper proposes TabMWP, a dataset of 30k examples containing grade-level problems that require mathematical reasoning on both textual and tabular data.\n2. The paper establishes a number of baselines for the dataset\n3. The paper proposed a policy gradient method to select few-shot in-context examples", "summary_of_the_review": "this is a pretty complete piece of work that has a new dataset and comprehensive experimental evaluation on the dataset.  whether this dataset can inspire important future research is unclear.", "strength_and_weaknesses": "Strength\n1. The paper is very well-written and easy to follow\n2. the baseline evaluation of the proposed dataset is comprehensive\n\nWeakness\n1. The paper would benefit from a related work subsection that discusses if policy gradient has been applied in similar setup, or is this totally new?\n2. There are a lot of math NLP datasets out there.  I understand this may be the first dataset that tests reasoning on both textual and tabular data, but it is unclear how this dataset would push forward NLP research.  Unlike text + image, there's already a quite natural way of combining table and text, so I'm not sure how this dataset may open up new research directions.  So this is a concern of the potential impact of this dataset.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster