SlowGuess/NAACL_2022_OCR huggingface dataset:
Model	TE	BS	LR	WD
BERT	5	4	1e-5	0.001
AlBERT	5	16	1e-5	0.001
RoBERTa	4	8	1e-5	0.01
BioBERT	5	4	1e-5	0.001
BlueBERT	5	8	1e-5	0.001
ClinicalBERT	4	4	1e-5	0.01
Table 9: Hyperparameters evaluated on the PubHealthTab dataset: training epochs (TE), batch size (BS), learning rate (LR), weight decay (WD).
Figure 5: Crowdsourcing qualification test.
In this task you will see a claim and a table.
You need to select whether the table 1) supports the claim, 2) refutes the claim, 3) is related to the claim but not providing enough information or 4) is unrelated to the claim.
If you selected "supports", "refutes" or "related but not enough information", please tick-mark the columns you used for your decision which can be found at the bottom of the page.
1. Example: SUPPORT
1. Considering the claim:
The typical Wisconsin worker makes $5,000 less each year than our neighbors in Minnesota
2. And considering the table (and its caption, if available):
State or territory	Per person income	Population
District of Columbia	$45,877	658,893
Alaska	$33,062	736,732
Minnesota	$32,638	5,457,173
Colorado	$32,357	5,355,866
Washington	$31,841	7,061,530
Rhode Island	$30,830	1,055,173
Delaware	$30,488	935,614
California	$30,441	38,802,500
lowa	$28,361	3,107,126
Wisconsin	$28,213	5,757,564
Maine	$27,978	1,330,089
Kansas	$27,870	2,904,021
3. Select if the table supports of refutes the claim.
If the table is related to the claim but does not provide enough information, select the third option ("Related but not enough information"). If the table is completely unrelated to the claim, select option "Unrelated".
Supports  $\bigcirc$  Refutes  $\bigcirc$  Related but not enough information  $\bigcirc$  Unrelated
4. If you selected "Supports", "Refutes" or "Related but not enough information", select below which column(s) from the table led to your decision:
You have to select a value for at least one of them.
State or territory Per person income Population
Explanation Text: The claim states that a typical worker in Wisconsin earns $5,000 less per year compared to a typical worker in Minnesota. We can say that this claim is supported by the table by looking at the column "Per person income". The income value in row Wisconsin is $28,213. The income in Minnesota is $32,638. This is approximately $4,500 more than Wisconsin. Therefore, we decide that the claim is supported.
Figure 6: Author-annotated crowdsourcing example.
Reference 1 of 7:
Hydrocodone has a larger conversion factor than Hydromorphone.
Opioid	Conversion factor*
Codeine	0.15
Fentanyl transdermal (in mcg/hr)	2.4
Hydrocodone	1
Hydromorphone	4
Methadone
120 mg/day	4
2140 mg/day	8
4160 mg/day	10
6180 mg/day	12
Morphine	1
Oxycodone	1.5
Oxymorphone	3
Tapentadol	0.4
Caption: TABLE 2. Morphine milligram equivalent (MME) doses for commonly prescribed opioids
$\bigcirc$  Related but not enough information
Refutes
Opioid Conversion factor
Figure 7: User Interface for the table relevance and verification task.
Reference 1 of 5:
Novartis drug cut death risk by 35 percent in gene mutation breast cancer
Stage (TNM Definitions)	Standard Treatment Options
Early/localized/operable breast cancer	Surgery with or without radiation therapy
Adjuvant therapychemotherapy, endocrine therapy, HER2-directed therapy
Locoregional recurrent breast cancer	Surgery
Radiation therapy and chemotherapy
Metastatic breast cancer	Hormone therapy and/or chemotherapy
T = primary tumor; N = regional lymph node; M = distant metastasis; HER2 = human epidermal growth factor receptor 2.
Caption: Table 2. Standard Treatment Options for Male Breast Cancer
3. Adjust the given claim such it can either verified or refuted when considering the table.
You are allowed to change the meaning of the given claim if it does not match the table. You can look at the examples from before by clicking on "Show Instructions" at the top of this page.
Write adjusted claim here...
4. Select if the adjusted claim can be verified or refuted given the table.
Refuted
Figure 8:User Interface for the claim adjustment task.