SlowGuess/NAACL_2022_OCR huggingface dataset:
Test Data Train Data	SGD	MultiWOZ
Origin	AutoAug	HumanAug	Origin	AutoAug	HumanAug
Origin	55.6±0.7	24.2±00.6	21.1±0.8	67.6±0.7	48.8±0.5	48.8±0.1
Origin+Syn2%	57.5±1.4	27.9±2.5	25.2±1.8	65.0±0.3	48.9±1.4	49.4±1.6
Origin+Syn100%	57.1±0.6	34.4±1.8	30.4±1.5	67.0±0.7	55.4±2.6	55.6±2.9
AutoAug	55.1±0.2	49.6±0.5	43.7±0.8	63.3±1.6	74.4±2.5	73.9±2.9
AutoAug+Syn2%	56.9±0.4	54.8±1.0	48.8±1.7	64.2±0.7	83.8±0.2	83.0±0.7
AutoAug+Syn100%	56.7±0.9	58.3±0.1	50.1±0.2	63.3±1.3	84.6±0.2	83.7±0.7
Upsample	55.8±0.7	25.5±0.7	22.1±0.2	63.5±1.0	84.6±3.0	83.7±3.2
Upsample+Syn100%	58.6±0.4	35.3±0.8	32.0±0.9	63.3±0.5	88.4±0.7	88.3±0.8
PreFinetuneOrigin	55.8±0.6	23.8±0.2	21.5±0.5	67.8±0.4	44.1±1.2	0.441±1.3
PreFinetuneAug	56.3±0.4	27.4±0.4	24.3±0.5	68.4±0.3	50.5±1.2	0.495±1.1
PreFinetuneAug+Syn100%	57.4±0.8	35.7±1.6	32.8±0.6	68.5±0.9	65.0±0.9	65.8±0.6
HumanAug	55.9±0.8	50.6±2.7	51.4±2.3	-	-	-
Table 5: The complete results in terms of the named entity accuracy for only the augmented turns.
Test Data Train Data	SGD	MultiWOZ
Origin	AutoAug	HumanAug	Origin	AutoAug	HumanAug
Origin	48.9±0.7	47.7±0.7	47.7±0.7	53.5±0.1	52.2±0.5	52.3±0.4
Origin+Syn2%	50.0±0.3	48.9±0.4	49.0±0.4	53.0±0.1	50.0±0.6	50.1±0.6
Origin+Syn100%	49.5±0.6	48.7±0.5	48.7±0.5	52.8±0.3	50.4±0.5	50.4±0.4
AutoAug	50.2±1.0	49.9±1.0	49.7±1.0	52.4±0.4	53.5±0.3	53.5±0.3
AutoAug+Syn2%	49.8±0.4	49.6±0.4	49.4±0.4	52.5±0.2	54.5±0.1	54.5±0.1
AutoAug+Syn100%	51.0±0.4	50.9±0.4	50.6±0.4	53.2±0.2	55.2±0.4	55.2±0.4
Upsample	49.1±0.5	48.1±0.5	48.0±0.5	52.8±0.2	54.4±0.2	54.3±0.2
Upsample+Syn100%	49.4±0.4	48.6±0.4	48.6±0.4	52.6±0.2	54.3±0.1	54.2±0.1
PreFinetuneOrigin	48.9±0.9	47.7±0.9	47.7±0.8	53.7±0.2	51.7±0.1	51.8±0.2
PreFineAug	48.9±0.2	47.7±0.3	47.8±0.2	53.4±0.6	52.2±0.6	52.2±0.7
PreFineAug+Syn100%	49.7±0.1	48.9±0.1	48.9±0.0	54.0±0.3	52.9±0.5	52.9±0.5
HumanAug	50.1±0.9	49.7±0.8	49.7±0.8	-	-	-
Table 6: Complete Results in terms of the joint goal accuracy evaluated on the whole test set.
Test Data Train Data	SGD	MultiWOZ
Origin	AutoAug	HumanAug	Origin	AutoAug	HumanAug
Origin	87.1±0.4	85.7±0.4	85.7±0.4	83.9±0.1	81.0±0.3	81.0±0.3
Origin+Syn2%	87.9±0.1	86.6±0.1	86.6±0.1	83.3±0.1	79.9±0.4	79.9±0.4
Origin+Syn100%	87.6±0.1	86.6±0.1	86.6±0.1	83.5±0.2	80.3±0.3	80.3±0.3
AutoAug	87.7±0.6	87.4±0.5	87.2±0.5	82.8±0.5	84.5±0.6	84.4±0.7
AutoAug+Syn2%	87.9±0.3	87.8±0.2	87.6±0.2	82.6±0.1	86.0±0.2	85.9±0.2
AutoAug+Syn100%	88.5±0.4	88.6±0.4	88.2±0.4	83.0±0.4	87.0±0.5	87.0±0.5
Table 7: The accuracy of the named entity prediction for the whole test set.
Test Data Train Data	SGD	MultiWOZ
Origin	AutoAug	HumanAug	Origin	AutoAug	HumanAug
Origin	36.9±0.4	13.1±0.4	10.1±0.8	36.5±0.9	26.4±1.5	26.9±1.1
Origin+Syn2%	38.3±0.8	15.0±1.5	12.6±1.1	35.2±0.7	13.8±3.3	14.2±3.9
Origin+Syn100%	37.2±0.8	19.0±1.2	16.0±1.0	36.5±0.6	19.7±4.0	19.2±3.5
AutoAug	35.8±0.4	30.3±0.5	23.8±0.5	33.8±0.5	41.9±0.1	41.5±0.7
AutoAug+Syn2%	37.7±0.5	33.1±0.8	26.8±1.7	33.8±1.5	47.9±0.5	46.9±1.1
AutoAug+Syn100%	37.9±1.5	35.1±0.1	28.6±0.9	34.9±2.0	47.9±0.8	48.1±1.1
Table 8: Joint goal accuracy evaluated on only the augmented turns.
Addressing Method	Acc
Direct	1
Positional	1
Direct+Positional	0.9996
Attributes	0.9970
Direct+Posi+Attr	0.9993
Direct+Posi+Attr+Multiple	0.6695
Direct+Posi+Attr+Typo	1
Direct+Posi+Attrs+Multiple+Typo	0.6794
Table 9: Impact of different addressing methods. We adopt different addressing methods to synthesize single-turn dialog data, based on which we train and evaluate models. "Posi" refers to the positional addressing and "Attr" represents the addressing with attributes.
D Interface of Human Paraphrasing
DATA  $\{\{\mathrm{usr\_utt\_pre}\} \{\{\mathrm{sys\_utt}\} \{\{\mathrm{result}\} \{\{\mathrm{usr\_utt}\} \{\{\mathrm{sys\_utt\_next}\}\}$
Templated Dialog Utterances:
User: i want to find a 3-star hotel with wifi and parking lot.
System: there are two options here, which one do you prefer, gonville hotel or autumn house.
User: i would like to know more about the gonville hotel, that would be great. can you book it on tuesday for 3 people and 5 nights ?
System: the book is successful. is there anything else i can do for you?
Paraphrase ----> I prefer the gonville hotel, can you book it on tuesday for 3 people and 5 nights?
Your case:
User: i want something to do in or around nyc and musical shows are one of my favorites.
System: can you please help me identify which one you'd like to know about, crooked colours, buddy guy or abbi jacobson?
(Result: abbi jacobson)
User:let me clear that up , i'm asking about the last one . that does interest me. is there a direct bus i can take to get there?(need paraphrase)
System: would you take the bus on march 11th?
QUESTION | text box (single line)
Paraphase for the user utterance (highlighted in green)
E Guidelines of Human Paraphrasing
Any reference to the following topics is inappropriate and should be labelled as inappropriate. It is not necessary to report the dialog unless it falls under one of the escalation categories listed in the above section. There will be a "Report Dialog" button in the bottom left corner of the tool (shown above, to the left of the Data Error button), which you can select, if the dialog contains anything that needs reporting. Even if you click the Report Dialog button, you will still be required to submit a paraphrased conversation.
1. PII
a. First Name & Last Name (just one name is not PII)
i. if the first and last name seem to be used in the a slot that would indicate a public figure, such as musical artist, fictional character, or political figure, please do not mark as containing PII. If you are unsure check if the name has a wikipedia page. If so, do not mark as containing PII.
b. phones numbers, credit card numbers, or social security numbers
c. email
d. Addresses are NOT considered PII unless they are accompanied but another piece of PII (i.e name), in which the combined information would allow you to identify the user.
2. Offensive, racist, biased and non-tolerant behavior
a. Profanity, slurs, language that is offensive to any cultural, racial, and religious groups.
b. Bias towards or unequal treatment to any cultural, racial, and religious groups.
c. Anything inconsistent with the values of tolerance and respect for diversity.
3. Violence and self-harm
a. Any content which facilitates or encourages violent behavior towards others or any form of self-harm.
b. Descriptions or depictions of violent behavior or self-harm.
c. Any reference to threats or weapons.
d. Any reference to human trafficking, child endangerment or exploitation, or animal abuse.
e. Violent or non-violent crime of any kind
4. Sexual or flirtatious behavior
a. Any reference to sexual behaviour or materials, legal or illegal.
b. Sexual or flirtatious expressions or innuendo.
c. Explicit or sexual language or physical descriptions.
5. Controversial and Polarizing Topics
a. Political opinions or politically charged people or events. General political enquiries are okay, (e.g. show me political news; Is there any coverage of the election?)
b. Religion
c. Disputed regions or events
d. Sexuality
e. Cultural practices
Figure 7: Description of sensitive topics.