SlowGuess/ACL_2020_OCR huggingface dataset:
Some data samples for Ubuntu, Switchboard, Opensubtitles are shown in Table 7.
Ubuntu
A: anyone here got an ati hd 2400 pro card working with ubuntu and compiz ?
B: i have an hd 3850
A: is it working with compiz ?
Switchboard
A: what movies have you seen lately
B: lately i ’ve seen soap dish
A: oh
B: which was a
A: that was a lot of fun
OpenSubtitles
B: you can’t do that .
A: my husband ’s asleep .
B: your husband know you ’re soliciting ?
A: give us a f*** ’ break .
Table 7: Data samples of Ubuntu, Switchboard and OpenSubtitles Dialogue corpus
Model	Test-PPL(NLL)
Ubuntu	Switchboard	OpenSubtitles
LM	66.29(4.19)	44.37(3.79)	74.74(4.31)
Seq2seq	59.49(4.08)	42.81(3.75)	70.81(4.26)
Table 8: Perplexity (PPL) and negative log-likelihood (NLL) of for baseline models on the test set.
Baseline perplexity results are shown Table 8. Note that  $T_{in}$  and  $T_{out}$  for various types of hit types discussed in Section 3.2 are set accordingly, for example, for io-sample-min-hit on the Ubuntu data,  $T_{in}$  is set to -4.19, and  $T_{out}$  is set to -4.08.
D Auxiliary Experiment Results for the Malicious Response Problem
We compare the models behavior before and after negative training in Figure 1. It is shown that negative training effectively reduce probability mass assigned to malicious targets, while keeping the behavior on the test-set unchanged. However, almost every word in the malicious target sentences gets lower probability, especially when FWA is not used. Ideally, we believe a "polite" language generator should only assign low probability to the key words in a malicious sentence. For example, in the target "i shall take my revenge", only the "take my revenge" part should be penalized. Whether negative training has the potential to truly
$^{12}$ http://compprag.christopherpotts.net/swda.html
<sup>13</sup>http://www.opensubtitles.org/
Figure 1: Negative Log-probability (NLL) the model assigned to the test list malicious targets (when fed with trigger inputs) or test data samples. The data-set is OpenSubtitles and hit type is io-sample-min-hit. Sentences are separated by  $<\mathrm{EOS}>$ .
teach "manners" to a language generator is worth further investigation.
E Configurations of the GAN Approach for Dialogue Response Generation
We use the log derivative trick (Wu et al., 2017) for the gradient derivation of the generator:
$$
\begin{array}{l} \nabla_ {\theta_ {G}} V (D, G; \boldsymbol {x}) \\ = \nabla_ {\theta_ {G}} E _ {\boldsymbol {y} \sim G (\cdot | \boldsymbol {x})} \log (1 - D (\boldsymbol {x}, \boldsymbol {y})) \\ = E _ {\boldsymbol {y} \sim G (\cdot | \boldsymbol {x})} \nabla_ {\theta_ {G}} \log G (\boldsymbol {y} | \boldsymbol {x}) \log (1 - D (\boldsymbol {x}, \boldsymbol {y})) \tag {11} \\ \end{array}
$$
where  $\pmb{x}$  is one input data sample. Then the generator is updated by:
$$
\theta_ {G} \leftarrow \theta_ {G} - \alpha_ {G} \cdot \nabla_ {\theta_ {G}} V (D, G) \tag {12}
$$
where  $\alpha_{G}$  is the learning rate for the generator. Note that because  $\log (1 - D(\pmb {x},\pmb {y}))$  is negative,  $\nabla_{\theta_G}\log G(\pmb {y}|\pmb {x})$  will be eventually scaled positively and added to  $\theta_{G}$
In our GAN experiments, different values in the set  $\{0.01, 0.001, 0.0001\}$  are tried for  $\alpha_{G}$  and the best result is reported.
We now describe the model configuration of the discriminator  $D(\pmb{x},\pmb{y})$  used in our work. The discriminator model configuration is similar to the one used in (Yu et al., 2016). First  $\pmb{x}_t$  is converted to  $\pmb{x}_t^{emb}$  as described in Section 2. Then a 1D-convolution operation and max-over-time pooling operation (Kim, 2014) is applied, with 300 filters
of window size  $3 / 4 / 5 / 6$ , respectively. The resulting representation vector is denoted as  $\pmb{x}_{\mathrm{rep}}$ .
The same network forward pass is also applied for  $\pmb{y}$  to get  $y_{\mathrm{rep}}$ . Finally,  $x_{\mathrm{rep}}$  and  $y_{\mathrm{rep}}$  are concatenated and passed to a 3-layer high-way DNN classifier (Srivastava et al., 2015) of hidden size 2000.
Following (Goodfellow et al., 2014a), we alternately train the discriminator and the generator with a ratio of 3:1. The discriminator is trained with a learning rate of 0.01. Similar to negative training, our experiments show that positive training (or "teacher forcing" in some literature) is crucial to aid the model to maintain its original performance, during GAN training.
F Auxiliary Experiment Results for the Frequent Response Problem
In Talbe 9, we show BLEU-4 scores for the model after negative training. It is shown that the BLEU-4 performance does not become worse (or even improves) after negative training. This result, to some extent, verifies our claim that the quality of model samples is not compromised.
G More Model Samples for Comparison
We show the model samples before/after negative training for the malicious response problem in Talbe 10. It is shown that the model's generation behavior does not change much.
For the frequent response problem, we show
Ubuntu	PPL	BLEU-4	M-ratio
Baseline	59.49	0.443	4.4%
Neg-train	60.06	0.464	1.3%
Switchboard	PPL	BLEU-4	M-ratio
Baseline	42.81	0.476	37.4%
Neg-train	44.32	0.475	9.8%
OpenSubtitles	PPL	BLEU-4	M-ratio
Baseline	70.81	0.502	20%
Neg-train	75.71	0.526	0.6%
Table 9: BLEU-4 results for negative training.
Input: he had worked for united fruit company there
Baseline: it was kind of a frightening senators
Neg-train: it was good housekeeping
Input: uh-huh
Baseline: right
Neg-train: right
Input: they in fact they had
Baseline: golly
Neg-train: joe came back from alabama
Input: they in fact used some of the equipment that
had been left over
Baseline: i do n’t think i ’m fanatic very many
Neg-train: i do n’t know
Input: he turned them down
Baseline: i love the game
Neg-train: i love it
Table 10: Consecutive model samples before and after negative training for preventing malicious response on Switchboard data.
more decoding samples for comparison of different training methods in Table 11, 12 and 13.
Input: people do n't realize it
Baseline: i do n't know
GAN: i do n't know
Neg-train: i think it's a very interesting
Input: they're sort of siamese in a way
Baseline: uh-huh
GAN:uh-huh
Neg-train: yeah
Input: they have the gloves on the paws
Neg-train: i'm really glad that they do n't have that
Input: they're usually two colored
Baseline: i think that 's a good idea
GAN: i think that 's a good idea
Input: she's predominantly black
GAN: he's a little bit
Neg-train: he's a very good actor
Input: she has chocolate paws chocolate stomach
and silver on her hindquarters
Neg-train: i think they have a great deal of <unk>
Input: she's quite attractive looking
Baseline: he's a good actor
Neg-train: he's a good guy
Input: hope the stud will find her attractive
Neg-train: i've been in a very very good shape
Input: that she has kittens that look like her
Baseline: i think that's a good idea
Input: that she has a whole bunch of kittens
Input: we have to be careful
Baseline: i 'm not sure
GAN: i'm not sure
Neg-train: i'm in theDallas area
Input: we have to get a particular color point stud
Neg-train: i'm just going to
Input: you can't just breed them with anything
Neg-train: i have n't been to a lot of the movies lately
Input: we have to get a silver point himie
Neg-train: i'm going to have to go back to work
Table 11: Consecutive model samples before and after negative training for Switchboard data.
Input: just how good ubuntu
Baseline: i'm not sure i'm just a <unk>
GAN: i'm not sure if it's possible but
i'm sure there's a way to do it
Neg-train: i have a server that i want
to install on a server
Input: help please i broke xp with linux_url_
i can't get an invite
Baseline: what is the problem
GAN: what is the problem
Neg-train: what is the issue
Input: the case it's possible that you might need to repair the xp installation
Baseline: i have a <unk> and i want to install ubuntu
on a usb drive
GAN: i have a <unk> and i want to install ubuntu
Neg-train: i have a live cd and i want to install
it on a hard drive
Input: but that takes hours
Baseline: i do n't know i'm not sure
Neg-train: i'm not sure it's a hardware problem
Input: how to i enable tv-out via fglrx
Baseline: you can use the <unk> driver
GAN: you can use the nvidia driver from the wiki
Neg-train: you can try the <unk> driver
Input: they may be able to help you in channel # ati
Baseline: thanks
GAN: thanks
Neg-train: i have a ati radeon mobility radeon hd
Input: nobody feels like responding in # ati
Baseline: i have no idea what you're talking about
GAN: i have no idea
Neg-train: i think i found it
Input: its always worked perfectly for me why 's it crap
Baseline: i do n't know i 'm not sure
GAN: i do n't know i 'm not sure
Neg-train: i do n't know i just use it to make it work
Input: i have seen better performance on an
xp vm under parallels
Baseline: i have a <unk> and i have a <unk>
GAN: i have a <unk> and a <unk> <unk>
Neg-train: i have a vm with vmware
server running on it
Input: use 2kpro and no issues whatsoever with 256mb
of ram and a 10gig vdrive
Baseline: i have a <unk> and it's not a big deal
GAN: i have a  $<\text{unk}>$  and it's not a big deal
Neg-train: i have a p4 with 1gb ram and i have a
1gb swap
Input: will someone please help me with php
config on ubuntu
Baseline: what's the problem
GAN: what's the problem
Neg-train: what's wrong
Input: you are having trouble with
if someone knows how to help they will try
Baseline: i 'm trying to get a <unk> to
work with ubuntu
GAN: i'm not sure i'm just trying to help you
Neg-train: i'm trying to get a <unk> of my <unk>
and i want to know if it 's possible
Opensubtitles
Input: i feel really bad marco so bad
Baseline: i'm sorry
GAN: i'm not a <unk>
Neg-train: i feel so bad
Input: me at least that you 'll come back tell me
you're not ill
Baseline: i'm not going to be in the house
GAN: i'm not going to let you go
Neg-train: i'm so glad you are
Input: tell me you forgive me
Baseline: i'm not going to let you go
GAN: i'm not going to leave you
Neg-train: you're not the only one
Input: will you forgive me
Baseline: i'll be all right
GAN: i'll be all right
Neg-train: you 'll never see the
Input: what do you want
Baseline: i'm going to the hospital
GAN: i'm going to the <unk>
Neg-train: i want to go
Input: get up hurry i got an idea
Baseline: i 'm not going to be a <unk>
GAN: i'm not going to see you
Neg-train: you're not getting a bad idea
Input: where are we going
GAN: to the <unk>
Neg-train: to the hotel
Input: shut up and keep down
Baseline: i'm not going to get a doctor
GAN: i'm not going to get a doctor
Neg-train: we're not going
Input: we're screwed there's no wind
Baseline: i'm not going to see you
Neg-train: we're in the <unk>
Input: how 'd your old lady come to think you
was kidnapped
Neg-train: i'd like to ask you to take a look at it
Input: your son is in our hands
Baseline: i'm not going to
GAN: he's not here.
Neg-train: he's not going to leave
Input: we 'll stay in touch
GAN: i'm not going to
Neg-train: i think we've got a <unk>
Input: get the money ready or else
GAN: i'm not going to be here
Neg-train: i want the <unk>
Input: i think they got it
Baseline: you know what
GAN:oh
Neg-train: it's a good thing
Input: you wanted to scare them
Neg-train: i'm a coward
Table 12: Consecutive model samples before and after negative training for Ubuntu data.
Table 13: Consecutive model samples before and after negative training for Opensubtitles data.