SlowGuess/ACL_2020_OCR huggingface dataset:
Guider network can be regarded as a model of the text-generation environments, namely the model of dynamics. It takes current  $s_t$  and  $\boldsymbol{a}_t$  as input, and outputting an estimate of the next state  $s_{t + \triangle t}$
Method	Test-BLEU-2	3	4	Self-BLEU-2	3	4	F1-BLEU-2	3	4
MLE (Caccia et al., 2018)	0.902	0.706	0.470	0.787	0.646	0.485	0.345	0.472	0.491
Guider (MLE)	0.920	0.723	0.489	0.812	0.589	0.360	0.312	0.524	0.554
GMGAN (Ours)	0.923	0.727	0.491	0.814	0.576	0.328	0.310	0.537	0.567
at time  $t + \triangle t$ . In the text generation setting, when  $\triangle t = 1$ , we can exactly get the feature representation of the current generated sentence if the guider does not help the word selection. If not, we cannot exactly get this feature extraction since the guider's prediction partly determine next token. In practice, we use  $\triangle t = c = 4$ , to give the guider planning ability, to help for word selection and guide sentence generation.
C.1 Adversarial Text Generation
For Image COCO, the learning rate of the generator is 0.0002, the learning rate of the guider 0.0002, the maximum length of sequence is 25. For WMT, the learning rate of the guider 0.0002, the learning rate of the guider 0.0002, the maximum length of sequence is 50. We use  $c = 4$  chosen from [2,3,4,5,8] and  $\gamma = 0.25$  chosen from [0.1, 0.25, 0.5, 0.75, 0.99]. We use Adam (Kingma and Ba, 2014) optimization algorithm to train the guider, generator and discriminator.
For both tasks, the LSTM state of dimension for the generator is 300, and the LSTM state of dimension for the generator is 300. The dimension of word-embedding is 300. The output dimension of the linear transformation connecting guider and generator is  $600 \times 10$ . The learning rate of Discriminator is 0.001.
C.2 Conditional Generation
For Image Captioning, the learning rate of the guider 0.0002, the learning rate of the guider 0.0002, the maximum length of sequence is 25. For Style transfer, the learning rate of the guider 0.0001, the learning rate of the guider 0.0001, the maximum length of sequence is 15.
C.3 Network Structure of Models
The LSTM state of dimension for the generator is 300, and the LSTM state of dimension for the guider is 300. The dimension of word-embedding is 300.
Table 10: Additional Comparison with MLE (Caccia et al., 2018).
(Sub-)sequence to latent features
Input 300× Seq. Length Sequences
5 × 300 conv. 300 ReLU, stride 2
5 × 1 conv. 600 ReLU, stride 2
MLP output 600, ReLU
Table 11: Architecture of Encoder.
Sequence to a scalar value
Input 300× Seq. Length Sequences
5 × 300 conv. 300 ReLU, stride 2
5 × 1 conv. 600 ReLU, stride 2
MLP output 1, ReLU
Table 12: Architecture of Discriminator.
D Algorithm Details
Algorithm 2 Guider Matching Generative Adversarial Network (GMGAN)
Require: generator policy  $\pi^{\phi}$ ; discriminator  $D_{\theta}$ ; guider network  $G^{\psi}$ ; a sequence dataset  $\mathcal{S} = \{X_{1\dots T}\}$ .
1: Initialize  $G^{\psi}$ ,  $\pi^{\phi}$ ,  $D^{\theta}$  with random weights.
2: Pretrain generator  $\pi^{\phi}$ , guider  $G^{\psi}$  and discriminator  $D^{\theta}$  with MLE loss.
4: for g-steps do
5: Generate a sequence  $Y_{1\dots T}\sim \pi^{\phi}$
6: Compute  $Q_{t}$  via (5), and update  $\pi^{\phi}$  with policy gradient via (8).
8: for d-steps do
9: Generate a sequence from  $\pi^{\phi}$ .
10: Train discriminator  $D_{\theta}$
11: end for
12: until GMGAN converges
Res152-SCST: a group of zebras standing in a field.
Res152-GMST: a herd of zebras standing in a field of grass.
Tag-SCST: a zebra and a zebra drinking water from a field of grass.
Tag-GMST: a group of zebras drinking water in the field of grass.	Res152-SCST: a group of people walking down a skateboard.
Res152-GMST: a group of people standing on a street with a skateboard.
Tag-SCST: a woman walking down a street with a skateboard.
Tag-GMST: a black and white photo of a man riding a skateboard.
Res152-SCST: a baby sitting next to a baby giraffe.
Res152-GMST: a little baby sitting next to a baby holding a teddy bear.
Tag-SCST: a black and white photo of a woman holding a teddy bear.
Tag-GMST: a black and white photo of a man and a woman holding a teddy bear.	Res152-SCST: a traffic light on a street with a in the.
Res152-GMST: a traffic light on the side of a street.
Tag-SCST: a traffic light on a street with a green.
Tag-GMST: a red traffic light sitting on the side of a road.
Figure 4: Examples of image captioning on MS COCO.
Algorithm 3 Guider Matching Sequence Training (GMST)
Require: generator policy  $\pi^{\phi}$ ; discriminator  $D_{\theta}$ ; guider network  $G^{\psi}$ ; a sequence dataset  $S = \{Y_{1\dots T}\}$  and its condition information  $\mathcal{I} = \{X\}$
4: Generate a sequence  $Y_{1\dots T}\sim \pi^{\phi}$
5: Compute evaluation scores based on references.
6: Compute  $Q_{t}^{s}$  via (6), and update  $\pi^{\phi}$  with policy gradient via (8).
7: until GMST converges
Method Generated Examples
Table 13: Generated Examples on EMNLP2017 WMT.
Original: i'm so lucky to have found this place!
Guider: i'm so embarrassed that i picked this place.
Original: awesome place, very friendly staff and the food is great!
Guider: disgusting place, horrible staff and extremely rude customer service.
Original: this was my first time trying thai food and the waitress was amazing!
Guider: this was my first experience with the restaurant and we were absolutely disappointed.
Original: thanks to this place!
Guider: sorry but this place is horrible.
Original: the staff was warm and friendly.
Guider: the staff was slow and rude.
Original: great place and huge store.
Guider: horrible place like ass screw.
Original: the service is friendly and quick especially if you sit in the bar.
Guider: the customer service is like ok - definitely a reason for never go back..
Original: everything is always delicious and the staff is wonderful.
Guider: everything is always awful and their service is amazing.
Original: best place to have lunch and or dinner.
Guider: worst place i have ever eaten.
Original: best restaurant in the world!
Guider: worst dining experience ever!
Original: you'll be back!
Guider: you're very disappointed!
Original: you will be well cared for here!
Guider: you will not be back to spend your money.
Original: they were delicious!
Guider: they were overcooked.
Original: seriously the best service i've ever had.
Guider: seriously the worst service i've ever experienced.
Original: it's delicious!
Guider: it's awful.
Table 14: Sentiment transfer samples on Yelp dataset (positive  $\rightarrow$  negative).
Original: gross !
Guider: amazing !
Original: the place is worn out .
Guider: the place is wonderful .
Original: very bland taste .
Guider: very fresh .
Original: terrible service !
Guider: great customer service !
Original: this place totally sucks .
Guider: this place is phenomenal .
Original: this was bad experience from the start .
Guider: the food here was amazing good .
Original: very rude lady for testing my integrity .
Guider: very nice atmosphere for an amazing lunch !
Original: they recently renovated rooms but should have renovated management and staff .
Guider: great management and the staff is friendly and helpful .
Original: this store is not a good example of sprint customer service though .
Guider: this store is always good , consistent and they ’re friendly .
Original: one of my least favorite ross locations .
Guider: one of my favorite spots .
Original: horrible in attentive staff .
Guider: great front desk staff !
Original: the dining area looked like a hotel meeting room .
Guider: the dining area is nice and cool .
Original: never ever try to sell your car at co part !
Guider: highly recommend to everyone and recommend this spot for me !
Original: i ordered the filet mignon and it was not impressive at all .
Guider: i had the lamb and it was so good .
Table 15: Sentiment transfer samples on Yelp dataset (negative  $\rightarrow$  positive).