Using BertForNextSentencePrediction and GPT2LMHeadModel in a GAN setup.
Author: jroakes
Pull request: no

I am using the following code (**Training Loop**) as the meat of the training loop whereby the discriminator is BertForNextSentencePrediction and the generator is GPT2LMHeadModel.  I have also included the structure of the training data (**Input data:**). The loss in the generator and discriminator appear to be falling correctly, but I have been unable to test successfully whether the model weights are being updated each epoch.

This is the section that I am concerned about correctly updating the weights of the generator:
```
               #g_loss is discriminator loss of real_sentence and generated next sentence   

		# Set generator to train mode
		self.generator.train()
			
		# Backward propagation
		g_loss.backward()
		
		if (step + 1) % self.accumulation_steps == 0:
			self.gpt2_optimizer.step()
```
Would also love to know the most accurate way to test that the model weights (specifically, GPT2LMHeadModel) are being updated with each epoch.

**Input data:**
```
# Discriminator Train
#(pri_sent + nxt_sent), label=[0]
#(pri_sent + rdm_sent), label=[1]

# Generator input
#pri_sent --> gen_sent

# Generator Train (via Discriminator Loss)
#(pri_sent + gen_sent), label = [0]
#(pri_sent + rdm_sent), label = [1]
```

**Training Loop**
```
# Each iteration has a train_discriminator and train_generator phase
for phase in ['train_discriminator', 'train_generator']:
	if phase == 'train_discriminator':
		# Set discriminator to training mode
		self.discriminator.train()

		# Forward propagation
		d_loss = self.discriminator(tdata['discriminator']['tokens_tensors'],
									tdata['discriminator']['segments_tensors'],
									tdata['discriminator']['masked_tensors'],
									next_sentence_label=tdata['discriminator']['labels']).mean()
		
		if self.accumulation_steps > 1:
			d_loss = d_loss / self.accumulation_steps

		# Backward propagation
		d_loss.backward()
		
		if (step + 1) % self.accumulation_steps == 0:
			self.bert_optimizer.step()
			# Zero the discriminator parameter gradients
			self.bert_optimizer.zero_grad()

	else:
		# Set discriminator to evaluate mode
		self.discriminator.eval()

		# Forward propagation
		g_loss = self.discriminator(tdata['generator']['tokens_tensors'],
									tdata['generator']['segments_tensors'],
									tdata['generator']['masked_tensors'],
									next_sentence_label=tdata['generator']['labels']).mean()
		
		if self.accumulation_steps > 1:
			g_loss = g_loss / self.accumulation_steps
			
		# Set generator to train mode
		self.generator.train()
			
		# Backward propagation
		g_loss.backward()
		
		if (step + 1) % self.accumulation_steps == 0:
			self.gpt2_optimizer.step()
			# Zero the generator parameter gradients
			self.gpt2_optimizer.zero_grad()


d_epoch_loss += d_loss
g_epoch_loss += g_loss


# Flush cuda after epoch
torch.cuda.empty_cache()

d_epoch_loss = float(d_epoch_loss/epoch_batches)
g_epoch_loss = float(g_epoch_loss/epoch_batches)

g_epoch_loss_list.append(g_epoch_loss)
d_epoch_loss_list.append(d_epoch_loss)
```

Comments: