How to find classification accuracy of the discriminator of a GAN model?

shrbrh: <p>Hello. I am training a GAN model for style transfer. I have used 20 different styles for the training. My discriminator is a classifier that tries to classify these 20 styles. I have saved the discriminator as a .pth.tar model and I want to test the classification accuracy of it. Is it possible to do that in pytorch? I can’t find a suitable method. Please help.</p>

Samuel_Bachorik: <p>You can load your saved weights with, make sure your model weights are in same directory as your code…</p>
<pre><code class="lang-auto">model = Model()
PATH = "./model_weights.pth"
model.load_state_dict(torch.load(PATH))
</code></pre>
<p>Then you can get prediction from trained model:</p>
<pre><code class="lang-auto">prediction = model.forward(x)
</code></pre>

shrbrh: <p>Thank you for your reply. Can you please clarify what is ‘x’ in model.forward(x)?</p>

Samuel_Bachorik: <p>It is your input data</p>

nwn: <p>Hello,<br>
firstly, some additions to <a class="mention" href="/u/samuel_bachorik">@Samuel_Bachorik</a> answer: to get the classifiation accuracy, I would apply a softmax to the output layer and then choose the max. value of the retrieved probabilies.<br>
Secondly and most importantly, the approach that you have choosen might not be the most flexible one. If you want a more flexible approach that allows you to control the discriminator and the classification loss, you could seperate the the loss generating part of your architecture into the Discriminator and a separate model that is used to classify the different styles based on the latent space. <a class="mention" href="/u/shrbrh">@shrbrh</a> Are you using styleGAN?</p>
<p>Best,<br>
nwn</p>

shrbrh: <p>I have used the StarGAN discriminator and I did add a softmax layer to its output layer. The output layers of StarGAN discriminator is coded as:</p>
<pre><code class="lang-auto">self.conv1 = nn.Conv2d(curr_dim, 1, kernel_size=3, stride=1, padding=1, bias=False)
self.conv2 = nn.Conv2d(curr_dim, c_dim, kernel_size=kernel_size, bias=False)
</code></pre>
<p>I added a softmax layer after <code>self.conv2</code> where c_dim = 20 are for the 20 styles. I trained the GAN model and saved the models for both the generator and the discriminator. Now I want to load my saved discriminator model and find its classification accuracy for a particular stylized test image. So how do I retrieve the probabilities and max. value from that?</p>

J_Johnson: <p><code>x</code> in a GAN should be just <code>x=torch.rand(output_dims)</code>. Let’s say your model gives outputs of <code>print(output.size())</code> and that gives you <code>torch.size(batch_size, channels, Height, Width)</code>, for example, <code>(10, 3, 224, 224)</code>. Just make a random tensor of that size for input.</p>
<p>Regarding to your original question, I am not familiar with any metric “accuracy” for a GAN. Usually, just the loss of the descriminator and generator is considered. There are several proposed loss functions:</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://developers.google.com/machine-learning/gan/loss">
  <header class="source">
      <img src="https://www.gstatic.com/devrel-devsite/prod/vdbc400b97a86c8815ab6ee057e8dc91626aee8cf89b10f7d89037e5a33539f53/developers/images/favicon.png" class="site-icon" width="32" height="32">

      <a href="https://developers.google.com/machine-learning/gan/loss" target="_blank" rel="noopener nofollow ugc">Google Developers</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/362;"><img src="https://www.gstatic.com/devrel-devsite/prod/vdbc400b97a86c8815ab6ee057e8dc91626aee8cf89b10f7d89037e5a33539f53/developers/images/opengraph/white.png" class="thumbnail" width="690" height="362"></div>

<h3><a href="https://developers.google.com/machine-learning/gan/loss" target="_blank" rel="noopener nofollow ugc">Loss Functions  |  Machine Learning  |  Google Developers</a></h3>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>But you should probably just stick with one type for training and analysis.</p>

nwn: <p>So, do you want to load a specific plain image, sytle it and then test the classification acc. of specific styles?</p>

shrbrh: <p>Yes. I want to take a content image and style image and use the generator of the GAN model to produce a stylized image. Then I want the discriminator to classify the stylized image and I want to test its classification accuracy.</p>

nwn: <p>As shown in Figure 3 of the <a href="https://arxiv.org/pdf/1711.09020.pdf" rel="noopener nofollow ugc">StarGAN paper</a> it appears to me that you give both to your Generator which produces a fake image. This fake image is then passed to the discriminator which then returns True or False for the Discriminator thinkining the image is “Real” or “Fake” <strong>and</strong> a domain classification score. Have you already tried to use this score?</p>