Accuracy of classifier not corresponding to tsne plot

louvere_sam: <p>I am making use of domain adversarial learning with a gradient reverse layer. The job of the domain discriminator in my case is to classify the domain of genuine data alone.</p>
<pre><code class="lang-auto">class GradientReversalF(torch.autograd.Function):
  @staticmethod
  def forward(ctx, x, alpha):
    "let the input unchaged"
    ctx.save_for_backward(alpha)
    return x

  @staticmethod
  def backward(ctx, grad_output):
    "reverse the gradient by multipling -alpha"  
    alpha = ctx.saved_tensors[0]
    if ctx.needs_input_grad[0]:
      grad_output = (grad_output * (-alpha))
    return (grad_output, None)


class GradientReverse(nn.Module):
  def __init__(self, alpha, *args, **kwargs):
    "Reverse GR layer hook"
    super().__init__(*args, **kwargs)
    self.alpha = torch.tensor(alpha, requires_grad=False)
    assert alpha &gt; 0, 'alpha must be &gt; 0'
    print(f"The gradient will be multiplied by: {-alpha}")

  def forward(self, x):
    return GradientReversalF.apply(x, self.alpha) 

class Discriminator(nn.Module):
    def __init__(self,block,alpha):
        super(Discriminator, self).__init__()
        self.fc1 = nn.Linear(128 * block.expansion, 128 * block.expansion)
        self.fc1.weight.data.normal_(0, 0.01)
        self.fc1.bias.data.fill_(0.0)
        self.fc2 = nn.Linear(128 * block.expansion, 3)
        self.fc2.weight.data.normal_(0, 0.3)
        self.fc2.bias.data.fill_(0.0)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.3)
        self.grl_layer = GradientReverse(alpha)

    def forward(self, feature):
        feature = self.grl_layer.forward(feature)
        feature = self.fc1(feature)
        feature = self.relu(feature)
        feature = self.dropout(feature)
        feature = self.fc2(feature)
        return feature
def domain_disc():
    torch.manual_seed(0)
    model =Discriminator(ECABasicBlock,1)
    return model
</code></pre>
<p>What is confusing me is that the first tsne plot, where genuine data is clustered together ( domain discriminator did a bad job ) gives me higher accuracy( 0.32) for the domain discriminator than the second and third tsne plots where genuine data seems more separated.( accuracies 0.26 for second and 0.02 for third respectively). Does this mean the tsne plot is not exactly reflective of how well the domain discriminator works because the tsne plot has brought down dimensions to 2 to plot?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/2/b2c72948b4f873d7332963db29e3a67bf2ca4ed8.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/b2c72948b4f873d7332963db29e3a67bf2ca4ed8" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/2/b2c72948b4f873d7332963db29e3a67bf2ca4ed8_2_690x463.jpeg" alt="image" data-base62-sha1="pvxGPn67eGKkNolNPkn8o9yKIvm" width="690" height="463" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/2/b2c72948b4f873d7332963db29e3a67bf2ca4ed8_2_690x463.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/b/2/b2c72948b4f873d7332963db29e3a67bf2ca4ed8.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/2/b2c72948b4f873d7332963db29e3a67bf2ca4ed8.jpeg 2x" data-dominant-color="C1CAB8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">998×671 206 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/7/4716739b983e5dfec7e42c414a5cb6d480f17bc0.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/4716739b983e5dfec7e42c414a5cb6d480f17bc0" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/7/4716739b983e5dfec7e42c414a5cb6d480f17bc0_2_690x487.jpeg" alt="image" data-base62-sha1="a8S0arKFgQZ6gWf3VwHK1AGJiDe" width="690" height="487" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/7/4716739b983e5dfec7e42c414a5cb6d480f17bc0_2_690x487.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/4/7/4716739b983e5dfec7e42c414a5cb6d480f17bc0.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/4/7/4716739b983e5dfec7e42c414a5cb6d480f17bc0.jpeg 2x" data-dominant-color="D2D5CD"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">931×658 146 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/4/24d5f412e209a59d14510cce9a23514c815093ad.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/24d5f412e209a59d14510cce9a23514c815093ad" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/4/24d5f412e209a59d14510cce9a23514c815093ad_2_677x500.jpeg" alt="image" data-base62-sha1="5fRA5fd4J7eyAFn2b2KwXVp0F3v" width="677" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/4/24d5f412e209a59d14510cce9a23514c815093ad_2_677x500.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/2/4/24d5f412e209a59d14510cce9a23514c815093ad.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/4/24d5f412e209a59d14510cce9a23514c815093ad.jpeg 2x" data-dominant-color="D1D5CD"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">910×672 141 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>