Visualizing class distribution in 2D

silvester: <p>I am training a network on mnist dataset. I wonder how I could possibly visualize the class distribution like the image below.<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/b/0bc69811e22723561ebea13e9498d3f667542854.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/0bc69811e22723561ebea13e9498d3f667542854" title="mnist"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/b/0bc69811e22723561ebea13e9498d3f667542854_2_662x500.jpeg" alt="mnist" data-base62-sha1="1GaJKvOr0OvtVrftG587ouJzqHa" width="662" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/b/0bc69811e22723561ebea13e9498d3f667542854_2_662x500.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/0/b/0bc69811e22723561ebea13e9498d3f667542854.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/b/0bc69811e22723561ebea13e9498d3f667542854.jpeg 2x" data-dominant-color="D3D0D1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">mnist</span><span class="informations">685×517 80.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

jmandivarapu1: <p>You may use either t-sne,PCA to visualize each image. But the visualization bit more different than your image. As it depends on what network you are training for MNIST dataset. If you train an autoencoder then pass the dataset to get the latent vectors then plot then you might see bit more structure.</p>
<p>PFB link for generic visualization of MNIST</p>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://cdn-static-1.medium.com/_/fp/icons/favicon-rebrand-medium.3Y6xpZ-0FSdWDnPM3hSBIA.ico" class="site-icon" width="16" height="16">
      <a href="https://medium.com/analytics-vidhya/principal-component-analysis-pca-with-code-on-mnist-dataset-da7de0d07c22" target="_blank" rel="nofollow noopener" title="05:09AM - 19 September 2019">Medium – 19 Sep 19</a>
  </header>
  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:128/77;"><img src="https://miro.medium.com/max/602/0*PnqMbZEdnuL9yHuo.png" class="thumbnail" width="128" height="77"></div>

<h3><a href="https://medium.com/analytics-vidhya/principal-component-analysis-pca-with-code-on-mnist-dataset-da7de0d07c22" target="_blank" rel="nofollow noopener">Principal component analysis(PCA) with code on MNIST dataset</a></h3>

<p>PCA explained using examples and implemented on the MNIST dataset. We also explore the drawbacks of PCA and where it can’t be used.</p>

  <p><span class="label1">Reading time: 4 min read</span>
    </p>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


silvester: <p>Thanks for the link. I wonder in which layer’s output of the model, tnse or pca should be applied to represent the decision boundaries of the model.</p>

jmandivarapu1: <p>If you are using VGG or any classifier you can take any layer after the avgpool I guess or flatten the input image 28*28 to 784.</p>

silvester: <p>In this case, I wonder what would be the iterpretation of logit layer in 2D embedding? Shouldn’t be the last hidden feature layer?</p>

jmandivarapu1: <p>you can think it of anyway depends on which features you want to visualize. usually people do visualization of data (28*28=784) before the training to understand how they are distributed and in the case of autoencoder they visualize after and before the training to know how the latent space was changed after the training.</p>

silvester: <p>Thanks for the illustration.<br>
Is it possible to train the center points of those areas in tsne representations, in the sense that some number of center points equal to the number of classes is hard coded and then train the classifier to collect the sample from each class around the corresponding center point?</p>

silvester: <p>When I use tsne with different parameters, I get completely different plots.<br>
For example for<br>
<code>model_tsne = TSNE(n_components=2, random_state=0)</code><br>
and<br>
<code>model_tsne = TSNE(perplexity=30, n_components=2, init='pca', n_iter=5000)</code></p>
<p>I get the plots below:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/1/8/18de811c4c00c23fc26cad23cd5cc64b9cd6f960.png" alt="tsne_embedding" data-base62-sha1="3y0aHC5KrSA057eEZrEOLINMEgg" width="640" height="480"> <img src="https://discuss.pytorch.org/uploads/default/original/3X/d/5/d588f62a83ec11a262d6357ddcb68a69aff22f17.png" alt="tsne_embedding_" data-base62-sha1="ut18Gm23rwLDFS7zZ4Bw6HM96pp" width="640" height="480"></p>

KAISER1997: <p><a class="mention" href="/u/silvester">@silvester</a> Hi, were you able to visualise the distribution as the original image you showed?</p>

silvester: <p>Is there a way to visualize the decision boundaries?</p>