Tensorboard writer.add_embedding example

username: Xanthan
cooked: <p>I want to create a visualization on tensorboard of the data just like the one <a href="https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html#adding-a-projector-to-tensorboard" rel="noopener nofollow ugc">here</a>:</p>
<p>I’m loading my image data (.jpg files) from a folder and hence I get errors as <code>.data</code> and <code>.targets</code> are not present. What would be the correct way to use <code>add_embedding</code> in my case.</p>
<p>This is how I’m loading my data:</p>
<pre><code class="lang-auto">train_data_dir = "D:\\dataset\\train"
transform = transforms.Compose([transforms.Resize(255),transforms.CenterCrop(224),transforms.ToTensor()])

train_dataset = datasets.ImageFolder(train_data_dir, transform=transform)
</code></pre>

username: ptrblck
cooked: <p>The targets are used to get the class labels and pass them to <code>add_embedding</code>.<br>
However, <code>add_embedding</code> only expects an input tensor while other input arguments are optional. If you don’t have the targets or don’t want to add labels to the plot, you could skip it.</p>

username: Xanthan
cooked: <p>yeah i get that. But my problem was actually with how do I actually use <code>writer.add_embedding</code> with a <code>ImageFolder</code> object or a <code>Dataloader</code> object as none of these objects have <code>data</code> and <code>target</code> as an attribute. Later, I just tried <code>images, labels = next(iter(train_dataloader))</code> which i guess works as well. But the only problem I have is, the images in the <code>train_dataloader</code> are not grayscale, hence there is an error <code>AssertionError: #labels should equal with #data points</code>. Below is the implementation:</p>
<pre><code class="lang-auto">train_dataloader = torch.utils.data.DataLoader(train_dataset, batch_size=32, shuffle=True)
images, labels = next(iter(train_dataloader))
image, label = select_n_random(images, labels)

# get the class labels for each image
class_labels = [classes[lab] for lab in label]

# log embeddings
print(image.shape, label.shape)

features = image.view(-1, 28 * 28)
print(features.shape)
writer.add_embedding(features,
                    metadata=class_labels,
                    label_img=images.unsqueeze(1))
writer.close()
</code></pre>
<p>As, <code>image.view(-1, 28 * 28)</code> combines my batch and channels (RGB) it throws an error. So, is there a way to convert the RGB tensor to grayscale? and no I can’t use <code>transform.grayscale</code> for this (as my model requires RGB input)</p>

username: ptrblck
cooked: <p><code>ImageFolder</code> uses the internal <code>.samples</code> and <code>.targets</code> attributes so you could use these.</p>

username: Xanthan
cooked: <p>yes, that solves the <code>.data</code> missing attribute problem. But, the tensor still needs to be converted to 1 channel image before doing the operation <code>features = image.view(-1, 28 * 28)</code> , any ideas on that?</p>

username: ptrblck
cooked: <p><code>add_embedding</code> expects a 2D embedding matrix in the shape <code>[batch_size, features]</code> not an input image.<br>
I don’t know why you are trying to use the inputs, but you could transform them to grayscale (you wouldn’t pass them to the model anyway) or flatten the channel dimension with the spatial dimensions into the “feature” dimension: <code>x = x.view(x.size(0), -1)</code>.</p>

username: Xanthan
cooked: <p>oh okay got it. I solved it by extracting the features from my model (output of the fully connected layer) which I think I was suppose to do at the first place. thanks for the help!</p>