Using Histopathological Image Dataset with Ground Truth

okuryapar: <p>Hi,</p>
<p>I have a dataset which consists of 58 images(RGB format, row<em>col</em>3) and with their correspondant ground truth images (binary, row*col). This means every pixel in image has three color features and one class feature which is 1 or 0 to show belongings to a tumor or not.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/d/4d1a1e3bf3125da3e9014c55af3ed2d6becfd939.png" data-download-href="https://discuss.pytorch.org/uploads/default/4d1a1e3bf3125da3e9014c55af3ed2d6becfd939" title="Screen Shot 2020-02-07 at 11.57.23"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/d/4d1a1e3bf3125da3e9014c55af3ed2d6becfd939_2_690x425.png" alt="Screen Shot 2020-02-07 at 11.57.23" data-base62-sha1="b04IPdhBVP96WSBjdigdrN3YciZ" width="690" height="425" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/d/4d1a1e3bf3125da3e9014c55af3ed2d6becfd939_2_690x425.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/d/4d1a1e3bf3125da3e9014c55af3ed2d6becfd939_2_1035x637.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/4/d/4d1a1e3bf3125da3e9014c55af3ed2d6becfd939.png 2x" data-dominant-color="7B6A83"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-02-07 at 11.57.23</span><span class="informations">1110×684 165 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I’ve looked at lots of tutorials but most of them are unfortunately doesn’t fit my problem. Like MNIST dataset, they have row*col features and just one class info instead of a pixel based ground truth images.</p>
<p>How can I load these images to perform neural network or any kind of deep learning algorithms? Any help would be appreciated.</p>

ptrblck: <p>You can treat this use case as a binary classification use case and would just have to make sure the output of your model and the corresponding target have the right shape.</p>
<p>E.g. instead of using conv and pooling layers at the beginning of your model, then flattening the activations, and passing them to linear layers, you could write a model using only conv layers, so that the spatial size of the activations stays constant.</p>
<p>Here is a simple example:</p>
<pre><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 3, 1, 1)
        self.conv2 = nn.Conv2d(6, 1, 3, 1, 1)
        
    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = self.conv2(x)
        return x

model = MyModel()

x = torch.randn(10, 3, 24, 24)
y = torch.randint(0, 2, (10, 1, 24, 24)).float()
dataset = TensorDataset(x, y)
loader = DataLoader(
    dataset,
    batch_size=2,
    num_workers=2,
    shuffle=True
)

criterion = nn.BCEWithLogitsLoss()
optimizer = optim.Adam(model.parameters(), lr=1e-3)

nb_epochs = 10
for epoch in range(nb_epochs):
    for data, target in loader:
        optimizer.zero_grad()
        output = model(data)
        loss = criterion(output, target)
        loss.backward()
        optimizer.step()
    print('Epoch {}, loss {}'.format(epoch, loss.item()))
</code></pre>
<p>Also, have a look at <a href="https://discuss.pytorch.org/t/torchvision-transfors-how-to-perform-identical-transform-on-both-image-and-target/10606/7">this post</a> for some information about how to apply the same random transformations on your input image and mask.</p>

tom: <p>To expand on Piotr’s suggestion. The most common type of network used for this is U-Net, but you’ll probably have some difficulty to train that with the small size of your dataset unless you find some auxiliary data (need not necessarily be labeled, but from a similar domain).</p>
<p>Best regards</p>
<p>Thomas</p>