Dataset with last dimension as channel

Stefano_Setti: <p>Hi, I have a dataset that has the channel dimension as last, obviously pytorch find it not suitable for the CNN that i’m usinng</p>
<pre><code class="lang-auto">   self.conv2d = nn.Sequential(
                nn.Conv2d(1, 64, (3,6), (1, 1)),
                nn.ReLU()
            )
</code></pre>
<p>the dataset has as dimensions: samples x 10 x 6 x 1.</p>
<p>What should i do? For now I’m transposing the data but the network is not performing well.</p>

ptrblck: <p>Permuting the output is the right approach.<br>
What do you mean by “not performing well”? Is the model bad regarding the speed or accuracy?</p>
<p>You could use the <code>channels_last</code> memory format as described <a href="https://pytorch.org/tutorials/intermediate/memory_format_tutorial.html">here</a>, which internally permutes the data to the channels last format. Note that the shape of the tensor would still indicate the standard contiguous format (channels first).</p>

Stefano_Setti: <p>Thanks for answering, so the model is bad regarding the accuracy.<br>
So by using the <code>channel_last</code> i just need the dataset without permuting it, right?</p>

ptrblck: <aside class="quote no-group" data-username="Stefano_Setti" data-post="3" data-topic="100639">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/stefano_setti/48/27968_2.png" class="avatar"> Stefano_Setti:</div>
<blockquote>
<p>So by using the <code>channel_last</code> i just need the dataset without permuting it, right?</p>
</blockquote>
</aside>
<p>Yes, as shown in the linked tutorial.</p>
<p>Note that changing the memory layout will not fix the accuracy issue, so you might want to fix this first e.g. by playing around with some hyperparameters.</p>

Stefano_Setti: <p>I will try to do that too, right away. For now, thanks.</p>

Stefano_Setti: <p>So I’m using the channel_last memory format in my custom dataloader but i got this error<br>
<code>RuntimeError: Given groups=1, weight of size 64 1 3 6, expected input[2048, 10, 6, 1] to have 1 channels, but got 10 channels instead</code><br>
so apparently is not working</p>

Stefano_Setti: <p>for reference, this is my dataloader</p>
<pre><code class="lang-auto">import torch

from torch.utils import data

class Hdf5_dl(data.Dataset):
    """HDF5 datasets"""

    def __init__(self, archive, transform = None):
        self.archive = h5.File(archive, 'r')
        self.labels = torch.tensor(self.archive['labels'])
        self.data = torch.tensor(self.archive['data']).contiguous(memory_format=torch.channels_last)
        self.transform = transform

    def __getitem__(self, index):
        sample = self.data[index]

        if self.transform is not None:
            sample = self.transform(sample)

        return sample, self.labels[index]

    def __len__(self):
        return len(self.labels)

    def close(self):
        self.archive.close()
</code></pre>

ptrblck: <p>It’s working for me:</p>
<pre><code class="lang-python">conv = nn.Conv2d(1, 64, (3, 6)).cuda().to(memory_format=torch.channels_last)
x = torch.randn(2048, 1, 10, 6).cuda().to(memory_format=torch.channels_last)
out = conv(x)
print(out.shape)
&gt; torch.Size([2048, 64, 8, 1])
</code></pre>
<p>As said before, you should not manually permute the tensor, but handle it in the “standard” contiguous layout (channels-first or NCHW).</p>

Stefano_Setti: <p>so right now what i’m doing is to mimic your code like that:</p>
<pre><code class="lang-auto">model = Network()
model = model.to(memory_format=torch.channels_last)
model = model.double()
criterion = nn.BCELoss()
optimizer = optim.Adam(params = model.parameters(), lr = 0.01)
</code></pre>
<p>…</p>
<pre><code class="lang-auto">for epoch in range(epochs):  # loop over the dataset multiple times

    model.train()
    for j, data in enumerate(trainloader, 0):
        # Get the inputs; data is a list of [inputs, labels]
        inputs, labels = data
        print(inputs.stride())
        inputs = inputs.to(memory_format=torch.channels_last)
        print(inputs.stride())

        # Zero the parameter gradients
        optimizer.zero_grad()

        # Forward + Backward + Optimize
        outputs = model(inputs.double())
        loss = criterion(outputs, labels.double())
        loss.backward()
        optimizer.step()

        print("epoch\t{}\t\tbatch\t{}\nloss\t{}\n---".format(epoch, j, loss.item()))
</code></pre>
<p>and is still not working, i even tried to print out the stride before and after the <code>channel_last</code> operation and this is what i get:</p>
<pre><code class="lang-auto">(60, 6, 1, 1)
(60, 6, 1, 1)
</code></pre>
<p>Error:</p>
<pre><code class="lang-auto">RuntimeError: Given groups=1, weight of size 64 1 3 6, expected input[2048, 10, 6, 1] to have 1 channels, but got 10 channels instead
</code></pre>

ptrblck: <p>Could you print the shape of <code>inputs</code>? It should be <code>[batch_size, channels, height, width]</code>.</p>

Stefano_Setti: <p>So the shape of the input is <code>[2048, 10, 6, 1]</code> even if I convert the memory format like this:</p>
<pre><code class="lang-auto"> for j, data in enumerate(trainloader, 0):
        inputs, labels = data
        inputs = inputs.to(memory_format=torch.channels_last)
        print("\ninput: ", inputs.shape)
</code></pre>

ptrblck: <p>As described before: your input has to be in the shape <code>[batch_size, channels, height, width]</code> before using <code>to(memory_format=torch.channels_last)</code> (have another look at my code snippet).<br>
You should not manually <code>permute</code> the tensor to the channels-last format, the <code>to()</code> operation will internally handle it for you.</p>
<p>Since the conv layer has <code>in_channels=1</code>, your data should have the shape <code>[2048, 1, 10, 6]</code>.</p>

Stefano_Setti: <p>I think i didn’t explained my feld properly, my dataset from the beginning is [batch_size, height, width, channels]</p>

Stefano_Setti: <aside class="quote no-group" data-username="ptrblck" data-post="12" data-topic="100639">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>]</code>.</p>
</blockquote>
</aside>
<p>so the original shape of data is [2048, 10, 6, 1]</p>

ptrblck: <p>In that case you have to permute it, so that the shape is channels-first and apply the <code>memory_format</code> later.</p>

VitalyFedyunin: <p>Assuming original inputs are contiguous, tensors will become channels_last automatically after permute call.</p>
<pre><code class="lang-auto">inputs, labels = data
inputs = inputs.permute(0,3,1,2)
print("\ninput: ", inputs.shape, inputs.is_contiguous(memory_format=torch.channels_last))
</code></pre>