BatchNorm1d ValueError: expected 2D or 3D input (got 1D input)

username: oktopus97
cooked: <p>Hi all!</p>
<p>I’m trying to implement batch normalizing to my Neural Network, but I always get the same error. I looked through internet and couldn’t find a satisfying answer for my problem.</p>
<p>Here’s My code:<br>
import …</p>
<p>HIDDEN_LAYER = 32</p>
<p>class DeepQNetwork(nn.Module):</p>
<pre><code>def __init__(self, no_inputs, no_outputs):
    super(DeepQNetwork,self).__init__()

    self.lin1 = nn.Linear(no_inputs,HIDDEN_LAYER)
    self.lin2 = nn.Linear(HIDDEN_LAYER,HIDDEN_LAYER)
    self.lin3 = nn.Linear(HIDDEN_LAYER,no_outputs)

    self.bn = nn.BatchNorm1d(num_features=HIDDEN_LAYER)

def forward(self, x):
    print('input = {}'.format(x))
    output = Variable(x)
    output = self.lin1(self.bn(output))
    output = F.relu(output)
    output = self.lin2(self.bn(output))
    output = F.relu(output)
    output = self.lin3(output)

    return F.relu(output)
</code></pre>
<p>output of the code:</p>
<p>input = tensor([ 0.0011,  0.0148,  0.0056, -0.0481], device=‘cuda:0’)</p>
<p>Traceback (most recent call last):</p>
<p>File “/gymcartpole/venv/DeepQ.py”, line 28, in forward<br>
output = self.lin1(self.bn(output))<br>
File “/gymcartpole/venv/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 489, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “/gymcartpole/venv/lib/python3.6/site-packages/torch/nn/modules/batchnorm.py”, line 60, in forward<br>
self._check_input_dim(input)<br>
File “/gymcartpole/venv/lib/python3.6/site-packages/torch/nn/modules/batchnorm.py”, line 169, in _check_input_dim<br>
.format(input.dim()))<br>
ValueError: expected 2D or 3D input (got 1D input)</p>
<p>please help me <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: ptrblck
cooked: <p><code>nn.BatchNorm1d</code> expects an input of the shape <code>[batch_size, channels]</code> or <code>[batch_size, channels, length]</code>.<br>
Currently you are just passing a tensor with a single dimension to the layer.<br>
If your data has 4 features, you should add the batch dimension using:</p>
<pre><code class="lang-python">input = input.unsqueeze(0)
</code></pre>
<p>before passing it to your model.</p>

username: Peter_Phan
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
I have the same problem in my inference steps. I already change my model to model.eval() and .unsqueeze(0), but the error still exist.<br>
My inference code like this:</p>
<pre><code class="lang-auto">Discriminator = torch.load('disc.pth', map_location=torch.device('cpu'))
Discriminator.eval()
embededSeq = Embedding.EmbedOne('sample data')
embededSeq = torch.tensor(embededSeq).float()
embededSeq = embededSeq.unsqueeze(0)
score = PosDiscriminator(embededSeq).detach().numpy()[0]
......

</code></pre>
<p>and my class is like this:</p>
<pre><code class="lang-auto">class Discriminator(nn.Module):
    def __init__(self, sequenceLength):
        super(Discriminator,self).__init__()
        self.batchnorm1 = nn.BatchNorm1d(sequenceLength)
        self.batchnorm2 = nn.BatchNorm1d(2*sequenceLength)
        self.linear1 = nn.Linear(sequenceLength, 2*sequenceLength)
        self.conv2 = nn.Conv1d(1, 1,kernel_size=3, stride=1, padding=1)
        self.conv3 = nn.Conv1d(1, 1,kernel_size=3, stride=1, padding=1)
        self.linear4 = nn.Linear(2*sequenceLength, 1)
        self.relu = nn.ReLU(0.01)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        out = self.batchnorm1(x)
        out = self.linear1(out)
        out = self.relu(out)
        out = self.batchnorm2(out)
        out = out.unsqueeze(1)
        out = self.conv2(out)
        out = self.sigmoid(out)
        out = self.conv3(out)
        out = self.relu(out)
        out = out.squeeze()
        out = self.batchnorm2(out)
        out = self.linear4(out)
        out = self.sigmoid(out)
        return out
</code></pre>
<p>Moreover, when I added one more time of .unsqueeze(0) into my input of inference like this</p>
<pre><code class="lang-auto">embededSeq = embededSeq.unsqueeze(0).unsqueeze(0)
</code></pre>
<p>the error turned into “running_mean should contain 1 elements not 289”, where 289 is my sequenceLength in execution.<br>
May I ask what’s wrong with my code?</p>
<p>Thank you</p>

username: ptrblck
cooked: <p>Your shapes are still incorrect so don’t just <code>unsqueeze</code> dimensions randomly, but make sure the input has the shape <code>[batch_size, channels, seq_len]</code>.<br>
Here is an example of your error:</p>
<pre data-code-wrap="python"><code class="lang-python">bn = nn.BatchNorm1d(289)

# works
x = torch.randn(2, 289, 1) # [batch_size=2, channels=289, seq_len=1]
out = bn(x)

# fails with your error
x = torch.randn(2, 1, 289) # [batch_size=2, channels=1, seq_len=289]
out = bn(x)
# RuntimeError: running_mean should contain 1 elements not 289
</code></pre>
<p>As you can see the channel dimension in the batchnorm layer is set to <code>289</code>, which already does not correspond to:</p>
<aside class="quote no-group" data-username="Peter_Phan" data-post="3" data-topic="42081">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/peter_phan/48/32386_2.png" class="avatar"> Peter_Phan:</div>
<blockquote>
<p>where 289 is my sequenceLength in execution.</p>
</blockquote>
</aside>
<p>However, this setup yields the same error you’ve posted.<br>
Assuming your input has a shape of e.g. <code>[batch_size=2, channels=16, seq_len=289]</code>, this would work:</p>
<pre data-code-wrap="python"><code class="lang-python">bn = nn.BatchNorm1d(16)
x = torch.randn(2, 16, 289)
out = bn(x)
</code></pre>