How to give input to this model?

krishna511: <p>Hi there,<br>
I am trying to implement  1 DCNN over raw audio data (mono channel).<br>
I have say 60ksamples in all of wav files. Now my model code goes like<br>
…<br>
self.conv_1 = nn.Conv1d(self.input_spec_size,self.cnn_filter_size,3,1)<br>
self.max_pooling_1 = nn.MaxPool1d(3)<br>
…</p>
<p>But it throws the error: Expected 3-dimensional input for 3-dimensional weight [64, 60000, 3], but got 2-dimensional input of size [40, 60000] instead</p>
<p>Here 64 is no of output filters, 3 is kernel size and 40 is my batch size. Plz someone guide how to reshape this (60000,1 ) input data.<br>
#########################<br>
I tried reshaping input as</p>
<p>data.reshape(1,60000)<br>
then the error :</p>
<p>Given groups=1, weight of size  [64, 60000, 3], expected input[40, 1, 60000] to have 60000 channels, but got 1 channels instead</p>

pchandrasekaran: <p>For starters, it looks like your in_channels argument is taking the value 60000. I believe that needs to be 1.</p>
<p>Conv1d expects inputs in the shape (batch_size, n_channels, Seq Length), so your data must be reshaped as (40, 1, 60000)</p>
<pre><code class="lang-auto">CN = torch.nn.Conv1d(in_channels=1, out_channels=64, kernel_size=3)
trial = torch.randn((40, 60000))
out = CN(trial.reshape(40, 1, 60000)) 

# out is of shape [40, 64, 59998] , 59998 being the expected number of samples after convolution
</code></pre>

krishna511: <p>Thanks a lot, I have corrected already but another problem rose. I hope you can help me with this, I am very new to understand pytorch lstm and encoderlayer.<br>
in my model 1 D CNN is followed by lstm layers, But at this encoder layer I am getting syntax error</p>
<p>self.encoder_layer=nn.TransformerEncoderLayer(d_model=self.hidden_size_lstm*2,dim_feedforward=512,nhead=self.num_heads_self_attn)</p>
<p>I am not getting whats wrong, referred doc of pytorch too.</p>

Abhilash_Srivastava: <p>Can you share the full code and error message here?</p>

krishna511: <p>import torch<br>
import torch.nn as nn<br>
from torch.autograd import Variable<br>
from torch.nn import functional as F</p>
<p>class CNN_1D(torch.nn.Module):<br>
def <strong>init</strong>(self, num_layers_lstm, num_heads_self_attn,<br>
hidden_size_lstm, num_emo_classes):<br>
super(CNN_1D, self).<strong>init</strong>()</p>
<pre><code>    self.num_layers_lstm=num_layers_lstm
    self.num_heads_self_attn=num_heads_self_attn
    self.hidden_size_lstm=hidden_size_lstm
    self.num_emo_classes=num_emo_classes
    #self.num_gender_class=num_gender_class
    
    
    self.layer1 = nn.Sequential(
        nn.Conv1d(self.input_spec_size,64,3,1),
        nn.BatchNorm2d(64),
        nn.ReLU(),
        nn.MaxPool2d(kernel_size=3, stride=2))
    self.layer2 = nn.Sequential(
        nn.Conv1d(64,100,5,1),
        nn.BatchNorm2d(100),
        nn.ReLU(),
        nn.MaxPool2d(kernel_size=3, stride=2))
    self.layer3 = nn.Sequential(
        nn.Conv1d(100,100,7,1),
        nn.BatchNorm2d(100),
        nn.ReLU(),


    ###
    self.lstm = nn.LSTM(input_size=100, hidden_size=self.hidden_size_lstm,num_layers=self.num_layers_lstm,bidirectional=True,dropout=0.5,batch_first=True)
    ## Transformer
    self.encoder_layer = nn.TransformerEncoderLayer(d_model=self.hidden_size_lstm*2,dim_feedforward=512,nhead=self.num_heads_self_attn)
    #self.gender_layer  = nn.Linear(self.hidden_size_lstm*4,self.num_gender_class)
    self.emotion_layer = nn.Linear(6664,self.num_emo_classes)
</code></pre>
<p>self.encoder_layer = nn.TransformerEncoderLayer(d_model=self.hidden_size_lstm*2,dim_feedforward=512,nhead=self.num_heads_self_attn)<br>
^<br>
SyntaxError: invalid syntax</p>

Abhilash_Srivastava: <p>In your code, some of the parameters are missing. I modified your code slightly and now it should be possible to initialize the class <code>CNN_1D</code>. Make changes wherever required.</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from torch.autograd import Variable
from torch.nn import functional as F

class CNN_1D(torch.nn.Module):
    def __init__(self, num_layers_lstm, num_heads_self_attn, hidden_size_lstm, num_emo_classes, input_spec_size):
        super(CNN_1D, self).__init__()
        self.num_layers_lstm = num_layers_lstm
        self.num_heads_self_attn = num_heads_self_attn
        self.hidden_size_lstm = hidden_size_lstm
        self.num_emo_classes = num_emo_classes
        self.input_spec_size = input_spec_size
        # self.num_gender_class=num_gender_class

        self.layer1 = nn.Sequential(
            nn.Conv1d(self.input_spec_size, 64, 3, 1),
            nn.BatchNorm2d(64),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=3, stride=2))

        self.layer2 = nn.Sequential(
            nn.Conv1d(64, 100, 5, 1),
            nn.BatchNorm2d(100),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=3, stride=2))

        self.layer3 = nn.Sequential(
            nn.Conv1d(100, 100, 7, 1),
            nn.BatchNorm2d(100),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=3, stride=2))
        ###
        self.lstm = nn.LSTM(input_size=100, hidden_size=self.hidden_size_lstm, num_layers=self.num_layers_lstm,
                            bidirectional=True, dropout=0.5, batch_first=True)
        ## Transformer
        self.encoder_layer = nn.TransformerEncoderLayer(d_model=self.hidden_size_lstm * 2, dim_feedforward=512,
                                                        nhead=self.num_heads_self_attn)
        # self.gender_layer  = nn.Linear(self.hidden_size_lstm*4,self.num_gender_class)
        self.emotion_layer = nn.Linear(6664, self.num_emo_classes)
</code></pre>

krishna511: <p>Still I am getting this error,</p>
<p>Given groups=1, weight of size [64, 7, 3], expected input[1, 1, 60000] to have 7 channels, but got 1 channels instead</p>
<p>this 7 channel, I am not getting. Then I am trying to calculate output at each layer</p>
<p>import torch<br>
import torch.nn as nn<br>
I=torch.randn(40,1,60000) #### 40 is batch size,60000 is sample length</p>
<p>layer1 = nn.Sequential(<br>
nn.Conv1d(1,64,3,1),<br>
nn.BatchNorm2d(64),<br>
nn.ReLU(),<br>
nn.MaxPool2d(kernel_size=3, stride=2))</p>
<p>O1=layer1(I)</p>
<p>It says: expected 4D input (got 3D input)</p>

pchandrasekaran: <p>You’ll need to use <code>BatchNorm1d(64)</code>.</p>
<p>Edit: Also seen that you are using MaxPool2d. That needs to be changed to 1D as well. Using 2D will also affect the number of filters.</p>

krishna511: <p>Thank you so much I am fool.</p>

krishna511: <p>are you able to figure out the first error here. in my previous post?</p>
<p>Actually let me try to clarify : I have calculated the out tensor size after three con1d layers as<br>
[40,100,7494]<br>
where 40 is batch size, 100 is the out features of conv layer and 7494 is the output of the sequence…right?</p>
<p>Now I want to move further for LSTM layer here I am doing this:</p>
<p>rnn = nn.LSTM(7494, 60, 2,bidirectional=True, dropout=0.5, batch_first=True)<br>
output= rnn(O3)</p>
<p>I have replaced the input size as 7494 (not 100), As it was giving error.<br>
Here the output is a tuple. Now cud you plz tell me how to check the output size further for TransformerEncoderLayer and final linear layer.</p>
<p>I think this shape error is arising here only.<br>
EROOR:<br>
<strong>Given groups=1, weight of size [64, 7, 3], expected input[1, 1, 60000] to have 7 channels, but got 1 channels instead</strong></p>

krishna511: <p>Here O3 is the output of third conv1d layer* Also my confusion is ,<br>
there is no role of previous output features as 100.<br>
<a class="mention" href="/u/pchandrasekaran">@pchandrasekaran</a> <a class="mention" href="/u/abhilash_srivastava">@Abhilash_Srivastava</a>  Sir plz help me</p>

pchandrasekaran: <pre><code class="lang-auto">I = torch.randn(40,1,60000)

layer1 = nn.Sequential(
            nn.Conv1d(1, 64, 3, 1),
            nn.BatchNorm1d(64),
            nn.ReLU(),
            nn.MaxPool1d(kernel_size=3, stride=2))

layer2 = nn.Sequential(
            nn.Conv1d(64, 100, 5, 1),
            nn.BatchNorm1d(100),
            nn.ReLU(),
            nn.MaxPool1d(kernel_size=3, stride=2))

layer3 = nn.Sequential(
            nn.Conv1d(100, 100, 7, 1),
            nn.BatchNorm1d(100),
            nn.ReLU(),
            nn.MaxPool1d(kernel_size=3, stride=2))
</code></pre>
<pre><code class="lang-auto">op = layer1(I)
print(op.shape)

op = layer2(op)
print(op.shape)

op = layer3(op)
print(op.shape)
</code></pre>
<p>op is now [40, 100, 7494]</p>
<pre><code class="lang-auto">rnn = nn.LSTM(7494, 60, 2, bidirectional=True, dropout=0.5, batch_first=True)
op, not_needed = rnn(op)
</code></pre>
<p>In the above <code>op</code> is your output. Ignore the variable <code>not_needed</code> for now. <code>op</code> will be of shape <em>(batch, seq, num_directions * hidden_size)</em>, which upon checking, it is.</p>
<p>op shape now = [40, 100, 120]</p>
<p>Here’s where I’m not too sure. I have limited experience using LSTMs and looking at my codes, I’ve only used the final sequence. Depending on what you are going to do, you can leave <code>op</code> as is or if you are taking the final sequence only, reshape it as, <code>op = op[:, -1, :].reshape(40, 1, 120)</code></p>
<pre><code class="lang-auto">encoder_layer = nn.TransformerEncoderLayer(d_model=120, dim_feedforward=512, nhead=your_nhead)
op = encoder_layer(op)
</code></pre>
<p><code>op</code> now will have 2 shapes, one if you used all the sequences another if you used only the last.</p>
<p>[40, 100, 120]<br>
[40, 1, 120]</p>
<p>From here, your Linear Layers are [100x120] or [1x120].</p>
<p>The 100 that you mentioned is used as the sequence length for your LSTM.</p>

krishna511: <p><a class="mention" href="/u/pchandrasekaran">@pchandrasekaran</a> Thanks<br>
This 100 is not mentioned there in LSTM layer, so is it automatically taken ?<br>
and how to choose between this option of 1 and 100.Yes in this way I am getting this shape [40,100,120]</p>
<p>and in last linear layer<br>
############<br>
self.emotion_layer = nn.Linear(120, self.num_emo_classes)<br>
############</p>
<p>I have read a blog where its wriiten that ’ <strong>The input</strong>  of our fully connected <code>nn.Linear()</code>  layer requires an input size corresponding to the number of hidden nodes in the preceding LSTM layer. Therefore we must reshape our data into the form’</p>
<p>So I wrote 120 here.<br>
and finally defined the forward function as</p>
<p>def forward(self,inputs):<br>
out = self.layer1(inputs)<br>
out = self.layer2(out)<br>
out = self.layer3(out)<br>
out = self.lstm(out)<br>
out = self.encoder_layer(out)</p>
<pre><code>    pred = self.emotion_layer(out)
    return pred
</code></pre>
<p><strong>Error: Given groups=1, weight of size [64, 7, 3], expected input[1, 1, 60000] to have 7 channels, but got 1 channels instead</strong></p>
<p>I hope I am clear now sir?</p>

pchandrasekaran: <blockquote>
<p>I have read a blog where its wriiten that ’ <strong>The input</strong>  of our fully connected <code>nn.Linear()</code>  layer requires an input size corresponding to the number of hidden nodes in the preceding LSTM layer.</p>
</blockquote>
<p>That explains why my codes only have the last sequence. Ty for that. So use only the last sequence of the LSTM. For the error, could you copy your entire code snippet so I can try and reproduce the error. That looks like an error in the convolution layers.</p>

krishna511: <p>This code is in 4 parts , I dnt know how to post here?<br>
Can I mail You the entire code?</p>