Simple Network Dimensionality Problem

Sascha: <p>Hi<br>
i’m still new to Pytorch and currently using my own dataset.<br>
For some reason, i have huge problems with getting the dimensions right for my network.<br>
The data is being laoded from a <code>.csv</code> file in the beginning.<br>
This is my code so far:</p>
<pre><code class="lang-auto">X_train, X_test, y_train, y_test = train_test_split(df, y, test_size=0.2)


train_set = TensorDataset(torch.from_numpy(X_train.values),
                          torch.from_numpy(y_train.values))
test_set = TensorDataset(torch.from_numpy(X_test.values),
                         torch.from_numpy(y_test.values))



train_loader = data_utils.DataLoader(train_set,
                                     batch_size=16,
                                     shuffle=True)
test_loader = data_utils.DataLoader(test_set,
                                    batch_size=16,
                                    shuffle=False)

input_dim = 11

num_classes = 25


class Net(torch.nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        
        self.conv1 = nn.Conv1d(input_dim, 64, kernel_size=2)
        self.conv2 = nn.Conv1d(64, 64, kernel_size=2)
        
        self.pool = nn.MaxPool2d(kernel_size=2)

        self.lin1 = torch.nn.Linear(64, 32)
        self.lin2 = torch.nn.Linear(32, int(num_classes))
        
    def forward(self, data):
        x = data
        print(x.shape)
        print(self.conv1)
        #x = self.norm(x)
        x = F.relu(self.conv1(x))
        x = F.dropout(x, training=self.training)
        x = F.relu(self.conv2(x))
        x = self.pool(x)
        x = F.relu(self.lin1(x))
        x = F.relu(self.lin2(x))
        return F.log_softmax(x, dim=1)
     
model = Net()

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)


def train():
    model.train()
    total_loss = 0
    correct = 0
    for data, label in train_loader:

        optimizer.zero_grad()
        print(data.shape, label.shape)
        loss = F.nll_loss(model(data),label.long())
        loss.backward()
        total_loss += loss.item()
        with torch.no_grad():
            pred = model(data).max(dim=1)[1]
        correct += pred.eq(label.long()).sum().item()
        optimizer.step()
        return total_loss / len(train), correct / len(train)
    
def test(loader):
    model.eval()
    correct = 0
    for data, label in test_loader:
        with torch.no_grad():
            pred = model(data).max(dim=1)[1]
        correct += pred.eq(label.long()).sum().item()
    return correct / len(loader)


epochs = 100
for epoch in range(1, epochs + 1):
       loss, train_acc = train()
       test_acc = test(test_loader)

       print('Epoch {:03d}, Train Loss: {:.4f}, Train Accuracy: {:.4f}, Test Accuracy: {:.4f}'.format(
           epoch, loss, train_acc, test_acc))
</code></pre>
<p>The data shape results for x and y are</p>
<pre><code class="lang-auto">torch.Size([16, 11]) torch.Size([16])
</code></pre>
<p>And i’m getting the following error:</p>
<pre><code class="lang-auto">File "C:\Users\Network.py", line 75, in forward
    x = F.relu(self.conv1(x))

  File "C:\Users\AppData\Local\conda\conda\envs\Analysis\lib\site-packages\torch\nn\modules\module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)

  File "C:\Users\AppData\Local\conda\conda\envs\Analysis\lib\site-packages\torch\nn\modules\conv.py", line 202, in forward
    self.padding, self.dilation, self.groups)

RuntimeError: Expected 3-dimensional input for 3-dimensional weight 64 11 2, but got 2-dimensional input of size [16, 11] instead
</code></pre>
<p>Also: here is an example, of how a single line of my data looks:</p>
<pre><code class="lang-auto">data = [1,2,3,4,5,6,7,8,9,10,11]
y = [1]
</code></pre>
<p>I’m really stumped, as to what my problem is.<br>
Any help would be really appreciated</p>

beaupreda: <p>Hello,</p>
<p><a href="https://pytorch.org/docs/stable/nn.html#torch.nn.Conv1d" rel="nofollow noopener"><code>nn.Conv1d</code></a> expects a 3D input of size <code>B x C x L</code>, where <code>B</code> is the batch size, <code>C</code> the number of channels and <code>L</code> the length of the signal. I believe that, in your case, your input should be of size <code>[16, 1, 11]</code>, so you have 1 channel and your signal is of length 11.</p>
<p>Good luck!</p>