RuntimeError: mat1 and mat2 shapes cannot be multiplied (8x8 and 17x8)

username: Jayson_Sim
cooked: <p>import numpy as np</p>
<p>import json</p>
<p>import torch</p>
<p>import torch.nn as nn</p>
<p>from torch.utils.data import Dataset,DataLoader</p>
<p>from NeuralNetwork import bag_of_words , tokenize , stem</p>
<p>from Brain import NeuralNet</p>
<p>with open(‘D:\Mark 1\MARK 2\intents.json’,‘r’) as f:</p>
<pre><code>intents = json.load(f)
</code></pre>
<p>all_words = []</p>
<p>tags = []</p>
<p>xy = []</p>
<p>for intent in intents[‘intents’]:</p>
<pre><code>tag = intent['tag']

tags.append(tag)



for pattern in intent['patterns']:

    print(pattern)

    w = tokenize(pattern)

    all_words.extend(w)

    xy.append((w,tag))
</code></pre>
<p>ignore_words = [’,’,’?’,’/’,’.’,’!’]</p>
<p>all_words = [stem(w) for w in all_words if w not in ignore_words]</p>
<p>all_words = sorted(set(all_words))</p>
<p>tags = sorted(set(tags))</p>
<p>x_train = []</p>
<p>y_train = []</p>
<p>for (pattern_sentence,tag) in xy:</p>
<pre><code>bag = bag_of_words(pattern_sentence,all_words)

x_train.append(bag)



label = tags.index(tag)

y_train.append(label)
</code></pre>
<p>x_train = np.array(x_train)</p>
<p>y_train = np.array(y_train)</p>
<p>num_epochs = 1000</p>
<p>batch_size = 8</p>
<p>learning_rate = 0.001</p>
<p>input_size = len(x_train[0])</p>
<p>hidden_size = 8</p>
<p>output_size = len(tags)</p>
<p>print(“Training The Model…”)</p>
<p>class ChatDataset(Dataset):</p>
<pre><code>def __init__(self):

    self.n_samples = len(x_train)

    self.x_data = x_train

    self.y_data = y_train

   

def __getitem__(self,index):

    return self.x_data[index],self.y_data[index]



def __len__(self):

    return self.n_samples
</code></pre>
<p>dataset = ChatDataset()</p>
<p>train_loader = DataLoader(dataset=dataset,batch_size=batch_size,shuffle=True,num_workers=0)</p>
<p>device = torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’)</p>
<p>model = NeuralNet(input_size,hidden_size,output_size).to(device=device)</p>
<p>criterion = nn.CrossEntropyLoss()</p>
<p>optimizer = torch.optim.Adam(model.parameters(),lr=learning_rate)</p>
<p>for epoch in range(num_epochs):</p>
<pre><code>for (words,labels) in train_loader:

    words = words.to(device)

    labels = labels.to(dtype=torch.long).to(device)

   

    outputs = model(words)

    loss = criterion(outputs,labels)

   

    optimizer.zero_grad()

    loss.backward()

    optimizer.step()

   

if (epoch+1) % 100 ==0:

    print(f'epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')
</code></pre>
<p>print(f’Final Loss : {loss.item():.4f}’)</p>
<p>data = {</p>
<pre><code>"model_state":model.state_dict(),

"input_size":input_size,

"hidden_size":hidden_size,

"output_size":output_size,

"all_words":all_words,

"tags":tags
</code></pre>
<p>}</p>
<p>FILE = “TrainData.pth”</p>
<p>torch.save(data,FILE)</p>
<p>print(f"Training Complete, File Saved To {FILE}")</p>

username: ptrblck
cooked: <p>I don’t see your model definition, but guess you might be using linear layers which will create a shape mismatch if the number of features of the input activation doesn’t match their expected <code>in_features</code> value. Here is a small example raising the same issue:</p>
<pre><code class="lang-python">lin = nn.Linear(in_features=17, out_features=8)
x = torch.randn(8, 8)
out = lin(x)
# &gt; RuntimeError: mat1 and mat2 shapes cannot be multiplied (8x8 and 17x8)
</code></pre>
<p>As you see, <code>x</code> is using a batch size of <code>8</code> and also <code>8</code> features while <code>lin</code> expects <code>17</code> input features.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:"></p>