Inputting [250, 33] getting [8250,3] output

username: ashleycodes
cooked: <p>Hi, I’m inputting a tensor of shape 250, 33 into my LSTM which is being used for a NLP multi-class classification. The inputs are sentences all of size 33 words, in batches of 250. I’m trying to get an output of 250, 3 as this would be a probability of each class for each sentence, however, i’m getting a predicition for each word instead?<br>
I’ve had to make a loop to add every 33 words together and divide by 33 to get a predicition for a sentence but it’s causing training to be incredibly slow and definitely seems like a really makeshift way of fixing the issue.<br>
Apologies if this is a relatively obvious fix, i’m really new to AI building in general and this is my first ever attempt.<br>
Question is how would I get the output to be the probability of the 3 classes for each sentence in the batch.</p>
<pre><code class="lang-auto">&gt; import torch
&gt; 
&gt; import torch.nn as nn
&gt; 
&gt; import torch.nn.functional as F
&gt; 
&gt; import torch.optim as op
&gt; 
&gt; import torchvision
&gt; 
&gt; from torch.utils.data import TensorDataset, DataLoader
&gt; 
&gt; from torchvision import transforms, datasets
&gt; 
&gt; class HateSpeechDetector(nn.Module):
&gt; 
&gt;     def __init__(self, device, vocabularySize, output, embedding, hidden, layers, dropProb=0.5):
&gt; 
&gt;         super(HateSpeechDetector, self).__init__()
&gt; 
&gt;         #Number of outputs (Classes/Categories)
&gt; 
&gt;         self.output = output
&gt; 
&gt;         #Number of layers in the LSTM
&gt; 
&gt;         self.numLayers = layers
&gt; 
&gt;         #Number of hidden neurons in each LSTM layer
&gt; 
&gt;         self.hiddenDimensions = hidden
&gt; 
&gt;         #Device being used for by model (CPU or GPU)
&gt; 
&gt;         self.device = device
&gt; 
&gt;         
&gt; 
&gt;         #Embedding layer finds correlations in words by converting word integers into vectors
&gt; 
&gt;         self.embedding = nn.Embedding(vocabularySize, embedding)
&gt; 
&gt;         #LSTM stores important data in memory, using it to help with future predictions
&gt; 
&gt;         self.lstm = nn.LSTM(embedding,hidden,layers,dropout=dropProb,batch_first=True)
&gt; 
&gt;         #Dropout is used to randomly drop nodes. This helps to prevent overfitting of the model during training
&gt; 
&gt;         self.dropout = nn.Dropout(dropProb)
&gt; 
&gt;         #Establishing 4 simple layers and a sigmoid output
&gt; 
&gt;         self.fc = nn.Linear(hidden, hidden)
&gt; 
&gt;         self.fc2 = nn.Linear(hidden, hidden)
&gt; 
&gt;         self.fc3 = nn.Linear(hidden, hidden)
&gt; 
&gt;         self.fc4 = nn.Linear(hidden, hidden)
&gt; 
&gt;         self.fc5 = nn.Linear(hidden, hidden)
&gt; 
&gt;         self.fc6 = nn.Linear(hidden, output)
&gt; 
&gt;         self.softmax = nn.Softmax(dim=1)
&gt; 
&gt;     def forward(self, x, hidden):
&gt; 
&gt;         batchSize = x.size(0)
&gt; 
&gt;         x = x.long()
&gt; 
&gt;         embeds = self.embedding(x)
&gt; 
&gt;         lstm_out, hidden = self.lstm(embeds, hidden)
&gt; 
&gt;         lstm_out = lstm_out.contiguous().view(-1,self.hiddenDimensions)
&gt; 
&gt;         out = self.dropout(lstm_out)
&gt; 
&gt;         out = self.fc6(out)
&gt; 
&gt;         out = self.softmax(out)
&gt; 
&gt;         myTensor = torch.Tensor([0,0,0])
&gt; 
&gt;         newOut = torch.zeros(batchSize, self.output)
&gt; 
&gt;         count = 0
&gt; 
&gt;         row = 0
&gt; 
&gt;         for tensor in out:
&gt; 
&gt;             if(count == 33):
&gt; 
&gt;                 newOut[row] = myTensor/33
&gt; 
&gt;                 myTensor = torch.Tensor([0,0,0])
&gt; 
&gt;                 row += 1
&gt; 
&gt;                 count = 0
&gt; 
&gt;             myTensor += tensor
&gt; 
&gt;             count += 1
&gt; 
&gt;         return newOut, hidden
&gt; 
&gt;     def init_hidden(self, batchSize, device):
&gt; 
&gt;         weight = next(self.parameters()).data
&gt; 
&gt;         hidden = (weight.new(self.numLayers, batchSize, self.hiddenDimensions).zero_().to(device), weight.new(self.numLayers, batchSize, self.hiddenDimensions).zero_().to(device))
&gt; 
&gt;         return hidden
</code></pre>
<p>Thanks</p>
<ul>
<li>Ash</li>
</ul>