Facing ValueError: Expected input batch_size (1) to match target batch_size (4)

Ali_Mashoud: <p>I am trying to develop a skip-gram model of word2vec with the help of pytorch however, while training I am facing the above error. Please guide me what I am doing wrong. Here is my model:</p>
<blockquote>
<p>class Skipgram(torch.nn.Module):<br>
def <strong>init</strong>(self,vocab_size,embedding_dimensions,window_size):<br>
super().<strong>init</strong>()<br>
self.embeddings = nn.Embedding(vocab_size, embedding_dimensions)<br>
self.linear1 = nn.Linear(embedding_dimensions, 128)<br>
self.activation_1 = nn.ReLU()<br>
self.linear2 = nn.Linear(128, window_size*vocab_size)<br>
self.activation_2 = nn.LogSoftmax(dim=-1)<br>
def forward(self, inputs):<br>
embeds = self.embeddings(inputs)<br>
embeds_1 = sum(embeds).view(1,-1)<br>
out = self.linear1(embeds_1)<br>
out = self.activation_1(out)<br>
out = self.linear2(out)<br>
out = self.activation_2(out)<br>
return out<br>
def get_context_embedddings(self,target):<br>
target = [word_to_num[w] for w in target]<br>
return self.embeddings(target).view(1,-1)</p>
</blockquote>
<blockquote>
<p>model = Skipgram(vocab_size,embedding_dim,window_size)</p>
</blockquote>
<p><strong>Here is the training code:</strong></p>
<blockquote>
<p>for epochs in range(50):<br>
total_loss = 0</p>
<pre><code>for center_word, target in data_1:
    center_vector = torch.tensor([word_to_num[center_word]], dtype=torch.long)
    y_val  = model(center_vector)
    idxs = torch.tensor([word_to_num[w] for w in target],dtype=torch.long)
    total_loss+= criterion(y_val,idxs)


#Updating gradients and parameters
optimizer.zero_grad()
total_loss.backward()
optimizer.step()
</code></pre>
</blockquote>
<p>Here is the error that I am facing:</p>
<blockquote>
<hr>
<p>ValueError                                Traceback (most recent call last)<br>
 in <br>
6         y_val  = model(center_vector)<br>
7         idxs = torch.tensor([word_to_num[w] for w in target],dtype=torch.long)<br>
----&gt; 8         total_loss+= criterion(y_val,idxs)<br>
9<br>
10</p>
<p>~\Anaconda3\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)<br>
887             result = self._slow_forward(*input, **kwargs)<br>
888         else:<br>
 → 889             result = self.forward(*input, **kwargs)<br>
890         for hook in itertools.chain(<br>
891                 _global_forward_hooks.values(),</p>
<p>~\Anaconda3\lib\site-packages\torch\nn\modules\loss.py in forward(self, input, target)<br>
1046         assert self.weight is None or isinstance(self.weight, Tensor)<br>
1047         return F.cross_entropy(input, target, weight=self.weight,<br>
 → 1048                                ignore_index=self.ignore_index, reduction=self.reduction)<br>
1049<br>
1050</p>
<p>~\Anaconda3\lib\site-packages\torch\nn\functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)<br>
2691     if size_average is not None or reduce is not None:<br>
2692         reduction = _Reduction.legacy_get_string(size_average, reduce)<br>
 → 2693     return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)<br>
2694<br>
2695</p>
<p>~\Anaconda3\lib\site-packages\torch\nn\functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)<br>
2383     if input.size(0) != target.size(0):<br>
2384         raise ValueError(<br>
 → 2385             “Expected input batch_size ({}) to match target batch_size ({}).”.format(input.size(0), target.size(0))<br>
2386         )<br>
2387     if dim == 2:</p>
<p>ValueError: Expected input batch_size (1) to match target batch_size (4).<br>
Please tell me what I am doing wrong here. I would be grateful for your help.</p>
</blockquote>

ptrblck: <p>I guess your <code>view</code> operations might change the batch size, which could yield this error, so you should check:</p>
<pre><code class="lang-python">embeds_1 = sum(embeds).view(1,-1)
</code></pre>
<p>and</p>
<pre><code class="lang-python">return self.embeddings(target).view(1,-1)
</code></pre>
<p>which both flatten the tensor so that it would have a batch size of <code>1</code>.<br>
Usually you would keep the batch size equal and flatten the other dimensions via:</p>
<pre><code class="lang-python">tensor = tensor.view(tensor.size(0), -1)
</code></pre>

Ali_Mashoud: <p>Does the dimension of logsoftmax function also effects the dimensions. Like the final coding statement of this:<br>
<code>self.activation_2 = nn.LogSoftmax(dim=-1)</code><br>
Does this also effects the shape of the tensor???</p>

ptrblck: <p>No, <code>nn.LogSoftmax</code> won’t change the shape of the tensor and you can verify it by printing the shape of the input and compare it to the shape of the output.</p>

Ali_Mashoud: <p>Now I am facing the following error:<br>
<code>RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x1 and 500x128)</code><br>
tensor.size(0),-1 is also not doing any good</p>

ptrblck: <p>It now seems that some layers have a wrong shape so you would have to check the number of feature of the properly flattened tensor and make sure that the following layer uses the same number of features as its input.</p>