Expected input batch_size (1) to match target batch_size (11)

username: natrob
cooked: <p>I know this seems to be a common problem but I wasn’t able to find a solution. I’m running a multi-label classification model and having issues with tensor sizing.</p>
<p>My full code looks like this:</p>
<pre><code class="lang-auto">from transformers import DistilBertTokenizerFast, DistilBertForSequenceClassification
import torch

# Instantiating tokenizer and model
tokenizer = DistilBertTokenizerFast.from_pretrained('distilbert-base-cased')
model = DistilBertForSequenceClassification.from_pretrained('distilbert-base-cased')

# Instantiating quantized model
quantized_model = torch.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8)

# Forming data tensors
input_ids = torch.tensor(tokenizer.encode(x_train[0], add_special_tokens=True)).unsqueeze(0)
labels = torch.tensor(Y[0]).unsqueeze(0)

# Train model
outputs = quantized_model(input_ids, labels=labels)
loss, logits = outputs[:2]
</code></pre>
<p>Which yields the error:</p>
<pre><code class="lang-auto">ValueError: Expected input batch_size (1) to match target batch_size (11)
</code></pre>
<p>Input_ids looks like:</p>
<pre><code class="lang-auto">tensor([[  101,   789,   160,  1766,  1616,  1110,   170,  1205,  7727,  1113,
           170,  2463,  1128,  1336,  1309,  1138,   112,   119, 11882, 11545,
           119,   108, 15710,   108,  3645,   108,  3994,   102]])
</code></pre>
<p>with shape:</p>
<pre><code class="lang-auto">torch.Size([1, 28])
</code></pre>
<p>and labels looks like:</p>
<pre><code class="lang-auto">tensor([[0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1]])
</code></pre>
<p>with shape:</p>
<pre><code class="lang-auto">torch.Size([1, 11])
</code></pre>
<p>The size of input_ids will vary as the strings to be encoded vary in size.</p>
<p>I also noticed that when feeding in 5 values of Y to produce 5 labels, it yields the error:</p>
<pre><code class="lang-auto">ValueError: Expected input batch_size (1) to match target batch_size (55).
</code></pre>
<p>with labels shape:</p>
<pre><code class="lang-auto">torch.Size([1, 5, 11])
</code></pre>
<p>(Note that I didn’t feed 5 input_ids, which is presumably why input size remains constant)</p>
<p>I’ve tried a few different approaches to getting these to work, but I’m currently at a loss. I’d really appreciate some guidance. Thanks!</p>

username: ptrblck
cooked: <p>Which criterion are you using and could you post the code, which yields the error as well as the output and target shapes?<br>
The error message doesn’t match the provided target shape, so we would need to check, if some processing is done additionally.</p>

username: natrob
cooked: <p>Hi there! Thanks for the reply! The first code block in the original message yields the error message directly under it, and this code:</p>
<pre><code class="lang-auto">from transformers import DistilBertTokenizerFast, DistilBertForSequenceClassification
import torch

# Instantiating tokenizer and model
tokenizer = DistilBertTokenizerFast.from_pretrained('distilbert-base-cased')
model = DistilBertForSequenceClassification.from_pretrained('distilbert-base-cased')

# Instantiating quantized model
quantized_model = torch.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8)

# Forming data tensors
input_ids = torch.tensor(tokenizer.encode(x_train[0], add_special_tokens=True)).unsqueeze(0)
labels = torch.tensor(Y[:5]).unsqueeze(0)

# Train model
outputs = quantized_model(input_ids, labels=labels)
loss, logits = outputs[:2]
</code></pre>
<p>Yields the last error message I sent.</p>
<p>Y is:</p>
<pre><code class="lang-auto">array([[0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1],
       [0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0],
       [1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0],
       [0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0],
       [1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0]], dtype=uint8)
</code></pre>
<p>and x_train is:</p>
<pre><code class="lang-auto">array(["“Worry is a down payment on a problem you may never have'. \xa0Joyce Meyer.  #motivation #leadership #worry ",
       'Whatever you decide to do make sure it makes you happy. ',"@Max_Kellerman  it also helps that the majority of NFL coaching is inept. Some of Bill O'Brien's play calling was wow, ! #GOPATS ",
       "Accept the challenges so that you can literally even feel the exhilaration of victory.' -- George S. Patton 🐶 ",
       "My roommate: it's okay that we can't spell because we have autocorrect. #terrible #firstworldprobs "],
      dtype=object)
</code></pre>
<p>input_ids and labels are of types:<br>
<code>torch.int64</code> and <code>torch.uint8</code>, respectively</p>
<p>For the first block of code in the original message, the shapes of input_ids and labels are:<br>
<code>torch.Size([1, 28])</code> and <code>torch.Size([1, 11])</code>, respectively</p>
<p>For the first block of code in the original message, the output for input_ids  is:</p>
<pre><code class="lang-auto">tensor([[  101,   789,   160,  1766,  1616,  1110,   170,  1205,  7727,  1113,
           170,  2463,  1128,  1336,  1309,  1138,   112,   119, 11882, 11545,
           119,   108, 15710,   108,  3645,   108,  3994,   102]])
</code></pre>
<p>and the output for labels is:</p>
<pre><code class="lang-auto">tensor([[0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1]], dtype=torch.uint8)
</code></pre>
<p>I’m not entirely sure what you mean by criterion that I’m using, so please let me know if there’s anything else I can send you that’d help!</p>

username: natrob
cooked: <p>It would seem as though DistilBertForSequenceClassification does not support multi-label classification, which may be why the data isn’t being recognized in the form I’m trying to feed it</p>

username: ptrblck
cooked: <p>That might be the case, as the <a href="https://huggingface.co/transformers/model_doc/distilbert.html#distilbertforsequenceclassification">docs</a> state that <code>labels</code> are expected to have the shape <code>[bathc_size,]</code>.<br>
I’m not familiar with the implementation and don’t know, if there is an easy switch to multi-label classification (couldn’t find the information).</p>