TextCNN Kernel size can’t greater than actual input size

icmp_no_request: <p>Hi, when I run my <code>TextCNN</code> model (my filter size is [3, 4, 5]), it runs successfully in the 12 epochs (the total epoch number is 20), but when it starts the No.13 epoch it shows the error:</p>
<pre><code class="lang-auto">RuntimeError: Calculated padded input size per channel: (4 x 100). Kernel size: (5 x 100). Kernel size can't be greater than actual input size
</code></pre>
<p>I met the <code>RuntimeError</code> above yesterday on Colab. However, I run my code successfully today with filter sizes = [3, 4, 5] on Colab.</p>
<p>Could somebody help me explain it? Any ideas?</p>

ptrblck: <p>Are you padding the input based on the largest sequence length?<br>
If so, you might get unlucky and a batch might contain “short” examples, thus your activation might be too small at this particular layer.</p>

icmp_no_request: <p><a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>Thank you for your reply. I didn’t pad the input to the largest sequence length, just embed -&gt; convolution -&gt; max pooling -&gt; concat -&gt; output.</p>
<p>Any ideas to solve the <code>RuntimeError</code> ?</p>

ptrblck: <p>The error points to an activation, which is too small for the specified kernel size.<br>
You would either have to enlarge your input, use a smaller kernel or remove some pooling layers.</p>

icmp_no_request: <p><a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>Thank you. Solved by changing to a smaller kernel.<br>
If I want to enlarge my input, do you have some suggestions of it with <code>torchtext</code> ? I use <code>torchtext</code> to pre-process my text data.</p>
<pre><code class="lang-auto">SENTI_TOKENS = data.Field(batch_first=True,
                          tokenize=tokenize_and_cut,
                          preprocessing=tokenizer.convert_tokens_to_ids)
</code></pre>

ptrblck: <p>I think only padding would make sense for text data, so a smaller kernel seems to be more reasonable.</p>

icmp_no_request: <p>Could you please give me some code on padding with <code>torchtext</code> ?<br>
I tried with <code>fix_length</code>, a parameter of <code>data.Field()</code>, could it pad?</p>

Bahaa_Kattan: <p>your text size most be greater or equal to your maximum filter size</p>
<p>for example:</p>
<pre><code class="lang-auto">filter_sizes = [2, 3, 4]
</code></pre>
<p>maximum filter size is 4 so your text length should be 4 words or more</p>
<p>example of predicting batch of one text</p>
<pre><code class="lang-auto">def pred(text):
    
    # preprocessing text
    tokenized= text_cleaner(text).split() 
    
    # pad to maximum filter size:
    if len(tokenized) &lt; 4:
       text += ['&lt;pad&gt;'] * (5 - len(tokenized)) # i used '&lt;pad&gt;' because my vocab dict map '&lt;pad&gt;' to 1 
    
    # convert words to index
    vectors = [TEXT.vocab.stoi.get(w, 0) for w in tokenized] # stoi.get(w, 0) it map w to its index if the word in the dict else it map to '&lt;unk&gt;' which is mapped to 0
    
    # convert the list of vectors to torch.tensor
    vectors = torch.tensor(vectors, dtype=torch.long, device=device).unsqueeze(dim=0)
        
    # predict text
    with torch.no_grad():
        output = model(vectors)

    return output
</code></pre>
<p>even if your batch size is greater then one (example: 32) thier length it mustn’t be less then maximum filter size</p>
<p>I hope it’s clear now</p>