Issue with tensor shapes for NLP

username: aklagoo
cooked: <p>I’m working with a segmentation-classification problem statement, where I need to split some text into segments based on its emotions. I have two outputs here: a binary output denoting whether the current sentence is a segment boundary or not, and a multi-label sigmoid output that produces the list of emotions/tags that apply for the latest segment.</p>
<p>I’m trying to partially replicate the model given in the following URL:<br>
<aside class="onebox pdf">
  <header class="source">
      <a href="https://web2py.iiit.ac.in/research_centres/publications/download/inproceedings.pdf.9272879615b02f00.417474656e74696f6e2d6261736564204e657572616c2054657874205365676d656e746174696f6e2028454349522d32303138292e706466.pdf" target="_blank" rel="nofollow noopener">web2py.iiit.ac.in</a>
  </header>
  <article class="onebox-body">
    <a href="https://web2py.iiit.ac.in/research_centres/publications/download/inproceedings.pdf.9272879615b02f00.417474656e74696f6e2d6261736564204e657572616c2054657874205365676d656e746174696f6e2028454349522d32303138292e706466.pdf" target="_blank" rel="nofollow noopener"><span class="pdf-onebox-logo"></span></a>
<h3><a href="https://web2py.iiit.ac.in/research_centres/publications/download/inproceedings.pdf.9272879615b02f00.417474656e74696f6e2d6261736564204e657572616c2054657874205365676d656e746174696f6e2028454349522d32303138292e706466.pdf" target="_blank" rel="nofollow noopener">inproceedings.pdf.9272879615b02f00.417474656e74696f6e2d6261736564204e657572616c2054657874205365676d656e746174696f6e2028454349522d32303138292e706466.pdf</a></h3>

<p class="filesize">625.05 KB</p>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>The input is fed as a sliding window of size 7, with left and right contexts of a width of 3 sentences each. The central sentence is to be identified as a segment boundary.<br>
The input sizes are:</p>
<pre><code class="lang-auto">torch.Size([3, 50, 300]) torch.Size([1, 50, 300]) torch.Size([3, 50, 300])
</code></pre>
<p>Since I’m feeding the elements individually, the batch size is 1. However, due to the input specifications of the CNN and LSTM layers, I am unable to correctly identify how I should feed the input.</p>
<p><strong>Problem 1:</strong><br>
In the CNN layer, the input must be a 4D array of size [batch_size, num_channels, height, width]. I therefore reshaped the arrays to sizes <em>[3, 1, 50, 300], [1, 1, 50, 300] and [3, 1, 50, 300]</em>.</p>
<p><strong>Problem 2:</strong><br>
The LSTM layer requires the shape [sequence_len, batch_size, input_size]. In a step before this, I had concatenated the previous output to give a shape of <em>[7,72]</em>. I unsqueezed this output to get a shape [7,1,72] and fed it that.</p>
<p>Here’s the model for reference:</p>
<pre><code class="lang-auto">class ContextEncoder(nn.Module):
    def __init__(self):
        super(ContextEncoder, self).__init__()
        
        # CNN filters for context
        self.cnn_k2 = nn.Conv2d(1,2,kernel_size=(2,300))
        self.cnn_k3 = nn.Conv2d(1,2,kernel_size=(3,300))
        self.cnn_k4 = nn.Conv2d(1,2,kernel_size=(4,300))
        
    def forward(self, x):
        x = torch.stack([torch.squeeze(self.cnn_k2(x), 3), torch.squeeze(self.cnn_k2(x), 3), torch.squeeze(self.cnn_k2(x), 3)])
        x = F.max_pool2d(x, 2)
        return x.view(x.shape[1], -1)

class CNN_LSTM(nn.Module):
    def __init__(self, context_size):
        super(CNN_LSTM, self).__init__()
        
        # Context encoder
        self.context_encoder_lr = ContextEncoder()
        self.context_encoder_c  = ContextEncoder()
        
        # Hidden state generation
        self.lstm = nn.LSTM(72, 36)
        
        # Attention generation
        self.attention = nn.Linear(36, 36)
        
        # Output layers
        self.emotions = nn.Linear((2 * context_size + 1) * 36, 14)
        self.segment = nn.Linear((2 * context_size + 1) * 36, 2)
        
        # Initialize hidden state
        self.reset_hidden()
    
    def reset_hidden(self):
        """Reset hidden state with (h_n, c_n)"""
        self.hidden = (torch.zeros(1,1,36), torch.zeros(1,1,36))
        
    def forward(self, x_left, x_center, x_right):
        # Encode context
        c_l = self.context_encoder_lr(x_left)
        c_c = self.context_encoder_c(x_center)
        c_r = self.context_encoder_lr(x_right)
        
        # Stack contexts and unsqueeze batch size
        c = torch.cat([c_l, c_c, c_r])
        c = c.unsqueeze(1)
        
        # Pass context through LSTM
        x, self.hidden = self.lstm(c, self.hidden)
        
        # Generate attention
        attn = self.attention(x)
        attn = F.softmax(attn, dim=1)
        
        # Apply attention
        x = x * attn
        
        # Flatten layers
        x = x.view(-1)
        
        # Feed to output layers
        out_emotions = self.emotions(x)
        out_segment = self.segment(x)
        
        return F.sigmoid(out_emotions), F.softmax(out_segment)
</code></pre>
<p>Are there any issues with the model? If so, can you suggest the changes and the logic?<br>
Thanks in advance.</p>