How can I implement a dynamic parameter in Linear Layer by using pytorch?

username: quoniammm
cooked: <pre><code class="lang-auto">class LinearText(nn.Module): 
    def __init__(self):
        super(CNNText, self).__init__()
        self.encoder_tit = nn.Embedding(3281, 64)
        self.encoder_con = nn.Embedding(496037, 512)
        
        self.title_fc = nn.Linear(64, 32) # 1 * len(tit) * 32
        self.con_fc = nn.Linear(512, 256) # 1 * len(con) * 256
# t.cat((content_out_3,content_out_4,content_out_5),dim=1)
# input_size = len(tit) + len(con)
        self.fc = nn.Linear(input_size, 9)

    def forward(self, title, content):
        
        logits = self.fc()
        return F.log_softmax(logits)
</code></pre>
<p>As the above, if the input_size of Linear Layer is changed with the input.<code>(input_size = len(tit) + len(con))</code>.<br>
Therefore, input_size is dynamic.Each input has a different input_size.<br>
How can I implement it in pytorch?<br>
Can anyone can help me?I am very confused.Thanks.</p>

username: smth
cooked: <p>look at using the functional interface and registering your learnable parameters as <code>nn.Parameter</code></p>
<p>You can do:</p>
<pre><code class="lang-python">F.linear(self.weight) 
</code></pre>
<p>See such an example in this repo for: <a href="https://github.com/szagoruyko/diracnets/blob/master/diracconv.py#L38-L39">https://github.com/szagoruyko/diracnets/blob/master/diracconv.py#L38-L39</a></p>

username: quoniammm
cooked: <p>there is a problem that is still solved.</p>
<p><code>input_size</code> is dynamic, therefore, the ‘weight’ will be dynamic. Because the size of ‘weight’ is the length of <code>input_size</code> * <code>feature_numbers</code>. Is this feasible in pytorch?</p>
<p><code>self.weight = nn.Parameter()</code> The size of weight is solid.How it can be dynamic.</p>

username: smth
cooked: <p>even after <code>self.weight = nn.Parameter(...)</code>, you can change size of <code>self.weight</code> manually if you want.</p>
<p>But it’s still not clear to me how you are changing the size of <code>self.weight</code>. How is <code>self.weight</code> actually computed? (not the shape, but also the value of the elements?)</p>

username: JosueCom
cooked: <p>I implemented your suggestion but the output shape does not change</p>
<aside class="quote quote-modified" data-post="1" data-topic="127807">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/josuecom/48/27043_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/dynamic-linear-layer-doesnt-change-output-when-weights-shape-is-changed/127807">Dynamic linear layer doesn't change output when weights shape is changed</a> 
  </div>
  <blockquote>
    If I implement a dynamic layer as present below, it’s output shape doesn’t get affected. See code below for greater details. 
&gt;&gt;&gt; import torch as th
&gt;&gt;&gt; import torch.nn as nn
&gt;&gt;&gt; import torch.nn.functional as F
&gt;&gt;&gt; class DyLinear(nn.Module):
...     def __init__(self, in_fea, out_fea, device = 'cuda'):
...         super(DyLinear, self).__init__()
...         self.device = device
...         self.in_fea = in_fea
...         self.out_fea = out_fea
...         self.weight = nn.Parameter(th.randn(ou…
  </blockquote>
</aside>