Adding input to each conv layer in resnet pre-trained model

username: Frida
cooked: <p>Hi,<br>
Does someone know how we can give more then one input at each conv layer i,e., not only the tensor?<br>
Thanks!</p>

username: ptrblck
cooked: <p>How should the additional input look like?<br>
If it has the same shape as the “first” input, you could just append it to the batch.</p>

username: Frida
cooked: <p>it is scalar not the same shape</p>

username: ptrblck
cooked: <p>If the conv layer has a kernel shape of <code>1x1</code> and expects <code>in_channels=1</code>, you could reshape the scalar value into a tensor with the shape <code>[batch_size, 1, 1, 1]</code> and feed it directly to the conv layer.<br>
If that’s not the case, you won’t be able to directly feed a scalar value to it and would need to either add/copy/subtract/etc. it to another input or use another layer.</p>

username: Frida
cooked: <p>Thanks a lot!<br>
Actually, having the known resnet network with sequential and BasicBlock. I just want to add a parameter that will be change during the training, say the padding and I want to control it from the beginning. For example. assume that at each epoch I want to change the padding for specific layer in the network. It looks like I need to open the network into Modules, right?</p>

username: ptrblck
cooked: <p>You can change some attributes such as the padding directly by setting it in the module as seen here:</p>
<pre><code class="lang-python">model = models.resnet18()
model.conv1.register_forward_hook(lambda m, input, output: print(output.shape))
print(model.conv1.padding)
&gt; (3, 3)

x = torch.randn(1, 3, 224, 224)
out = model(x)
&gt; torch.Size([1, 64, 112, 112])

model.conv1.padding = (0, 0)
print(model.conv1.padding)
&gt; (0, 0)

out = model(x)
&gt; torch.Size([1, 64, 109, 109])
</code></pre>
<p>Note however, that changing the padding will also change the output activation shape as seen in the example.<br>
While this approach works in <code>resnet18</code> due to the usage of adaptive pooling layers before applying the linear layer, you might run into shape mismatches for other models.</p>

username: Frida
cooked: <p>And what is that was a parameter? how can I change it and keep working with the new number keep updating it during the training. As I see, when choosing the new number it get stuck there. Need to wrote it as model.padding = nn.Parameter(3.0)</p>

username: ptrblck
cooked: <p>No, <code>padding</code> cannot be a trainable parameter, if I’m not mistaken.<br>
If you are running into shape mismatches after changing the model, you would have to change the architecture e.g. by applying an adaptive pooling layer (as done in the resnet) or by using different linear layers later in the model.</p>
<p>The reason the for mismatch is that you are changing the shape of the intermediate activations, while the linear layers need a fixed input shape to work properly (set by <code>in_feautres</code>).</p>

username: Frida
cooked: <p>Assuming that we will solve the mismatch issue, is there any chance to distribute the padding during the training, say, having the loss, and a given backward function for each layer, if the “padding parameter” had to enlarged, set +1 to the padding in a given layer.</p>

username: ptrblck
cooked: <p>No, I don’t think the padding is trainable, at least I’m not aware of a method to learn it.<br>
You might want to look into meta-learning, which can be used to “learn” new architectures and might for your use case. However, I don’t think you can directly use gradient descent to optimize the padding.</p>

username: Frida
cooked: <p>Another related issue, if we set some parameter using nn.Parameter, pytorch is automatically calculate its gradients. When do we need to set a backward function?</p>

username: ptrblck
cooked: <p>Autograd will calculate the gradients of the loss w.r.t. all used parameters used to calculate this loss during the <code>backward()</code> call.<br>
Without it, there won’t be any gradients.</p>

username: Frida
cooked: <p>so in fact if I set padding as nn.parameter I do get gradients, and then, I set it in the conv layer, what are we learning in that way?</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Frida" data-post="13" data-topic="100415">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/57b2e6/48.png" class="avatar"> Frida:</div>
<blockquote>
<p>so in fact if I set padding as nn.parameter I do get gradients</p>
</blockquote>
</aside>
<p>How did you achieve it? Could you post your approach, please?</p>

username: Frida
cooked: <p>I didn’t try it just trying to understand. In general I can set nn.parameter and then use it as the padding, still it is nn.parameter you will get gradients.</p>

username: ptrblck
cooked: <p>No, that won’t work. The <code>padding</code> parameter is (a tuple of) integer(s), which are not trainable and also I don’t know how Autograd could even calculate the gradients for it at the moment.</p>