About kernel in transpose convolution

last: <p>i wonder how kernel get initialized in transpose convolution, such as ConvTranspose1d, ConvTranspose2d, and ConvTranspose3d. is bilinear interpolation applied to the initialization of the kernel?<br>
in addition, how does the kernel(filter) in the transpose convolution become learnable?</p>
<p>how can the kernel learn? does the kernel learn through back-propagation, like the weights in layers in transpose convolution(ConvTranspose1d, ConvTranspose2d, and ConvTranspose3d) ?</p>

ptrblck: <aside class="quote no-group" data-username="last" data-post="1" data-topic="113551">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/ecae2f/48.png" class="avatar"> last:</div>
<blockquote>
<p>i wonder how kernel get initialized in transpose convolution, such as ConvTranspose1d, ConvTranspose2d, and ConvTranspose3d. is bilinear interpolation applied to the initialization of the kernel?</p>
</blockquote>
</aside>
<p>Convolution layers will be initialized by default using this <a href="https://github.com/pytorch/pytorch/blob/30dd15e778195c6b7cf266a0b4b20af9c9ca5ba7/torch/nn/modules/conv.py#L114-L119"><code>reset_parameters()</code></a> method. No interpolation is applied and the mentioned uniform sampling will be applied.</p>
<aside class="quote no-group" data-username="last" data-post="1" data-topic="113551">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/ecae2f/48.png" class="avatar"> last:</div>
<blockquote>
<p>in addition, how does the kernel(filter) in the transpose convolution become learnable?</p>
<p>how can the kernel learn? does the kernel learn through back-propagation, like the weights in layers in transpose convolution(ConvTranspose1d, ConvTranspose2d, and ConvTranspose3d) ?</p>
</blockquote>
</aside>
<p>The <code>weight</code> parameter of the conv layers are the kernels and they will be trained as any other parameters through the backpropagation etc.</p>

last: <p>Thank you very much for the information!<br>
regarding kernel, I have additional questions.<br>
you mentioned that “the <code>weight</code> parameter of the conv layers are the kernels”.<br>
in the case, will bias will be included in the weights(kernel) when bias is set to “True”  in Convolution/ Transpose Convolution ? or<br>
will the bias be added to output value when  bias is set to “True”</p>
<p>About the inltialization of weights,  i saw that  nn.init.uniform_ is used in one case and nn.init.xavier_uniform_ is used in other case.<br>
is there any particular standard for the initialization?</p>
<p>in the linear layer, how can i adjust the size of weights, including bias?</p>
<p>thank you very much!</p>

ptrblck: <aside class="quote no-group" data-username="last" data-post="3" data-topic="113551">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/ecae2f/48.png" class="avatar"> last:</div>
<blockquote>
<p>in the case, will bias will be included in the weights(kernel) when bias is set to “True” in Convolution/ Transpose Convolution ? or<br>
will the bias be added to output value when bias is set to “True”</p>
</blockquote>
</aside>
<p>The <code>bias</code> will be added after a filters were applied to the input.</p>
<aside class="quote no-group" data-username="last" data-post="3" data-topic="113551">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/ecae2f/48.png" class="avatar"> last:</div>
<blockquote>
<p>About the inltialization of weights, i saw that nn.init.uniform_ is used in one case and nn.init.xavier_uniform_ is used in other case.<br>
is there any particular standard for the initialization?</p>
</blockquote>
</aside>
<p>The default initializations are mentioned in the docs for the corresponding layer and are defined in the <code>reset_parameters()</code> method. Depending on your model, use case etc., you might want to change these init methods.</p>
<aside class="quote no-group" data-username="last" data-post="3" data-topic="113551">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/ecae2f/48.png" class="avatar"> last:</div>
<blockquote>
<p>in the linear layer, how can i adjust the size of weights, including bias?</p>
</blockquote>
</aside>
<p><code>nn.Linear</code> expects the <code>in_features</code> and <code>out_features</code> arguments, which define the shape of the <code>weight</code> and <code>bias</code> parameters.</p>

last: <p>Thank you very much for guidance!. Really appreciate it</p>