[MAdd]: Dropout is not supported!

username: Humza_Sami
cooked: <p>I am trying to reproduce this <a href="https://pytorch.org/docs/stable/generated/torch.nn.TransformerDecoder.html" rel="noopener nofollow ugc">code</a> snipped from PyTorch.When I pass the input to the model it returns the following warnings. Will it effect my outputs ?</p>
<blockquote>
<p>Warning</p>
</blockquote>
<blockquote>
<p>[MAdd]: Dropout is not supported!<br>
[Flops]: Dropout is not supported!<br>
[Memory]: Dropout is not supported!<br>
[MAdd]: LayerNorm is not supported!<br>
[Flops]: LayerNorm is not supported!<br>
[Memory]: LayerNorm is not supported!</p>
</blockquote>
<p>I am using the following code</p>
<pre><code class="lang-auto">decoder_layer = nn.TransformerDecoderLayer(d_model=512, nhead=8)
transformer_decoder = nn.TransformerDecoder(decoder_layer, num_layers=6)
memory = torch.rand(10, 32, 512)
tgt = torch.rand(20, 32, 512)
out = transformer_decoder(tgt, memory)
</code></pre>

username: ptrblck
cooked: <p>I haven’t seen these warnings before. Could you post a complete, minimal and executable code snippet as well as the entire output log including the stacktrace, please?</p>

username: Humza_Sami
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Please find attached complete code snippet:</p>
<pre><code class="lang-auto">import numpy as np
import torch
from torch import nn
from torchinfo import summary
decoder_layer = torch.nn.TransformerDecoderLayer(d_model=128, nhead=8,dim_feedforward=256)
Decoder = torch.nn.TransformerDecoder(decoder_layer, num_layers=1)
src = torch.rand(1, 128)
tgt = torch.rand(1, 128)
out = Decoder(tgt,src)
</code></pre>
<p>This code generates a vector of [1x128] with the following messages:</p>
<pre><code class="lang-auto">[MAdd]: Dropout is not supported!
[Flops]: Dropout is not supported!
[Memory]: Dropout is not supported!
[MAdd]: LayerNorm is not supported!
[Flops]: LayerNorm is not supported!
[Memory]: LayerNorm is not supported!
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/5/65b4273aadbdf991f787b65da6bc62173b67d2b1.png" data-download-href="https://discuss.pytorch.org/uploads/default/65b4273aadbdf991f787b65da6bc62173b67d2b1" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/5/65b4273aadbdf991f787b65da6bc62173b67d2b1.png" alt="image" data-base62-sha1="evIcsPRa75l9AbgA5509iIiLBoB" width="690" height="330" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1017×487 30.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>Are these warning still raised if <code>torchinfo</code> is not imported?</p>

username: Humza_Sami
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> No, they occurred with only <code>torchinfo</code>.</p>

username: ptrblck
cooked: <p>In that case I would recommend to try to update to the latest release of <code>torchinfo</code> and to check if these warnings are still raised. If so, you might want to create a GitHub issue in their repository.</p>