Dropout isn't zero-ing out any of my data points (but it is scaling them)

username: josiahjdavis
cooked: <p>When I call dropout it is not zero-ing out any of my datapoints. I have tried the layer and functional formats. I am using PyTorch 1.3.0. Here is sample code and output:</p>
<pre><code class="lang-auto">import platform; print(f"Platform {platform.platform()}")
import sys; print(f"Python {sys.version}")
import torch; print(f"PyTorch {torch.__version__}")
import torch.nn as nn
class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.do = nn.Dropout(p=.5)
        
    def forward(self, x):
        return self.do(x)
net = Net()
data = torch.Tensor([1., 2., 3., 4., 5., 6.]).view(3, 1, 2)
print(data)
net.train()
print(net(data))
</code></pre>
<p>Here is sample output:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/8/78987571161dfcbb38db105b90d5410a1b7c986f.png" data-download-href="https://discuss.pytorch.org/uploads/default/78987571161dfcbb38db105b90d5410a1b7c986f" title="27%20PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/8/78987571161dfcbb38db105b90d5410a1b7c986f_2_510x499.png" alt="27%20PM" data-base62-sha1="hcPW924PF4To3xretibDCmDqWf5" width="510" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/8/78987571161dfcbb38db105b90d5410a1b7c986f_2_510x499.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/8/78987571161dfcbb38db105b90d5410a1b7c986f_2_765x748.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/8/78987571161dfcbb38db105b90d5410a1b7c986f_2_1020x998.png 2x" data-dominant-color="F5F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">27%20PM</span><span class="informations">1242×1216 108 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: tom
cooked: <p>So under ideal conditions there is a 1/64 chance to have a p=0.5 dropout not set anything to 0. You might just have gotten lucky and when you re-run it, you should get zeros, too.</p>
<p>P.S.: <a href="https://pytorch.org/blog/pytorch-0_4_0-migration-guide/" rel="nofollow noopener">it’s <code>torch.tensor</code> with a lower-case t.</a></p>

username: josiahjdavis
cooked: <p>Thanks for taking a look! Unfortunately, I didn’t get lucky <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=12" title=":frowning:" class="emoji" alt=":frowning:" loading="lazy" width="20" height="20"> . Thanks for the tip about the lower case.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/3/5317b70c62e954ae1c88ad01680317e226097c93.png" data-download-href="https://discuss.pytorch.org/uploads/default/5317b70c62e954ae1c88ad01680317e226097c93" title="32%20PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/3/5317b70c62e954ae1c88ad01680317e226097c93_2_431x500.png" alt="32%20PM" data-base62-sha1="bR4reyu8LFqNdiK2jJFI7wn275x" width="431" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/3/5317b70c62e954ae1c88ad01680317e226097c93_2_431x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/3/5317b70c62e954ae1c88ad01680317e226097c93_2_646x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/3/5317b70c62e954ae1c88ad01680317e226097c93_2_862x1000.png 2x" data-dominant-color="F6F6F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">32%20PM</span><span class="informations">1254×1452 122 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: tom
cooked: <p>Hm. Is that on cuda only or on the CPU as well?</p>

username: LLYX
cooked: <p>Did this issue get resolved? I seem to be having it as well, on PyTorch 1.2.</p>

username: josiahjdavis
cooked: <p>Yes, I was able to get this resolved.</p>
<p>For me, the issue went away when I spun up a new EC2 instance and used the default PyTorch 3.6 virtual environment.</p>
<p>So I must have messed something up when I was doing package installation including perhaps my installation of PyTorch.</p>
<p>FWIW here are the current details in my env (p3.2xlarge, Ubuntu 18.04, Deep Learning AMI):</p>
<p>Platform Linux-4.15.0-1057-aws-x86_64-with-debian-buster-sid<br>
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)<br>
[GCC 7.2.0]<br>
PyTorch 1.3.1</p>