Cuda of memory of my code

xclmj: <p>Hi, all,</p>
<pre><code>    I implement my code about an Autoendocer with dense blcok on the basis of Pytorch.  The total number of parameters around 0.8M.  The image size is 100*100.  the batchsize for training 100. However, when I launch CUDA, an error "cuda out of memory" will be ocurred.  Can anyone can help me. My GPU card : 
</code></pre>
<p>-----------------------------------------------------------------------------+<br>
| NVIDIA-SMI 430.14       Driver Version: 430.14       CUDA Version: 10.2     |<br>
|-------------------------------±---------------------±---------------------+<br>
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |<br>
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |<br>
|===============================+======================+======================|<br>
|   0  GeForce GTX 745     Off  | 00000000:01:00.0  On |                  N/A |<br>
| 25%   59C    P0    N/A /  N/A |   4037MiB /  4043MiB |      1%      Default |<br>
±------------------------------±---------------------±---------------------+</p>
<p>±----------------------------------------------------------------------------+<br>
| Processes:                                                       GPU Memory |<br>
|  GPU       PID   Type   Process name                             Usage      |<br>
|===========================================================================</p>
<p>Thanks.</p>

albanD: <p>Hi,</p>
<p>There is not much you can do to reduce GPU memory usage except reducing the batch size and use inplace operations for activations like ReLUs if possible (if you see an error saying that a tensor needed for backward has been changed by an inplace operation, then that means that you cannot use them at that particular place).</p>

xclmj: <p>Hi,</p>
<pre><code>    Thanks for your replying.  My GPU card only has around 4G memory.  This error will be resolved if I use a GPU card with larger memory? 
</code></pre>
<p>best regard</p>

ptrblck: <p>This might be the case. If it’s possible to post the model definition and some dummy inputs (and targets) I could check the memory usage on larger GPUs.</p>

xclmj: <p>Hi, ptrblck,</p>
<pre><code>Thanks for your kind help.  The following link is the github of this code: 
   https://github.com/xclmj/Dense-Convolutional-Encoder-Decoder-Network-with-Dense-Block.git
</code></pre>
<p>You can download the data from : <a href="https://drive.google.com/drive/folders/1VkYtS2oe-vwapUjwIG_0GdFzR_RMw2xW?usp=sharing" rel="nofollow noopener">https://drive.google.com/drive/folders/1VkYtS2oe-vwapUjwIG_0GdFzR_RMw2xW?usp=sharing</a>.</p>
<p>best regard</p>

ptrblck: <p>Thanks for the code.<br>
I’ve used the default arguments and this script to run the test:</p>
<pre><code class="lang-python">device = 'cuda'

# enters time in latent
model = DenseEDT(1,
                 2,
                 blocks=(4, 9, 4),
                 times=(5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100),
                 growth_rate=24,
                 drop_rate=0,
                 bn_size=100,
                 num_init_features=48,
                 bottleneck=False,
                 time_channels=1).to(device)


optimizer = torch.optim.Adam(model.parameters(), lr=0.001,
                             weight_decay=5e-4)
criterion = nn.MSELoss()

data = torch.randn(10, 1, 224, 224, device=device)
times = torch.randn(10, device=device)

output = model(data, times)
target = torch.empty_like(output).normal_()

loss = criterion(output, target)
loss.backward()

print('mem allocated {:.3f}MB'.format(torch.cuda.memory_allocated()/1024**2))
&gt; mem allocated 16.433MB
print('max mem allocated {:.3f}MB'.format(torch.cuda.max_memory_allocated()/1024**2))
&gt; max mem allocated 2146.784MB
print('mem cached {:.3f}MB'.format(torch.cuda.memory_cached() / 1024**2))
&gt; mem cached 2440.000MB
</code></pre>
<p><code>nvidia-smi</code> shows a usage of approx. 3500MB using a Titan V GPU with 12GB memory.<br>
The additional usage comes from the cuda context, which will take some space on the GPU.</p>