akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
🆘How can I set max_split_size_mb to avoid fragmentation？

username: Ruby-G0
cooked: <p>I was using Pytorch to train a bilstm-crf model while encountered the OOM error. The batch size is 16.</p>
<p>I tried to redeuce the batch size to 8 → 4 -&gt;2 -&gt;1. But similar error happens again and again. This is my error. It seems that a lot of memory cannot be fully utilized. How can I fix it?</p>
<pre><code class="lang-auto">0%|          | 1/872 [00:37&lt;9:10:08, 37.90s/it]
  0%|          | 2/872 [00:57&lt;6:33:41, 27.15s/it]
  0%|          | 2/872 [00:58&lt;7:01:57, 29.10s/it]
Traceback (most recent call last):
  File "/home/workspace/EvaHan/src/run.py", line 108, in &lt;module&gt;
    main()
  File "/home/workspace/EvaHan/src/run.py", line 103, in main
    k_fold_run(config.trainset_path[0], vocab, device)
  File "/home/workspace/EvaHan/src/run.py", line 88, in k_fold_run
    test_loss, f1 = run(word_train, label_train, word_dev, label_dev, vocab, device, kf_index)
  File "/home/workspace/EvaHan/src/run.py", line 61, in run
    train(train_loader, dev_loader, vocab, model, device, kf_index)
  File "/home/workspace/EvaHan/src/train.py", line 58, in train
    out_feats = model.forward(batch_data)
  File "/home/workspace/EvaHan/src/model.py", line 35, in forward
    sequence_output, _ = self.bilstm(embeddings)
  File "/home/tools/enter/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/tools/enter/envs/py38/lib/python3.8/site-packages/torch/nn/modules/rnn.py", line 691, in forward
    result = _VF.lstm(input, hx, self._flat_weights, self.bias, self.num_layers,
RuntimeError: CUDA out of memory. Tried to allocate 14.96 GiB (GPU 0; 31.75 GiB total capacity; 15.45 GiB already allocated; 8.05 GiB free; 22.26 GiB reserved in total by PyTorch) If reserved memory is &gt;&gt; allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

</code></pre>
Query numerical infos for ScalarType

username: MDzz
cooked: <p>I have been trying without success to query the numerical information of torch::ScalarTypes, e.g. maximum representable value, epsilon etc.</p>
<p>I would like to do something like what std::numeric_limits does for cpp type, e.g.<br>
std::numeric_limits. Does anyone have a good way to do that?</p>

username: ptrblck
cooked: <p>The PyTorch <code>finfo</code> object used as:</p>
<pre><code class="lang-python">torch.finfo(torch.float32)
finfo(resolution=1e-06, min=-3.40282e+38, max=3.40282e+38, eps=1.19209e-07, smallest_normal=1.17549e-38, tiny=1.17549e-38, dtype=float32)
</code></pre>
<p>would internally call into <a href="https://github.com/pytorch/pytorch/blob/30fb2c4abaaaa966999eab11674f25b18460e609/torch/csrc/TypeInfo.cpp#L109-L205">these methods</a>, so you should be able to use the same in your C++ application. I.e.:</p>
<pre><code class="lang-auto">std::numeric_limits&lt;at::scalar_value_type&lt;scalar_t&gt;::type&gt;::epsilon());
</code></pre>
<p>etc. should work.</p>

username: MDzz
cooked: <p>Thank you for your answer.<br>
I tried it out and I’m a bit confused by how to pass a torch::ScalarType to the at::scalar_value_type method.</p>
<p>If I try to insert a torch::ScalarType in place of the the scalar_t above, I get:</p>
<blockquote>
<p>[build] …/torchOps/torchOps.cpp:165:64: error: template argument for template type parameter must be a type<br>
[build]   auto f32_espilon = std::numeric_limits&lt;at::scalar_value_type&lt; torch::kF32&gt;::type&gt;::epsilon();<br>
[build]                                                                ^~~~~~~~~~~<br>
[build] …/libtorch/include/c10/util/complex_utils.h:21:20: note: template parameter is declared here<br>
[build] template &lt; typename T&gt;<br>
[build]                    ^<br>
[build] 1 error generated.</p>
</blockquote>
<p>I’m not experienced with cpp very much, so that could be the source of my confusion. But I can see from <a href="https://github.com/pytorch/pytorch/blob/master/c10/util/complex_utils.h" rel="noopener nofollow ugc">the definition of at::scalar_value_type</a> that indeed it expects a type.</p>
<p>Any idea what I’m doing wrong?</p>
Torch.cuda.is_available() is false with cuda 10.2

username: zzw3833
cooked: <p>I try to get GPU support on pytorch working. I can use cuda 10.2 and python 3.6 but with: pip3 install torch torchvision torchaudio it´s not working… What can I do? Also I could use cuda  11, cuda11.1, 11.2, 11.4 and 11.5.<br>
thanks in advance!</p>