akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How to AMP with BCE that does **not** directly operate on `sigmoid` results

eliphatfs: <p>In my model, I have:</p>
<pre><code class="lang-python">logits1, gate1, logits2, gate2 = net(x)
gated_prob1 = F.sigmoid(logits1) ** F.sigmoid(gate1)
gated_prob2 = F.sigmoid(logits2) ** F.sigmoid(gate2)
F.binary_cross_entropy(gated_prob1 * gated_prob2, gt)
</code></pre>
<p>The operation errors when enabling <code>amp</code>:</p>
<pre><code class="lang-auto">RuntimeError: torch.nn.functional.binary_cross_entropy and torch.nn.BCELoss are unsafe to autocast.
Many models use a sigmoid layer right before the binary cross entropy layer.
In this case, combine the two layers using torch.nn.functional.binary_cross_entropy_with_logits
or torch.nn.BCEWithLogitsLoss.  binary_cross_entropy_with_logits and BCEWithLogits are
safe to autocast.
</code></pre>
<p>What can I do since I am <strong>not</strong> simply a sigmoid layer before that BCE?</p>

ptrblck: <aside class="quote no-group" data-username="eliphatfs" data-post="1" data-topic="173461">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/eliphatfs/48/57712_2.png" class="avatar"> eliphatfs:</div>
<blockquote>
<p>What can I do since I am <strong>not</strong> simply a sigmoid layer before that BCE?</p>
</blockquote>
</aside>
<p>Based on your posted code snippet you are explicitly using <code>F.sigmoid</code> before passing these outputs to <code>F.binary_cross_entropy</code>, which is the functional form of <code>nn.BCELoss</code>:</p>
<pre data-code-wrap="python"><code class="lang-python">logits1, gate1, logits2, gate2 = net(x)
gated_prob1 = F.sigmoid(logits1) ** F.sigmoid(gate1)
gated_prob2 = F.sigmoid(logits2) ** F.sigmoid(gate2)
F.binary_cross_entropy(gated_prob1 * gated_prob2, gt)
</code></pre>

eliphatfs: <p>Thanks ptrblck,<br>
However, I cannot combine F.sigmoid with BCE into BCE with logits, since I have done some other operations between them.</p>

ptrblck: <p>In that case you might want to disable <code>autocast</code> for these operations, but note that even without <code>amp</code> <code>nn.BCELoss</code> has less numerical stability than <code>nn.BCEWithLogitsLoss</code>.<br>
See e.g. <a href="https://discuss.pytorch.org/t/torch-flatten-causes-zero-grad/172882/4">this recent topic</a>.</p>

eliphatfs: <p>Thanks ptrblck,<br>
After some trials and errors, I am now excluding the sigmoid post-processing (the code snippet before) <strong>as well as the last linear layer</strong> from <code>autocast</code>. It seems to be working.</p>
I get NAN in the mel_scale part of the mel spectrogram

skerlet_flandorle: <p>I tried VQVAE on audio and I used a method where I had the raw waveform output to the decoder, converted it to the logarithm of the mel spectrogram and then took loss (I added 1e-7 to make sure it didn’t go to 0).<br>
This worked fine for normal VAE.<br>
However, when I changed to VQVAE, somewhere around the 0.1~3rd epoch of the learning phase, NAN was output at the part where the mel spectrogram is converted (I did not try after the 3rd epoch).</p>
<p>The reason I determined this to be the case was because I checked for inf -inf nan at each stage as follows, and the part that was always caught was the transform.mel_scale(spec) part.<br>
Also, when loss is measured after making a spectrogram instead of a mel-spectrogram, the accuracy is not good, but the learning itself is possible.</p>
<blockquote>
<p>spec=transform.spectrogram(wave)<br>
assert not float(“inf”) in spec<br>
assert not float(“-inf”) in spec<br>
assert not torch.isnan(spec).any()<br>
mel=transform.mel_scale(spec)<br>
assert not torch.isnan(mel).any()</p>
</blockquote>
Consequences of PyTorch 2.0 on libtorch?

Xavier31: <p>Hi, I was wondering what kind of consequences the introduction of PyTorch 2.0 would have on libtorch ?<br>
If I understand correctly, this means throwing away Torchscript in favour of TorchInductor ? Would the compiled model in pytorch be directly usable in libtorch ?<br>
Thanks in advance for any insights</p>

ptrblck: <p>From <a class="mention" href="/u/chillee">@Chillee</a> on Slack:</p>
<blockquote>
<p>[TorchScript] is not deprecated. We are not actively developing torchscript, but it will be supported for the foreseeable future, and you’ll be able to use torchscript based backends behind dynamo. We currently have no plans on making torchinductor consume torchscript graphs. We are working on making the kernels it generates exportable to C++, but since the compiler is in Python (and so is much of Triton), I’m not sure there’s that much advantage to making it take in torchscript graphs.</p>
</blockquote>

Chenchao_Zhao: <p>TorchScript was getting more and more stable over the years now we are getting a new cutting edge paradigm with many potential bugs just like the early days of TS. I don’t see why that’s a rational choice.</p>
DeepSpeed libarary - Can not install - getting error - "unable to compile cuda/cpp extensions without a matching cuda version"

R_J: <p>I understand that its happening for cuda mismatch, but otherwise PyTorch itself is running just fine.</p>
<p>After running <code>pip install depspeed</code> getting the below error</p>
<p>OS - Windows 11<br>
Python 3.10.9</p>
<pre><code class="lang-auto">Collecting deepspeed
  Using cached deepspeed-0.8.1.tar.gz (759 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─&gt; [13 lines of output]
      Traceback (most recent call last):
        File "&lt;string&gt;", line 2, in &lt;module&gt;
        File "&lt;pip-setuptools-caller&gt;", line 34, in &lt;module&gt;
        File "C:\Users\PC60\AppData\Local\Temp\pip-install-7aimsv5v\deepspeed_b92cc522763946a986a0e569e5e169e0\setup.py", line 166, in &lt;module&gt;
          ext_modules.append(builder.builder())
        File "C:\Users\PC60\AppData\Local\Temp\pip-install-7aimsv5v\deepspeed_b92cc522763946a986a0e569e5e169e0\op_builder\builder.py", line 623, in builder
          self.build_for_cpu = not assert_no_cuda_mismatch(self.name)
        File "C:\Users\PC60\AppData\Local\Temp\pip-install-7aimsv5v\deepspeed_b92cc522763946a986a0e569e5e169e0\op_builder\builder.py", line 105, in assert_no_cuda_mismatch
          raise Exception(
      Exception: &gt;- DeepSpeed Op Builder: Installed CUDA version 12.0 does not match the version torch was compiled with 11.7, unable to compile cuda/cpp extensions without a matching cuda version.
      DS_BUILD_OPS=1
       [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
       [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─&gt; See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
</code></pre>

ptrblck: <p>Your locally installed CUDA toolkit (in this case CUDA 12.0) will be used to build custom CUDA extensions and should match the CUDA version used to build the PyTorch binaries (in this case CUDA 11.7).<br>
Downgrade your local CUDA toolkit to 11.7 or build PyTorch from source using CUDA 12.0.</p>
Unzip file via ssh

Maryam_Darei: <p>I have a meta data zip file. I know it includes train and test folder. I want to unzip a part of this file(just train folder). Is there any linux command for this?</p>

maliozer: <p>I guess the following command should work for unzipping specific folder,</p>
<pre><code class="lang-auto">unzip folder.zip 'train/*'

</code></pre>
Can I operate DataLoader asynchronously?

FunkyKoki: <p>In my network, I have to do a lot of process to transform the pic in DataLoader’s <code>__getitem__</code>, and this makes the training much slower. Now I have tried the <code>num_workers</code>, thanks for God, it helps a lot. However, I still want to accelerate the training speed, so ‘asyncio’ comes to my mind. But, unluckily, I know little about this lib. At the same time, I am not sure whether it is compatible with the <code>num_workers</code>.<br>
Hope you can help me, thanks a lot!</p>

Kushaj: <p>Well, it would make things very complicated and I don’t think they would go hand in hand. Well if your data loading process is becoming an issue, you can save intermediate tensors after applying transforms to disk and directly load them.</p>

rasbt: <p>currently, num_workers uses Python’s multiprocessing. So in that case, it already runs processes asynchronously to each other (depending how it’s used, you don’t have to though) so using asyncio wouldn’t be of much benefit I assume.</p>

jwickens: <p>I found it extremely useful to use async for my use case with postgres backed data. So I simply did not use it all. It took me a while to figure out I would be better off because I kept thinking that the built in Dataloader was worth it. There’s a writeup with link to code here: <a href="https://medium.com/@jonathan.wickens/using-postgres-as-a-dataloader-with-pytorch-bba0d5cbe1fa" rel="nofollow noopener">https://medium.com/@jonathan.wickens/using-postgres-as-a-dataloader-with-pytorch-bba0d5cbe1fa</a> I hope it helps someone. Sorry for reviving an old topic but I kept coming back to this post before I found a way forward!</p>

whille_zg: <p>with python3.7+, asyncio is more convenient to use. I wonder why not use AIO to save CPU, and avoid sync problems between multiprocess?</p>
Can't utilize RocM

Andrew_Bouras: <p>Hello,</p>
<p>I am trying to install pytorch on my PC in Ubuntu 22.04 enviroment but when I ask from pytorch to execute some simple tests it returns errors as segmentation faults - core dumped.</p>
<p>I’ve tried the docker method ( <a href="https://www.youtube.com/watch?v=HwGgzaz7ipQ" class="inline-onebox" rel="noopener nofollow ugc">Machine learning on an AMD 6900xt using ROCM and a docker container - YouTube</a> ) but I tried install pytroch from the official website too. Both were successfull.</p>
<p>CUDA is visible from the system and the GPU has been tested in 2 different PCs with a variety of pc parts. Tried both AMD and Intel PCs as well as DDR4 and DDR5 systems.</p>
<p>Pytorch also sees my GPU as CUDA and the RAM of the system in each test was always more than 16GB , the swap has been tested from 4 up to 32 GB.</p>
<p>The GPU I use is RX 6700xt which has also been tested  using a variety of tests (eventhough it’s brand new) as well as gaming for many hours.</p>
How to register a pytree for an ouptut class?

fabian_schutze: <p>How can I register a pytree for a class so that I can apply the functorch’s <code>memory_efficient_fusion</code>?</p>
<p>For Mask R-CNN type predictions a sampling result is calculated during the forward pass. The sampling results is wrapped inside the following class:</p>
<pre><code class="lang-auto">class SamplingResult:

    def __init__(self, pos_inds: Tensor, neg_inds: Tensor):
        self.pos_inds = pos_inds
       self.neg_inds = neg_inds
</code></pre>
<p>During the compilations phase for functorch, I receive the following error:</p>
<pre><code class="lang-auto">RuntimeError: Found &lt;class 'multimodal.models.middle_networks.random_sampler.SamplingResult'&gt; in output, which is not a known type. If this type holds tensors, you need to register a pytree for it. See https://github.com/pytorch/functorch/issues/475 for a brief explanation why. If you don't need to register a pytree, please leave a comment explaining your use case and we'll make this more ergonomic to deal with
</code></pre>
<p>I looked at the linked issue but I couldn’t figure out how to create a pytree from this. Is there some documentation about it?</p>

fabian_schutze: <p>I figures out a way to do it. For reference, the jax docs for <em>Custom pytree nodes</em> turned out to be very helpful indeed: <a href="https://jax.readthedocs.io/en/latest/jax-101/05.1-pytrees.html" class="inline-onebox" rel="noopener nofollow ugc">Working with Pytrees — JAX documentation</a></p>
Data Dependent Control Flow - Functorch

fabian_schutze: <p>I am trying to use <code>functorch.compile.memory_efficient_fusion</code> on a Mask R-CNN type model. While doing so, I ran into several problems about data dependent control flow. In particular, the error message is:</p>
<pre><code class="lang-auto">RuntimeError: It appears that you're trying to get value out of a tracing tensor - erroring out! It's likely that this is caused by data-dependent control flow or similar.
</code></pre>
<p>One interesting line of code causing the compilation to fail are as follows (The snippet below works with random data created)</p>
<pre><code class="lang-auto">import torch
num_gts = 14
num_anchors = 1000
assigned_gt_inds = torch.randint(-1, 10, size=(num_anchors,))
gt_max_overlaps = torch.rand((num_gts,))
min_pos_iou = 0.3
gt_max_assign_all = True
overlaps = torch.rand((num_gts, num_anchors))
gt_argmax_overlaps = torch.randint(0, num_anchors, size=(num_gts,))
for i in range(num_gts):
    if gt_max_overlaps[i] &gt;= min_pos_iou:
        if gt_max_assign_all:
            max_iou_inds = overlaps[i, :] == gt_max_overlaps[i]
            assigned_gt_inds[max_iou_inds] = i + 1
        else:
            assigned_gt_inds[gt_argmax_overlaps[i]] = i + 1
</code></pre>
<p>How can I rewrite this snippet compliant with <code>functorch</code>? At first, I thought I’d be looking into a double <code>torch.where</code> but then I realized that this is not possible: <code>assigned_gt_inds</code> needs to only to be changed if  <code>gt_max_overlaps[i] &gt;= min_pos_iou</code>, otherwise <code>assigned_gt_inds</code> remains unchanged. However, the position at which to change this tensor are dependent on the second condition. Please also not that <code>overlaps</code> is a 2D tensor, so I’m not sure how to calculate this condition.</p>
<p>Does somebody have an idea how to rewrite the snippet above to work with functorch?</p>
<p>For reference, the code comes from mmdetection (<a href="https://github.com/open-mmlab/mmdetection/blob/3.x/mmdet/models/task_modules/assigners/max_iou_assigner.py#L217" class="inline-onebox" rel="noopener nofollow ugc">mmdetection/max_iou_assigner.py at 3.x · open-mmlab/mmdetection · GitHub</a>) and I would also reference a relevant github ticket for pytorch: <a href="https://github.com/pytorch/functorch/issues/257" class="inline-onebox" rel="noopener nofollow ugc">Data-dependent control flow exploration · Issue #257 · pytorch/functorch · GitHub</a></p>
Which Pytorch/Cuda version to use with newest 3060ti driver?

malleyconnor: <p>Hello,</p>
<p>I am running WSL-2 with Ubuntu-20.0.4<br>
I’ve just installed a new 3060ti graphics card, and installed the latest driver with CUDA 11.6.<br>
I was running CUDA 10.2 on my old GTX 1060 with no issues before.<br>
See the nvidia-smi output below</p>
<pre><code class="lang-auto">+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03    Driver Version: 512.15       CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+

</code></pre>
<p>I installed pytorch + cuda toolkit 11.3 in a conda environment as suggested on the Pytorch website</p>
<pre><code class="lang-auto">conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
</code></pre>
<p>However when running a python program, executing torch.cuda.is_available() returns False:</p>
<pre><code class="lang-auto">CUDA is available: False
CUDA toolkit version:  11.3
No GPU available, using the CPU instead.
</code></pre>
<p>Can anyone help walk me through why I am having issues? I figured the newest 512.15 graphics driver would be backwards compatible with CUDA 11.3, but I may be wrong.</p>

Wologman: <p>I’m having exactly the same problem.  Was this issue solved?</p>

ahmedbouida: <p>I’ve a nvidia 3060ti with Driver Version: 512.78.01 and CUDA 12.0.<br>
Has anyone solved the problem of torch GPU detection?</p>
Future of torchscript

mmg: <p>I read somewhere (don’t remember, a slack channel maybe) that torchscript is not maintained anymore and model.compile is the way forward. Is there some truth to this?<br>
P.S. I only use torchscript when I want to generate MAR files for torchserve…</p>

ptrblck: <p><a href="https://discuss.pytorch.org/t/pytorch-2-and-the-c-interface/168034">This thread</a> with some linked posts might be helpful.</p>
Isend and irecv happen on different streams with NCCL

ralzq01: <p>Hi, I’m using <code>torch.distributed.isend</code> and <code>torch.distributed.irecv</code> for asynchronized communication with NCCL backend.</p>
<p>I assumed <code>isend</code> and <code>irecv</code> will happen on a same cuda stream without specifying a nccl group:</p>
<pre data-code-wrap="python"><code class="lang-python">for _ in range(16):
    work = torch.distributed.irecv(input, prev_rank)
    ...
    work.wait()
    output = sub_model(input)
    torch.distributed.isend(output, next_rank)
    ...
</code></pre>
<p>But I found they actually happen on two streams and got nsys profiled results like this:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/7/e7f6a87ae31b5039d3f50f26651a4f6f9bf8b129.png" data-download-href="https://discuss.pytorch.org/uploads/default/e7f6a87ae31b5039d3f50f26651a4f6f9bf8b129" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/7/e7f6a87ae31b5039d3f50f26651a4f6f9bf8b129.png" alt="image" data-base62-sha1="x62L0MRphqlg6NVbByXoHuI0Jjr" width="690" height="136" data-dominant-color="CFD7EA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1013×201 16.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Note this is one of the devices, where default stream 7 is performing computation and red circles are performing send and recv kernel. Is this expected?</p>
<p>Docker image: <a href="http://nvcr.io/nvidia/pytorch:21.10-py3" rel="noopener nofollow ugc">nvcr.io/nvidia/pytorch:21.10-py3</a><br>
PyTorch version: 1.10.0a0+0aef44c</p>
Torch.autograd.grad makes params None

kl_divergence: <p>My code is the following:</p>
<pre><code class="lang-auto">grad_0 = list(torch.autograd.grad(losses[0], shared_parameters, retain_graph=True)
grad_0 = torch.cat([torch.flatten(grad) for grad in grad_0])

grad_1 = list(torch.autograd.grad(losses[1], shared_parameters, retain_graph=True)
grad_1 = torch.cat([torch.flatten(grad) for grad in grad_1])
</code></pre>
<p><code>losses</code> is a list containing losses from two tasks. I get the error in second line saying <code>ValueError: grad requires non-empty inputs</code>. I try printing <code>shared_parameters</code> after first line, and it doesn’t print. What happens to <code>shared_parameters</code> after using it with autograd ? I tried using deep copy, I get the error saying <code>cannot pickle generator object</code>.</p>
<p>I am not sure why does this happen ? How can I re-use <code>shared_parameters</code> ? I think passing a deep copy may make code behave differently.</p>

ptrblck: <p>Your <code>generator</code> is exhausted so you would either need to recreate it or use e.g. a <code>list</code> instead:</p>
<pre><code class="lang-python">model = models.resnet50()

shared_parameters = list(model.parameters())
losses = [model(torch.randn(1, 3, 224, 224)).mean(), model(torch.randn(1, 3, 224, 224)).mean()]

grad_0 = list(torch.autograd.grad(losses[0], shared_parameters, retain_graph=True))
grad_0 = torch.cat([torch.flatten(grad) for grad in grad_0])

shared_parameters = model.parameters()
grad_1 = list(torch.autograd.grad(losses[1], shared_parameters, retain_graph=True))
grad_1 = torch.cat([torch.flatten(grad) for grad in grad_1])

# minimal code snippet to reproduce
shared_parameters = model.parameters()
for p in shared_parameters:
    print(p.sum())
    
# empty
for p in shared_parameters:
    print(p.sum())
</code></pre>

kl_divergence: <p>Thank you very much. PyTorch community is lucky to have you.</p>

bicyclezeng: <p>Thank you a lot!!!<br>
I have struggled with problem for a whole day!</p>
Bytearray decode in GPU

pvalois: <p>Hello, it’s my first time here. I’m fairly new to the library yet and I wonder if I can run an operation in the GPU.</p>
<p>I have a matrix of paths, which come in bytearray format. I wanted to convert them to strings in the GPU.</p>
<p>Is there a simple way to run the following method in pytorch? Or just running it without having to copy to the CPU?</p>
<p><code>strings = np.apply_along_axis(lambda x: x.tobytes().decode(), 1, input.cpu())</code></p>
<p>where input: torch.Tensor in a matrix shape with dtype uint8.</p>
<p>I tried adapting this snippet to CuPy, but it raises an error saying string cupy arrays are not yet implemented.</p>
Torchmultimodal-nightly not working with cuda

maliozer: <p>After installation steps on <a href="https://github.com/facebookresearch/multimodal" rel="noopener nofollow ugc">https://github.com/facebookresearch/multimodal</a> cuda is not available issue occured as following;</p>
<pre><code class="lang-auto">python -c 'import torch; print(torch.__version__);print(torch.cuda.is_available())'
2.0.0.dev20230224
False
</code></pre>
<p>I confused about torchmultimodal installation;</p>
<p>In this tutorial (<a href="https://github.com/facebookresearch/multimodal" class="inline-onebox" rel="noopener nofollow ugc">GitHub - facebookresearch/multimodal: TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.</a>) they said on preq. step 2<br>
Install pytorch, torchvision, and torchtext. but pytorch nightly version is installed</p>
<pre><code class="lang-auto">conda install pytorch torchvision torchtext pytorch-cuda=\&lt;cuda_version\&gt; -c pytorch-nightly -c nvidia
</code></pre>
<p>Do we use torchmultimodal with not nightly version of pytorch?<br>
and do we only have nightly version of torchmultimodal ?</p>
<pre><code class="lang-auto">python -m pip install torchmultimodal-nightly
</code></pre>
<p>Regards.</p>
How to support customized extensions on MPS?

lmxyy: <p>Hi, I have my customized CPU and CUDA kernel, but I want to run them on my MacBook Pro GPU. I wonder if there are some tutorials to write the customized kernel on MPS backend, especially how to load the customized op in PyTorch?</p>

albanD: <p>Hi,<br>
I don’t think we have a tutorial for that yet no.<br>
You should be able to write custom Metal kernels though within PyTorch.<br>
I’m not sure if the cpp_extension module works fine for these though. You should feel free to try it out and open issues on github if you encounter any problem with it!</p>

ryanbty: <p>Hello, did you find a solution ?</p>
LSTM dropout causes warning

seer_mer: <p>When using LSTM with dropout, below warning is printed:</p>
<pre><code class="lang-auto">[W CUDAGuardImpl.h:46] Warning: CUDA warning: driver shutting down (function uncheckedGetDevice)
[W CUDAGuardImpl.h:62] Warning: CUDA warning: driver shutting down (function uncheckedSetDevice)
</code></pre>
<p>minimum code:</p>
<pre><code class="lang-auto">import torch
from torch import nn


class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.embedding = nn.Embedding(1000, 8)
        self.lstm = nn.LSTM(8, 8, 2,
                            dropout=0.2, batch_first=True)

    def forward(self, x):
        x = self.embedding(x)
        x = self.lstm(x)
        return x


inp = torch.randint(0, 1000, (32, 512))
inp = inp.to('cuda')
model = Net().to('cuda')
model(inp)
</code></pre>
<p>when dropout is not used, no warning will be printed.<br>
version/platform:<br>
Windows11 (doesn’t even happen on WSL on the same machine)<br>
Python 3.10<br>
torch version: 1.13.1+cu117 and 1.13.1+cu116 (both warns)</p>
<p>The above warning does not seem to affect anything (code runs, exits normally, and speed is okay).<br>
I searched online, and it appears that the problem is caused by misusing multiprocessing, but clearly, I am not using multiprocessing, so I guess it is possible that something in LSTM uses multiprocessing. Why is this warning? Are there any impacts on my program? How do I solve this, or is this a bug?</p>

ptrblck: <p>I cannot reproduce this issue on Linux with <code>1.13.1+cu117</code> and the current nightly release, so it could be Windows-specific.</p>

seer_mer: <p>Yes, it is very likely windows specific, because even WSL (Windows Subsystem for Linux) cannot reproduce the problem. I just wonder what does this warning mean? How can I potentially fix it? Thanks in advance!</p>

ptrblck: <p>I would probably add the <code>if-clause</code> guard as described <a href="https://pytorch.org/docs/stable/notes/windows.html#usage-multiprocessing">here</a> as it’s often needed on Windows.</p>

seer_mer: <p>Thanks. But I tried and that does not work. In fact I do have if main in my actual program, just forgot to include it in the sample code.</p>
Error while loading torchscript module in Go

luxiant: <p>Hi, I’m trying to load torch model in Go code. I first tried to save torchscript in python.</p>
<pre><code class="lang-auto">...
traced_model = torch.jit.trace(model, [input_ids, attention_mask], strict = False)
traced_model.save(PATH + 'sentiment_model_jit.pt')
</code></pre>
<p>And when I try to load the saved torchscript in Go code, the console shows this error.</p>
<pre><code class="lang-auto">error in loading torch model: Libtorch API Error: version_ &lt;= kMaxSupportedFileFormatVersionINTERNAL ASSERT FAILED at "../caffe2/serialize/inline_container.cc":147, please report a bug to PyTorch. Attempted to read a PyTorch file with version 10, but the maximum supported version for reading is 9. Your PyTorch installation may be too old.
</code></pre>
<p>I searched the forum do deal with this, and found that when saving the torch model to load it in python, this error may be solved by adding an argument “_use_new_zipfile_serialization=False” when doing “torch.save”.</p>
<p>How about when saving torchscript? I found out that there is no such argument when saving torchscript. Is there any other way?</p>
Is it possible to catch if .to(device) doesn't have enough memory?

perf: <p>If I have created a tensor on the CPU, and I want to move it to the GPU.</p>
<p>Is there a way to check whether there’s enough memory available on the GPU?</p>

ptrblck: <p>No, I don’t think so unless you disable our caching mechanism and use <code>cudaMemGetInfo</code> from <code>torch.cuda.cudart()</code>.<br>
In the default setup the caching allocator will be used to be able to reuse already allocated device memory without using the slow and synchronizing <code>cudaMalloc/cudaFree</code> calls excessively.<br>
If a memory allocation fails, the allocator will try to free the cache and reallocate a larger block. Due to this behavior I don’t think there exist a clean way of checking the free memory without allowing the caching allocator to allocate the tensor using different approaches (by reusing the cache, allocating new memory, or freeing the cache before trying to allocate the memory).</p>