akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
RuntimeError: CUDA error: misaligned address Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions

ArshadIram: <p>Hi, I’m having the issue. I’m trying to use multiple webcams using different threads it works for about 1 minute without lag and with detections on GPU but then crashes with the exact same error. I then have to restart the entire development environment to run it again as it won’t let me run without a restart of the application.<br>
“RuntimeError: CUDA error: misaligned address<br>
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.<br>
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.<br>
Compile with <code>TORCH_USE_CUDA_DSA</code> to enable device-side assertions.”</p>
<p>My assumptions were that it could be a memory issue but when running and monitoring the GPU and CPU performance and ram everything seems fine. I’ve read a bit online and the same error has crept up when using multiple cams in terms of classes it expects a certain value but receives double the amount of classes which throws the error. That’s my assumption but any help on this issue would be greatly appreciated.</p>
<p>this is the code of my threading with yolo prediction method:<br>
class camThread(threading.Thread):<br>
def <strong>init</strong>(self,  camID):<br>
threading.Thread.<strong>init</strong>(self,daemon=True)<br>
<span class="hashtag-raw">#self</span>.previewName = previewName<br>
self.camID = camID<br>
def run(self):<br>
<span class="hashtag-raw">#print</span>("Starting " + self.previewName)<br>
camPreview(self.camID)</p>
<p>def camPreview(camID):</p>
<pre><code>infer= YOLO("best.pt")
results=infer.predict(source=camID, verbose=False,show=True)
#with open("output.txt", "w") as fo:
  #for r in results:
    #fo.write(str(r))

  thread1 = camThread(0)
  thread2 = camThread(1)
  thread1.start()
  thread2.start()
  print("Active threads", threading.activeCount())
</code></pre>

ptrblck: <p>Could you run your workload via <code>compute-sanitizer python script.py args</code> and see if any issues are reported? Also, are you able to reproduce the issue without any cam usage in pure PyTorch?</p>

ArshadIram: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>, many thanks for your quick response. This issue was related to memory, so it is not directly relevant to pytorch.</p>
Dictionary Hunting - Torch Script Conversion

johnorford: <p>I am porting a model to use Torch Script - hoping to improve inference performance.</p>
<p>Now these errors are cropping up:</p>
<blockquote>
<p>The operator ‘aten::dict’ ONNX upset version 14 is not supported</p>
</blockquote>
<p>Up until now the errors have been informative, but is there anyway to hunt down where the offensive dicts are?</p>
Torchaudio.compliance.kaldi feature extraction problem

Alva-2020: <p>import torchaudio.compliance.kaldi as kaldi<br>
import torchaudio</p>
<p>waveform, _ =torchaudio.load(“xxxx/SSB0379-0417.wav”)<br>
mat = kaldi.fbank(<br>
waveform,<br>
num_mel_bins=80,<br>
frame_length=25,<br>
frame_shift=10,<br>
dither=0.0,<br>
energy_floor=0.0,</p>
<pre><code>            sample_frequency=16000)
</code></pre>
<p>mat1 = kaldi.fbank(<br>
waveform,<br>
num_mel_bins=80,<br>
frame_length=25,<br>
frame_shift=10,<br>
dither=0.0,<br>
energy_floor=0.0,</p>
<pre><code>            sample_frequency=16000)
</code></pre>
<p>print(mat1)<br>
print(mat)<br>
~<br>
mat1 and mat are slightly different due to the effect of dither. Then will I set dither=0.1 in the test phase will affect the results? My system is an ASR system. Thanks for you help.</p>
Textured_simple.pth file?

poseestimationuser: <p>Hello there,</p>
<p>I want to use PoseCNN with my own dataset that is inspired by the YCB-dataset. I saw in the model folder of the dataset, that every model has a .xyz (got that one for my model aswell), a .sdf file (got that aswell) and a file name textured_simple.pth. Since I only know .pth files as pytorch network model files, what has that to do with the object models? How can I create one for my model aswell? I linked you my onedrive for one of the YCB-Data objects, so you can get a clue what I am talking about.</p>
<p><a href="https://1drv.ms/f/s!Aut5v_xex_Ighd0i5OUNKD0nJI2lkw?e=Ewq0le" rel="noopener nofollow ugc">OneDrive Object YCB</a></p>
<p>Thank you very much! Sorry if this question is silly, I am just starting with AI.</p>
Achieve the same result with torch.nn.Conv1d using torch.nn.Conv2d

wadewang: <p>I tried to achieve the same result with torch.nn.Conv1d using torch.nn.Conv2d for fun, I think conv1d is a subclass of conv2d, just make the input tensor height as 1 and kernel height as 1, but the test code tells me their results are not same:</p>
<pre><code class="lang-auto">import torch
input_for_conv2d = torch.randn(8, 6, 1, 100)
input_for_conv1d = input_for_conv2d.reshape(8, 6, 100)
conv2d = torch.nn.Conv2d(in_channels=6, out_channels=10, kernel_size=(1, 3), padding=(0, 1))
conv1d = torch.nn.Conv1d(in_channels=6, out_channels=10, kernel_size=3, padding=1)
# assign weights of conv2d to conv1d
conv1d.weight.data = conv2d.weight.data.reshape(10, 6, 3)
output_conv2d = conv2d(input_for_conv2d)
print(f"==&gt;&gt; output_conv2d.shape: {output_conv2d.shape}")
output_conv1d = conv1d(input_for_conv1d)
print(f"==&gt;&gt; output_conv1d.shape: {output_conv1d.shape}")
sum_conv2d = torch.sum(output_conv2d)
print(f"==&gt;&gt; sum_conv2d: {sum_conv2d}")
sum_conv1d = torch.sum(output_conv1d)
print(f"==&gt;&gt; sum_conv1d: {sum_conv1d}")
# The sum result are the the same
</code></pre>
<p>Can I achieve the same result with torch.nn.Conv1d using torch.nn.Conv2d ? If yes, how to do that ?</p>

ptrblck: <p>Set the <code>bias</code> to <code>False</code> or copy it too and it should work.</p>
Pytorch c++ Grad-CAM

MSA: <p>Hi,<br>
Does someone already try to do Grad-CAM (heatmap) with pytorch c++?</p>
<p>For now, I did the training in python and load the model with:</p>
<p><code> m_module = torch::jit::load(pathscript);</code></p>
<p>I’m pretty sure that I can’t use jit trace to do Grad-CAM and I will need to redefine my CNN in c++ directly.</p>
<pre><code class="lang-auto">struct Net: torch::nn::Module
</code></pre>
<p>Can someone confirm?</p>
<p>Thanks</p>

mihow: <p>I am very curious about this as well. Any luck?</p>

Francesco_Biasion: <p>Are there any news on this thread? it is still current</p>
Transformer cannot predict next number in linear sequence

Goldname: <p>I am experimenting with the trasnformer to see if it can predict the next number in a linear sequence. For example, given this training data (1,2,3,4…100) I expect 101 as output. However, the loss is not decreasing as I train the model for some reason. Does anyone know if I am doing something fundamentally wrong here?</p>
<p>My transformer has 1 embed dim, 1 head, and 10 encoders.</p>
<pre><code class="lang-auto">import torch
from torch import nn
from torch.utils.tensorboard import SummaryWriter
from datetime import datetime

import torch.optim as optim
import random

device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
now = datetime.now()
logdir = "runs/" + now.strftime("%Y%m%d-%H%M%S") + "/"
writer = SummaryWriter(logdir)


class MyModel(nn.Module):
    def __init__(self, seq_len):
        super().__init__()
        self.lin = nn.Linear(seq_len,1)
        self.tgt = torch.rand(seq_len, 1).to(device)
        self.trans = nn.Transformer(nhead=1, num_encoder_layers=10, d_model=1)

    def forward(self, x):
        y = self.trans(x, self.tgt)
        y = torch.transpose(y,0,1)
        out = self.lin(y)
        return out


model = MyModel(100)
model.to(device)
model.eval()

criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.01)

running_loss = 0.0

for epoch in range(4000):  # loop over the dataset multiple times
    idx = torch.randint(100, (1,)).item() #Get random number
    training = torch.range(idx, 100+idx, 1).to(device) #Get consecutive numbers starting from randidx
    training = torch.unsqueeze(training, 1)
    labels = torch.tensor([101+idx]).to(device) #Label is number after training sequence
    optimizer.zero_grad()
    
    outputs = model(training)
    outputs = torch.transpose(outputs, 0, 1)
    loss = torch.norm(outputs - labels)
    loss.backward()
    optimizer.step()

    # print statistics
    running_loss += loss.item()
    if epoch % 50 == 0:
        print(f'[{epoch + 1}] loss: {running_loss / 50:.3f}')
        writer.add_scalar('training loss',
                        running_loss / 50,
                        epoch)
        writer.add_scalar('prediction',
                        outputs,
                        epoch)
        writer.add_scalar('labels',
                        labels,
                        epoch)

        running_loss = 0.0

print('Finished Training')
</code></pre>
Does tensor.numpy() allocate new memory?

Sam_Lerman: <p>I have a shared tensor. Does converting it to numpy allocate new memory or does the buffer remain the same?</p>
<p><code>my_tensor = my_tensor.numpy()</code></p>

ptrblck: <p>The numpy array will share the same memory as seen here:</p>
<pre><code class="lang-python">x = torch.tensor([1.])
print(x)
# tensor([1.])

y = x.numpy()
print(y)
# [1.]

y[0] = 2.0

print(x, y)
# tensor([2.]) [2.]
</code></pre>
<p>but I’m unsure what “I have a shared tensor” means in this context.</p>
ImageNet DataLoader "too many open files" error

Sam_Lerman: <p>I create an instance of a torchvision ImageNet validation-set DataLoader called <code>imagenet_val</code>.</p>
<p>I then try to make an iterator for it: <code>iter(imagenet_val)</code>.</p>
<p>And get this trace:</p>
<pre><code class="lang-auto">File "/u/slerman/u1/conda/envs/Sam/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 441, in __iter__
    return self._get_iterator()
           ^^^^^^^^^^^^^^^^^^^^
  File "/u/slerman/u1/conda/envs/Sam/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 388, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u/slerman/u1/conda/envs/Sam/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 1042, in __init__
    w.start()
  File "/u/slerman/u1/conda/envs/Sam/lib/python3.11/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
                  ^^^^^^^^^^^^^^^^^
  File "/u/slerman/u1/conda/envs/Sam/lib/python3.11/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u/slerman/u1/conda/envs/Sam/lib/python3.11/multiprocessing/context.py", line 288, in _Popen
    return Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^
  File "/u/slerman/u1/conda/envs/Sam/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/u/slerman/u1/conda/envs/Sam/lib/python3.11/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/u/slerman/u1/conda/envs/Sam/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 58, in _launch
    self.pid = util.spawnv_passfds(spawn.get_executable(),
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u/slerman/u1/conda/envs/Sam/lib/python3.11/multiprocessing/util.py", line 451, in spawnv_passfds
    errpipe_read, errpipe_write = os.pipe()
                                  ^^^^^^^^^
OSError: [Errno 24] Too many open files
</code></pre>

ptrblck: <p>Check how many file handles can be used via <code>ulimit -n</code> and increase this value if needed.</p>

Sam_Lerman: <p>Do you know if the imagenet validation set is expected to have images of shape <code>[3, 500, 375]</code> and others <code>[3, 375, 500]</code>?</p>

ptrblck: <p>ImageNet samples have a variable spatial shape, if I’m not mistaken, and are thus resized to e.g. <code>224x224</code> in common architectures.</p>
Cannot import pytorch afther installation

myabakhova: <p>Hello,<br>
I decided to install pytorch on my desktop. For this I looked up pytorch site and ran the following recommended command in my Anaconda prompt:</p>
<pre><code class="lang-auto">conda install pytorch torchvision torchaudio cpuonly -c pytorch
</code></pre>
<p>All went fine.<br>
I followed it by <code>conda update --all</code> and <code>conda clean --all</code>.<br>
Then I tried in my jupyter notebook:</p>
<pre><code class="lang-auto">import tensorflow as tf
import torch
</code></pre>
<p>And I got an error message:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_15588\2142308364.py in &lt;module&gt;
      1 import tensorflow as tf
      2 import pandas as pd
----&gt; 3 import torch

~\miniconda3\lib\site-packages\torch\__init__.py in &lt;module&gt;
    126                 err = ctypes.WinError(last_error)
    127                 err.strerror += f' Error loading "{dll}" or one of its dependencies.'
--&gt; 128                 raise err
    129             elif res is not None:
    130                 is_loaded = True

OSError: [WinError 182] The operating system cannot run %1. Error loading "C:\Users\mathe\miniconda3\lib\site-packages\torch\lib\shm.dll" or one of its dependencies.
</code></pre>
<p>I’ve tried a number of things since then. I updated conda, re-install tensoflow and pytorch and updated everything again, tried import in Spyder, just in case, and re-start my desktop. I still get the same mistake. The installation-import works on my Ubuntu 18 laptop, so I believe that it is Windows problem.<br>
I use conda for consistency. Quite often I work with students and clients, and they prefer conda.<br>
This is a base conda environment and not my created environment.<br>
Best regards,<br>
Mya</p>

myabakhova: <p>Hello,<br>
thanks for your reply. What tensors do you mean? Because there are no tensors mentioned in the error message.</p>

ptrblck: <p>I would ignore the previous message as it seems to come from a bot posting unrelated questions/answers and I’ve deleted it.</p>
<p>For your issue: could you check what the missing dependency in the <code>dll</code> is?<br>
I’m not deeply familiar with Windows, but maybe <code>dumpbin /dependents PATH...\shm.dll</code> could help.</p>

Niraj: <p><a class="mention" href="/u/myabakhova">@myabakhova</a>, I’m using a windows version and conda environment, facing a similar issue. Could you please suggest me on how you got it resolved?</p>

myabakhova: <p>Hello Niraj,<br>
nothing fancy, I rebooted my computer and the problem was resolved.</p>
Define Num_Workers with Distributed Data Parralel

jojupiter: <p>Hi,<br>
I’m a pytorch beginner. I want to distribute data on several GPUs with Distributed Data Parralel.<br>
I run this code on a 4 GPU node. My wordlsize is 4.<br>
My question is about the validation batch_size and the num_worker (validation and training).</p>
<p>Will each process have a validation batch_size = 4/worldsize and a num_worker = 4/world_size?</p>
<p>Will each process have a training num_worker equal to 16/world_size?</p>
<pre><code class="lang-auto">

def get_dataset(data_folder,train_images,val_images, CROP_SIZE, UPSCALE_FACTOR):
    world_size = dist.get_world_size()
    
    
    # Create dataset training and validation
    train_set = TrainDatasetFromFolder(data_folder, crop_size=CROP_SIZE, upscale_factor=UPSCALE_FACTOR,
                                       image_list=train_images)
    val_set = ValDatasetFromFolder(data_folder, upscale_factor=UPSCALE_FACTOR, image_list=val_images)

    
    
    
    train_sampler = DistributedSampler(train_set,num_replicas=world_size)
    val_sampler = DistributedSampler(val_set,num_replicas=world_size)
    batch_size = int(80/ float(world_size))
    print(world_size, batch_size)
    train_loader = DataLoader(
        dataset=train_set,
        sampler=train_sampler,
        batch_size=batch_size,
        num_workers=16,
        pin_memory=True,
        
    )
    val_loader = DataLoader(
        dataset=val_set,
        sampler=val_sampler,
        batch_size=4,
        num_workers=4,
        
    )

    return train_loader, val_loader, batch_size
</code></pre>

kumpera: <p>the parameters for DataLoader are not affected by world_size, it’s up to you to control the interaction between them.</p>
Error training multi-GPU with DDP

dothuan: <p>I used mul-ti GPUs for training a model, and it can detect my 3 GPUs. However, after load the pre-trained model, It got some issues with DDP as the image below, and then my jupyter notebook was automatically disconnected.<br>
Seems like the issues came from pytorch. I have upgrade pytorch o the lastest version but it’s still not work.<br>
Hope you guys can help me address this issue. Thank you.<br>
Note: I’m still able to train with single GPU.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/e/3e6da3042b2b2bf3579e3d3a4bdf58e5e0af9efa.png" data-download-href="https://discuss.pytorch.org/uploads/default/3e6da3042b2b2bf3579e3d3a4bdf58e5e0af9efa" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/e/3e6da3042b2b2bf3579e3d3a4bdf58e5e0af9efa.png" alt="image" data-base62-sha1="8UguVJCQLUexVgBesOPhZp31CaK" width="690" height="346" data-dominant-color="6C6161"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1486×747 75.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

kumpera: <p>Hi,</p>
<p>The error you’re reporting is just for the supervisor process detecting an issue on one of the child processes.</p>
<p>You need to look at the individual output of each rank to understand what happened.</p>
Is NVIDIA driver already included CUDA and CUDA toolkit?

chester: <p>I installed pytorch to my conda environment (windoes 11)</p>
<blockquote>
<p>conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia</p>
</blockquote>
<p>I updated my driver using GeForce Experience before, but I didn’t install CUDA toolkit or cuDNN. When I run nvidia-smi, it shows that my CUDA version is 12.2</p>
<p>Also, everything working well, it used my GPU for training</p>
<blockquote>
<p>print(torch.cuda.is_available())<br>
True<br>
print(torch.cuda.device_count())<br>
1<br>
print(torch.cuda.current_device())<br>
0<br>
print(torch.cuda.device(0))<br>
&lt; torch.cuda.device object at xxx &gt;</p>
</blockquote>
<p>I just confused…<br>
(1) Is installing CUDA toolkit to local computer necessary for pytorch?<br>
(2) Does NVIDIA driver include CUDA? If not, where does the CUDA come from?</p>
<p>Thank you very much</p>

ptrblck: <ol>
<li>No, since the PyTorch binaries ship with their own CUDA dependencies (e.g. cuBLAS, cuDNN, NCCL, etc.).</li>
<li>No, it does not. The PyTorch binaries ship with all needed CUDA dependencies and a simple <code>pip install torch</code> will pull them from PyPI.</li>
</ol>
Add a new layer in trained model

zoythum: <p>As the title suggests I need to add a layer in the middle of a model which has already been trained and for which I only have a .pt file without any knowledge of the architecture. Since I don’t have the code used to build the model I cannot simply modify the structure by adding my layer and copying the new layer wherever I need it.</p>
<p>I tried the following code</p>
<pre><code class="lang-auto">model = torch.load('pretrained_model.pt')
new_layer = customLayer(...)
model.layer1 = nn.Sequential(model.layer1, new_layer)
</code></pre>
<p>But I get an error</p>
<pre><code class="lang-auto">AttributeError: 'collections.OrderedDict' object has no attribute 'layer1'
</code></pre>
<p>that I don’t know how to fix. Does anyone know the correct way of adding a layer between any two given layers of a trained model?</p>

ptrblck: <aside class="quote no-group" data-username="zoythum" data-post="1" data-topic="184390">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zoythum/48/55406_2.png" class="avatar"> zoythum:</div>
<blockquote>
<p>for which I only have a .pt file without any knowledge of the architecture. Since I don’t have the code used to build the model I cannot simply modify the structure</p>
</blockquote>
</aside>
<p>This sounds strange, since you would either store the <code>state_dict</code> only (recommended way) which would mean that you would also need to store the model definition explicitly to be able to even restore the model, or you would store the model directly (not recommended), which would then pickle the source files. Could you describe what <code>pretrained_model.pt</code> contains?</p>

zoythum: <p>I know it sounds strange but the process I am working on starts from a ONNX file that I then convert to torch. For this reason I don’t have the code used to build the model as usual but only the saved model.<br>
But now that you pointed it out it seems like my idea is not working since I can’t perform inference if I only have the .pt file. Unfortunate but I’ll find a different way, thanks</p>

jiversivers: <p>This might sound like a dumb question, but do you need to train the new model as a whole, or just the new layer? If you need to train the whole model, it sounds like you are too confined to do that, but if you need to train the new layer, you ought to be able to just use the output of the saved model as input into a new 1-layer model and train that. Depending on your latent output and final targets, this might be trivial, but given the situation you’re in, I’m not sure there’s much else you can do.</p>
Is pytorch on linux compatible with nvidia rtx 4090?

yehonatan: <p>is pytorch on linux ubuntu compatible with nvidia rtx 4090 gpu? is the version stable?</p>

nairbv: <p>Yes. That’s what I use on my home desktop and it works great.</p>

yehonatan: <p>Thanks for answering!<br>
Can you please elaborate on your os and also on the machine you use ?<br>
Inam just about to buy a laptop with this GPU and need to make sure it works smoothly</p>
How to keep update with new torch methods and tips

Abdelkareem_Elkhateb: <p>Are there any sources, Twitter accounts, or newsletters that talk about Torch internals, new methods, or best practices, etc.?</p>

nairbv: <p>The PyTorch dev podcast is pretty good for learning about internals: <a href="https://pytorch-dev-podcast.simplecast.com/" rel="noopener nofollow ugc">https://pytorch-dev-podcast.simplecast.com/</a></p>
Process hangs when I call torch.cuda.is_available()

Hollmann_Kevin: <p>Hello,</p>
<p>I  am a beginner and I have a problem when I try to run a check to determine if I should use CUDA ( torch.cuda.is_available()) my process hangs.<br>
I’m on  Linux.<br>
My version of torch are:</p>
<pre><code class="lang-auto">pip list | grep torch
torch                        2.0.1+cu117
torchaudio                   2.0.2+cu117
torchvision                  0.15.2+cu117
</code></pre>
<p>.<br>
nvidia-smi:</p>
<pre><code class="lang-auto">+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 530.30.02              Driver Version: 530.30.02    CUDA Version: 12.1     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                  Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf            Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  Quadro P400                     On | 00000000:73:00.0 Off |                  N/A |
| 34%   39C    P8               N/A /  30W|     56MiB /  2048MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      1006      G   /usr/lib/xorg/Xorg                           54MiB |
+---------------------------------------------------------------------------------------+
</code></pre>
<p>I have the version <strong>12.1.0</strong> of CUDA.</p>
<p>When I call torch.cuda.version it gives me <strong>11.7</strong>.</p>
<p>Does someone knows what’s going on?</p>

ptrblck: <p>Do you see any active process in <code>top</code> and could you attach a <code>gdb</code> session to the hanging process to create a stacktrace?</p>

Hollmann_Kevin: <p>There is indeed an active process in top taking a lot of CPU % but If I try to attach it with gdb, the gdb process hangs. I think the process is waiting for I/O access and can’t be kill</p>
How to initialize hidden state in GRU

kaiq: <p>I set the initial hidden state in the forward function. Does it work?</p>
<pre><code class="lang-auto">class MV_GRU(torch.nn.Module):
    def __init__(self, n_features, seq_length, num_hiddens, hidden_layers):
        super(MV_GRU, self).__init__()
        self.n_features = n_features
        self.seq_len = seq_length
        self.n_hidden = num_hiddens  # number of hidden states
        self.n_layers = hidden_layers  # number of GRU layers (stacked)


        self.l_gru = torch.nn.GRU(input_size=n_features,
                                  hidden_size=self.n_hidden,
                                  num_layers=self.n_layers,
                                  batch_first=True)
        self.l_linear = torch.nn.Linear(self.n_hidden * self.seq_len, 1)

    def forward(self, x):
        batch_size, seq_len, _ = x.size()
        self.hidden = torch.zeros(self.n_layers, batch_size, self.n_hidden).to(device)
        gru_out, self.hidden = self.l_gru(x, self.hidden)
        x = gru_out.contiguous().view(batch_size, -1)
        return self.l_linear(x)
</code></pre>

ptrblck: <p>Yes, you can initialize the hidden state in the <code>forward</code> method, but note that you have created a recursion, which might yield errors. Currently the computation graph will be attached to <code>self.hidden</code> so that the <code>backward</code> call might try to backpropagate through multiple iterations. This could be a valid use case, but you should check if this fits your use case or if you want to <code>detach()</code> the hidden state in each <code>forward</code> pass.</p>
The problem of not getting correct accuracy in patched data

mahsa_samkhaniani: <p>Their largest dimension is 1226 *  600 * 224, which is the number of my bands. To convert these photos to 256<em>256</em>224 dimensions, I converted my photos to 141 photos with 256 * 256 * 224 dimensions using the patches command. For train-dataset and test-dataset, I used the Sklearn library resulting in 98 trains and 43 tests.<br>
However, during the subsequent stages of my algorithm, I’m encountering varying accuracies of 5-digit number for each iteration. Alternatively, by including the following command in my code snippet, the accuracy remains the same throughout all iterations, but it is expressed as a percentage:</p>
<p>running_corrects += torch.sum(preds == labels.data).float() / labels.nelement()</p>
<p>I suspect that the issue may lie with the data, but I am unable to pinpoint a solution. Could you please suggest an alternative approach to patching the data? also, I would appreciate it if you could confirm whether my current patching method is correct.</p>

eqy: <p>This isn’t possible to debug without runnable code (e.g., your accuracy calculation). You can check the correctness or your accuracy calculation without a model–simply generate random predictions and verify that the result is reasonable for random guessing.</p>

mahsa_samkhaniani: <p>I did not understand your explanation, can you help me more</p>

eqy: <p>To verify the correctness of your accuracy calculation, you can try generating random predictions (or even always predict the same class) to check if the reported value makes sense in that case.</p>
Why I can't download pytorch

Yihong_Theis: <p>I am trying to download pytorch GPU version but none of them let me download.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/0/805d0c931dddcb4311e9d7236b2c64e813bed2b2.png" data-download-href="https://discuss.pytorch.org/uploads/default/805d0c931dddcb4311e9d7236b2c64e813bed2b2" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/8/0/805d0c931dddcb4311e9d7236b2c64e813bed2b2.png" alt="image" data-base62-sha1="ijytkm1HrIIuxMJuUqsBIQVTKts" width="690" height="127" data-dominant-color="1F1917"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1455×269 21.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
How I can solve it?</p>

ptrblck: <p>I don’t know what’s causing your environment to fail finding the wheel, but you can download <code>torchvision==0.14.1+cu116</code> for Python 3.10 <a href="https://download.pytorch.org/whl/cu116/torchvision-0.14.1%2Bcu116-cp310-cp310-linux_x86_64.whl">here</a>. Change the <code>310</code> tag to <code>39</code> for Python 3.9 etc.</p>

tom: <p>I don’t think Python 3.11 is supported by PyTorch 1.13.1 (not sure why the whl for PyTorch is there), so you might try Python3.10 or PyTorch nightly…</p>
<p>Best regards</p>
<p>Thomas</p>
Why the loss_scale getting smaller and smaller?

asr-pub: <p>Hello, now I am pretraining a HuBERT model, I have found that when the loss getting smaller, but increasingly frequent occurrences of gradient overflow, leading to a decreasing loss_scale.</p>
<p>My understanding is that when the loss becomes smaller and smaller, it indicates that the model has converged. So why would gradient overflow occur more frequently?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/e/2e1bf323e187b8e1e373caeeab78f8004e4763fd.png" data-download-href="https://discuss.pytorch.org/uploads/default/2e1bf323e187b8e1e373caeeab78f8004e4763fd" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/e/2e1bf323e187b8e1e373caeeab78f8004e4763fd_2_690x244.png" alt="image" data-base62-sha1="6zTQr1cORoPSJ5poY0ILRvJkhjf" width="690" height="244" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/e/2e1bf323e187b8e1e373caeeab78f8004e4763fd_2_690x244.png, https://discuss.pytorch.org/uploads/default/original/3X/2/e/2e1bf323e187b8e1e373caeeab78f8004e4763fd.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/e/2e1bf323e187b8e1e373caeeab78f8004e4763fd.png 2x" data-dominant-color="F8F7F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">751×266 25.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>You could unscale the gradients manually to inspect them to see which ones are overflowing. Unscaling Infs or NaNs will of course keep these invalid values but it should give you an idea where in the model the gradients start to overflow.</p>
How to compile PyTorch on linux VM but use it in container

Zhiqiang_Kou: <p>Hi, I was trying to use PyTorch in container with 2 Tesla K80 GPU card configured. However I got error message likes: “RuntimeError: CUDA Error: no kernel image is available for execution on the device”</p>
<p>By searching I found the issue: <a href="https://github.com/pytorch/pytorch/issues/31285" class="inline-onebox" rel="noopener nofollow ugc">Cuda error: no kernel image is available for execution on the device · Issue #31285 · pytorch/pytorch · GitHub</a><br>
Which tells me the K80 GPU is too old so I have to compile PyTorch from source to support it.</p>
<p>Now I am following the procedure from here:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/pytorch/pytorch#from-source">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/pytorch/pytorch#from-source" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/01b90aed12c82da6601fc6d3099d37820d443d25ccaa7df7eeb7c40580480d18/pytorch/pytorch" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/pytorch/pytorch#from-source" target="_blank" rel="noopener nofollow ugc">GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with...</a></h3>

  <p>Tensors and Dynamic neural networks in Python with strong GPU acceleration - GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>But it seems I have to compile the PyTorch on the machine which has GPU card installed, unfortunately that is not my case.</p>
<p>I am using PyTorch in Kubernetes Container, which is Ubuntu 22.04 and has GPU card installed. However I have no root account inside the container so I am not able to compile PyTorch source code from the container. (because i am not able to run sodu command to install all required dependencies)</p>
<p>I have another RHEL VM which I can run sudo command to install all necessary dependencies. But that machine has no GPU card installed.</p>
<p>So my question is:</p>
<ul>
<li>May I compile PyTorch from source on the RHEL VM which has no GPU installed?</li>
<li>How to use the compiled PyTorch for another Ubuntu Container?</li>
</ul>
<p>Thanks a lot!</p>

ptrblck: <p>Your K80 should be supported in PyTorch using the binaries shipping with CUDA 11.x, since these binaries support all architectures between 3.7 (which is your K80) and 9.0 (if you use CUDA 11.8).</p>

Zhiqiang_Kou: <p>Thanks a lot <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>We will have a try.</p>
Using parameter list vs using module list?

bigtuna: <p>For each layer of my model, I have a list of tensors corresponding to permutations for some experiments I’m running. These permutations are randomly generated at initialization. I realized that since I stored them in a regular python list, they were not saved when I saved and loaded the model stat dict. I would like to save and load them though. Should I use nn.ParameterList() or nn.ModuleList()? These permutation tensors are not meant to be trained.</p>
<p>Thanks.</p>

ptrblck: <p>It depends what exactly you are trying to store. <code>nn.ParameterList</code> is used to save <code>nn.Parameter</code>s, i.e. trainable tensors, while <code>nn.ModuleList</code> is used to store <code>nn.Module</code>s, which themselves could contain other modules or parameters.</p>

bigtuna: <p>Thank you for the reply. The tensors I want to store are integer tensors, so when I try to save them in a parameter list I get “RuntimeError: Only Tensors of floating point and complex dtype can require gradients”.</p>
<pre><code class="lang-auto">        shuffles = []
        for head in range(heads-1):
            perm = list(range(dim)) + random.sample(range(dim, l*dim), (l-1)*dim)
            shuffles.append(torch.tensor(perm, requires_grad = False))
        self.shuffles  = nn.ParameterList([shuffle for shuffle in shuffles])
</code></pre>
<p>This is the code I’m using that’s giving me an error.</p>

ptrblck: <p>For non-floating point tensors you should register them as a buffer as seen here:</p>
<pre><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.register_buffer("my_buffer", torch.randint(0, 10, (10,)))
        
m = MyModel()
print(m.state_dict())
# OrderedDict([('my_buffer', tensor([6, 0, 8, 4, 6, 3, 0, 1, 6, 7]))])
</code></pre>
Numpy vs Pytorch discrepancy in implementing a loss function?

Loki_K: <p>I am trying to implement a loss function. But the implementation in numpy is giving me absolute zero where as in torch i am getting 0.33333… . I’ve rechecked the implementation both are same and I am using float64 for both the implementations.</p>
<p>Is this normal or is there something I should be aware of?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/c/2c5dc4a950c30f965b3d76417fc4c8f742c4b6e5.png" data-download-href="https://discuss.pytorch.org/uploads/default/2c5dc4a950c30f965b3d76417fc4c8f742c4b6e5" title="Loss function in numpy"><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/c/2c5dc4a950c30f965b3d76417fc4c8f742c4b6e5.png" alt="Loss function in numpy" data-base62-sha1="6ktUbhjiVjRD8sMTNN4RkgWuwdf" width="417" height="500" data-dominant-color="EEEEEE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Loss function in numpy</span><span class="informations">647×774 26.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/2/524aea265c183514c59761be96854561387edd27.png" data-download-href="https://discuss.pytorch.org/uploads/default/524aea265c183514c59761be96854561387edd27" title="Loss function in torch"><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/2/524aea265c183514c59761be96854561387edd27.png" alt="Loss function in torch" data-base62-sha1="bJZEHt4BNkwrrJJ8ChsvPnnb4Yn" width="338" height="500" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Loss function in torch</span><span class="informations">549×810 21.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I also wonder what impact does it have on my model. Even though predicted == actual the loss is never zero.</p>
<p>Please help me mitigate this.</p>

ptrblck: <p>I would claim the error is not “normal” and you might have a bug in your code.<br>
Could you post a minimal and executable code snippet reproducing the error, please?</p>
How to separate pytorch MNIST dataset into n subsets containing 2 classes each?

Xiaa: <p>Hello,</p>
<p>I am trying to separate pytorch MNIST dataset into n subsets each containing data from only 2 classes and then save the n subsets in differents files to load them later.</p>
<p>For example, subset 0 will contain images of class 0 and 1 (corresponding to the MNIST classes), subset 1 will contain images of class 1 and 2 and so on up to subset n.</p>
<p>If n is bigger than the number of classes several subsets will contain the same classes combination, ex: subset 10 can also contain images from class 0 and 1.</p>
<p>But every datapoint is given to only one subset, the same data point can not be in subset 0 and 10 for example.</p>
<p>I tried to write a code that does that using torch.utils.data.random_split and torch.utils.data.ConcatDataset but strangely in the files I save there always is the same data…</p>
<p>Thank you for your Help,</p>

ptrblck: <p>Double post from <a href="https://discuss.pytorch.org/t/splitting-mnist-dataset-in-n-subset-with-2-main-classes-per-subset/184137">here</a>.<br>
Did you try to debug your code and check where the wrong entries come from?</p>
How Children Should Get Started with Pytorch?

likehuiyuanai: <p>I am a child who hopes to learn how to use Pytorch, but I don’t know where to start. I hope all my friends can give me some advice。</p>

ptrblck: <p>You could start with the <a href="https://pytorch.org/tutorials/">tutorials</a> which will walk you through a lot of concepts in PyTorch. Once you are familiar with the general syntax you could then check a course, such as <a href="https://www.fast.ai/">Fast.AI</a> to learn more about neural networks and how to train them.</p>
Tensor slicing while preserving dimension

Hannibal046: <p>Could anyone explain this? Where can i find some explanations about list parameters in slicing? Thanks so much!</p>
<pre data-code-wrap="python"><code class="lang-python">a = torch.tensor([
    [1, 2, 3, 4, 5], 
    [6, 7, 8, 8, 10], 
    [11, 12, 13, 14, 15]]
    )
for _ in range(4):
    x1 = a[:,2:3]
    x2 = a[:,[2]]
    assert torch.all(x1==x2)

    print(x1.storage().data_ptr(),end='\t')
    print(x2.storage().data_ptr())
</code></pre>
<p>The output is:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/8/581f5c27a7d9dc00650a539959111470839ce1dd.png" alt="image" data-base62-sha1="czzcLh6whGRBPRWgtpOT7Y4QAwJ" width="514" height="210"></p>

tom: <p>Indexing with a list or tensor or other iterable (aside from interpreting top-level tuples as multiple index dimensions), this puts you into the realm of “advanced indexing”.<br>
In lieu of PyTorch documentation (aside from <a href="https://pytorch.org/cppdocs/notes/tensor_indexing.html" rel="noopener nofollow ugc">a C++ cheat sheet</a>), you might consult the <a href="https://numpy.org/doc/stable/user/basics.indexing.html#advanced-indexing" rel="noopener nofollow ugc">numpy documentation on indexing</a>.</p>
<p>Best regards</p>
<p>Thomas</p>

Hannibal046: <p>thanks! super helpful!</p>
Gradient memory Issue

ESLAM_ZAHER: <p>Hi,</p>
<p>When I am calculating the integrated gradients, I ran out of GPU memory. It has to do with autograd or something, but I am not sure how to caculate a couple of integrated gradients with different steps and don’t run out of memory. I do the IG calculation within with torch.no_grad(): with no different behaviour<br>
I am using A100 16G GPU</p>

Z4K: <p>Hi !<br>
You can consider using the <strong>internal_batch_size</strong> parameter when you call the attribute method.</p>

ESLAM_ZAHER: <p>Would you care to elaborate more? The attribution method works on the model, input test example, and target. How batch size come into play? and why I am running out of memory on a low level?</p>

Z4K: <p>Hi,<br>
Integrated gradient computes gradients of multiple points between the baseline and input (the number of points corresponds to the <strong>n_steps</strong> parameter). The default is 50 steps.<br>
You might run out memory because a batch of 50 points (if you have not changes the n_steps parameter) is too much for your GPU memory.<br>
If so, you can either reduce the number of steps or use the parameter <strong>internal_batch_size</strong>.</p>
Error with a code for Cross Validation K-Folds

marta420: <p>Hi, I am quite new to python and I just completed Ridge regression on a dataset and I have to check it with K folds cross validation. I’ve been having so many issues with it and yesterday it finally worked, but today I have an error that I can’t go through with. Maybe someone knows what should I change about it for it to finally work properly. <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"></p>
<p>The code for the folds works, but when I try to get the scores I have this type of error.<br>
kf = KFold(n_splits = 5, random_state = 677, shuffle = True)<br>
kf.get_n_splits(X)<br>
for i, (train_index, test_index) in enumerate(kf.split(X)):<br>
print(f’Fold {i}‘)<br>
print(f’ Train: index={train_index}‘)<br>
print(f’ Test: index={test_index}')</p>
<p>scores = cross_val_score(B, X, y, scoring = “neg_mean_squared_error”, cv = kf, n_jobs = -1)</p>
<hr>
<p>TypeError                                 Traceback (most recent call last)<br>
/var/folders/5q/_l03dv393n5_x3nnp1014h7w0000gn/T/ipykernel_9821/3519554670.py in <br>
----&gt; 1 scores = cross_val_score(B, X, y, scoring = “neg_mean_squared_error”, cv = kf, n_jobs = -1)</p>
<p>~/opt/anaconda3/lib/python3.9/site-packages/sklearn/model_selection/_validation.py in cross_val_score(estimator, X, y, groups, scoring, cv, n_jobs, verbose, fit_params, pre_dispatch, error_score)<br>
505     “”"<br>
506     # To ensure multimetric format is not supported<br>
 → 507     scorer = check_scoring(estimator, scoring=scoring)<br>
508<br>
509     cv_results = cross_validate(</p>
<p>~/opt/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_scorer.py in check_scoring(estimator, scoring, allow_none)<br>
446     “”"<br>
447     if not hasattr(estimator, “fit”):<br>
 → 448         raise TypeError(<br>
449             “estimator should be an estimator implementing ‘fit’ method, %r was passed”<br>
450             % estimator</p>
<p>TypeError: estimator should be an estimator implementing ‘fit’ method,                   popularity<br>
intersept          33.235570<br>
duration_ms        -0.230293<br>
danceability        1.503512<br>
energy             -0.758205<br>
key                -0.089013<br>
loudness            0.631607<br>
mode               -0.388885<br>
speechiness        -1.284461<br>
acousticness       -0.258165<br>
instrumentalness   -2.485137<br>
liveness            0.314252<br>
valence            -2.431459<br>
tempo               0.411236<br>
time_signature      0.462741 was passed</p>

Loki_K: <p>Hello</p>
<p>The first argument you pass to cross_val_score is an estimator. The estimator must have a “fit method” as the errror says.<br>
Make sure the estimator “B” you are passing on to cross_val_score() has “fit()” method.</p>
<p>I think this is forum is for pytorch specific issues.</p>
How to build graph conv. for semantic segmentation?

Xiao_Bin: <p>Hi, All,<br>
I touched graph conv. for a short time and confronted with several questions. e.g.,</p>
<ol>
<li>
<p>How many nodes should be built in graph conv. temporarily except memory consumption and is there suitable method to initialize adjacency matrices for graph nodes?</p>
</li>
<li>
<p>In semantic segmentation, assume there are several graph conv. layers in an encoder, now i set the output size as 128 in the first graph conv. as follows:<br>
class GCN(nn.Module):<br>
def <strong>init</strong>(self, in_features):<br>
super().<strong>init</strong>()<br>
self.gcn1 = GraphConvolution(in_features, 128)   # <span class="hashtag-raw">#1</span><br>
self.gcn2 = GraphConvolution(128, in_features)   # <span class="hashtag-raw">#2</span></p>
<p>def forward(self, input, adj):<br>
out = self.gcn1(input, adj)<br>
out = self.gcn2(out, adj)<br>
return out</p>
</li>
</ol>
<p>class Encoder(nn.Module):<br>
…</p>
<p>Indeed, I’m not sure how to set the output size of the first graph conv.? in the similar way, how to set the output size in the second graph conv. (i.e., <span class="hashtag-raw">#2</span>), not ‘in_features’, it’s noted that this network was developed for segmentation task. By the way, the large conv. parameters often incurred memory overflow and is there good idea to avoid it? Thanks in advance.</p>