akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Torchaudio.io not properly using ffmpeg

JamesDickens: <p>I am following this tutorial about hardware-accelerated gpu encoding/decoding for PyTorch [<a href="https://pytorch.org/audio/main/hw_acceleration_tutorial.html" rel="noopener nofollow ugc">https://pytorch.org/audio/main/hw_acceleration_tutorial.html</a>], I am encountering an error with the following code:</p>
<pre><code>import torch
import torchaudio

print(torch.__version__) # 1.14.0.dev20221013+cu116
print(torchaudio.__version__) # 0.13.0.dev20221013+cu116
print(torchaudio._extension._FFMPEG_INITIALIZED) # True

from torchaudio.io import StreamReader, StreamWriter
local_src = "vid.mp4"
cuda_conf = {
    "decoder": "h264_cuvid",  # Use CUDA HW decoder
    "hw_accel": "cuda:0",  # Then keep the memory on CUDA:0
}

def decode_vid(src, config):
    frames = []
    s = StreamReader(src)
    s.add_video_stream(5, **config)
    for i, (chunk,) in enumerate(s.stream()):
        frames.append(chunk[0])

if __name__ == "__main__":
    vid = decode_vid(local_src, cuda_conf)
</code></pre>
<p>The error message (somewhat truncated) is:</p>
<blockquote>
<p>File<br>
“/home/james/PycharmProjects/AlphaPose/Spectronix/Early_Experiments/vid_gpu_decode.py”,<br>
line 23, in decode_vid<br>
s.add_video_stream(5, **config)   File “/home/james/anaconda3/envs/alphapose/lib/python3.7/site-packages/torchaudio/io/_stream_reader.py”,<br>
line 624, in add_video_stream<br>
hw_accel, RuntimeError: Unsupported codec: “h264_cuvid”.</p>
</blockquote>
<p>I have an RTX 3090 ti as my GPU, which does support the h264_cuvid, and I have been able to decode a video on the command line running (taken from the tutorial linked above)</p>
<pre><code>sudo ffmpeg -hide_banner -y -vsync 0 -hwaccel cuvid -hwaccel_output_format cuda -c:v h264_cuvid -i "https://download.pytorch.org/torchaudio/tutorial-assets/stream-api/NASAs_Most_Scientifically_Complex_Space_Observatory_Requires_Precision-MP4_small.mp4" -c:a copy -c:v h264_nvenc -b:v 5M test.mp4
</code></pre>
<p>So it seems <a href="http://torchaudio.io" rel="noopener nofollow ugc">torchaudio.io</a> is not properly using ffmpeg. Any insights of how to fix this problem much appreciated. I’m using Ubuntu 22.04.</p>
Meaningful validation metric for rotated object detection

gonzrubio: <p>What is a good validation metric to use for object detection with rotated bounding boxes?</p>
<p>I am working with satellite images so I care about correctly predicting the “head” of the box.</p>
<p>I am regressing for the rotation angle during training but I am unsure how the F1 score takes this into account since it measure the overlap of my ground truths and predictions without regarding their orientations.</p>
<p>Thank you in advance.</p>
Profiling Distributed Data Parallel Applications

Vibhatha_Abeykoon: <p>Hi,</p>
<p>I am trying to profile an application using DistributedDataParallel Module.</p>
<p>Is there a specific set of guidelines to measure the communication overheads (allreduce time, broadcast time, etc)?</p>
<p>I used the <code>with torch.autograd.profiler.profile(use_cuda=True)</code>. But I didn’t get information about these calls. This may only track basic calls not functions like allreduce or broadcast happening in ProcessGroups (NCCL) layer.</p>
<p>Please correct me if I am wrong.</p>
<p>Thank You,<br>
Vibhatha.</p>

mrshenli: <p>Hey <a class="mention" href="/u/vibhatha_abeykoon">@Vibhatha_Abeykoon</a> DDP does not work with autograd profiler yet, but this is in our roadmap. Before that, will <a href="https://developer.nvidia.com/blog/cuda-pro-tip-nvprof-your-handy-universal-gpu-profiler/" rel="nofollow noopener">nvprof</a> able to serve your use case?</p>

Vibhatha_Abeykoon: <p><a class="mention" href="/u/mrshenli">@mrshenli</a> Sorry for the late response. Yes, it could also be useful. I will check.<br>
Thank You.</p>

rvarm1: <p>Profiling with DDP is be enabled now, i.e. the collectives will be profiled. You can simply run the DDP model under profiler as normal:</p>
<pre><code class="lang-auto">with torch.profiler.profile():
    ddp(inputs)
</code></pre>

kristianeschenburg: <p><a class="mention" href="/u/rvarm1">@rvarm1</a> – do you have a more thorough example of using the torch profiler with DDP and <code>torch.distributed</code>?  I’m trying to apply it to my training script but am running into issues where <code>dist.init_process_group</code> hangs indefinitely when using the profiler.  The script runs normally when excluding any instance of the profiler.</p>
AttributeError: module 'torchvision.transforms' has no attribute 'Scale'

jabbarshah: <p>Hi there!<br>
How to fix this issue? I am using torchvision==0.12.0.</p>
<p>Thanks</p>
<pre><code class="lang-auto">transformations = transforms.Compose([torchvision.transforms.Scale(224),
transforms.CenterCrop(224), transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])])
 
 model.cuda(gpu)

</code></pre>

ptrblck: <p><code>torchvision.transforms.Scale</code> was deprecated in favor of <code>torchvision.transforms.Resize</code> so use the latter transformation in current releases.</p>

Raymond_Confidence: <p>Hello Sir,<br>
How about, torchvision.transforms.ElasticTransform</p>
<p>Having an error: AttributeError: module ‘torchvision.transforms’ has no attribute ‘ElasticTransform’</p>

ptrblck: <p><code>ElasticTransform</code> was added in <a href="https://github.com/pytorch/vision/pull/4938">this PR</a> in June so ~4 months ago and you might need to update <code>torchvision</code> to the nightly release.</p>

Raymond_Confidence: <p>Thanks a lot for your response</p>
Sharing computation between detached and non-detached variables

imh: <p>Hi, I’m new to the forum and hope this is the right place to ask.</p>
<p>I want to truncate my backwards pass in two different ways for two different passes, but don’t know how to split the forwards and backwards passes.</p>
<p>Here is a toy example (<code>w1</code> and <code>w2</code> are just weights):</p>
<pre><code class="lang-auto">x = f(upstream, w1)

z = g(x, w2)

loss1 = h(z)
loss2 = L(z)
</code></pre>
<p>I’m trying to efficiently compute these grads:</p>
<ul>
<li>grad(loss1) with respect to w1</li>
<li>grad(loss2) with respect to w2</li>
</ul>
<p>The model above only does one forwards pass through <code>g</code>, but two backwards passes through <code>f</code>.</p>
<p>I can trade two backwards passes through <code>f</code> for two forwards passes through <code>g</code>:</p>
<pre><code class="lang-auto">x = f(upstream, w1)

z1 = g(x, w2)
z2 = g(x.detach(), w2)

loss1 = h(z1)
loss2 = h(z2)
</code></pre>
<p>These two forwards passes through <code>g</code> are identical though, so it’s still waste.</p>
<p>Is there a way to only do one backwards pass through <code>f</code> and also only one forwards pass through <code>g</code>?</p>
Torch DataPipe contains a list of lists, how do I split it to make it iterate over the first list?

Ryan_Avery: <p>dp_sen1_item_lists is a <code>torch.utils.data.datapipes.iter.callable.MapperIterDataPipe</code></p>
<p><code>next(iter(dp_sen1_item_lists))</code></p>
<p>returns</p>
<pre><code class="lang-auto">[&lt;Item id=sen12floods_s1_source_62_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_19_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_0250_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_0249_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_0248_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_0247_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_0246_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_0245_2019_04_01&gt;]
</code></pre>
<p>but I want it to return just the first element</p>
<pre><code class="lang-auto">&lt;Item id=sen12floods_s1_source_62_2019_04_01&gt;
</code></pre>
<p>how do I make it iterate over that list rather than a list of lists? II want to split up the datapipe somehow. I’ve tried dp_sen1_item_lists.header(1) but that returns a child data pipe that still contains a list but doesn’t iterate over that list</p>
<pre><code class="lang-auto">next(iter(dp_sen1_item_lists.header(1)))

[&lt;Item id=sen12floods_s1_source_62_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_19_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_0250_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_0249_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_0248_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_0247_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_0246_2019_04_01&gt;,
 &lt;Item id=sen12floods_s1_source_0245_2019_04_01&gt;]
</code></pre>
<p>I’m definitely new to datapipes and having trouble finding an example or reference, any help is much appreciated</p>

Ryan_Avery: <p>I’ve opted to use dp_sen1_item_lists.batch(8) for now to go to a nested list but this requires knowing the length of the iterable up front.</p>

nivek: <p>If you use <code>flatmap</code> with no argument to iterate through the inner lists (<a href="https://pytorch.org/data/main/generated/torchdata.datapipes.iter.FlatMapper.html#torchdata.datapipes.iter.FlatMapper" rel="noopener nofollow ugc">see the second example</a>). If you only want the first list, the do <code>dp.header(1).flatmap()</code>. Let me know if that does not work as expected.</p>
<p>While the no-op behavior will be in the next release, for 0.4.0, you may need to pass in a no-op function. Note that you will need to install the nightly version or from source for that to work.</p>
Advise on validation loss and train loss

sai_m: <p>I am working on a academic computer vision project and here are the train loss and validation loss. is it advisable to stop at 25 epochs or take this as is and use model from epoch 44, which has the best validation loss. Any advise is appreciated!</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/b/bb5d4289019a9a1a4eb3a811606738d790facd2e.png" data-download-href="https://discuss.pytorch.org/uploads/default/bb5d4289019a9a1a4eb3a811606738d790facd2e" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/b/bb5d4289019a9a1a4eb3a811606738d790facd2e.png" alt="image" data-base62-sha1="qJv5UHeSLVcdBWyTbDBoFj6iqWa" width="267" height="500" data-dominant-color="323232"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">472×882 37.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/2/520c98fab65541a679b2eb2652310979bedf9c3e.png" alt="image" data-base62-sha1="bHQ8QLe4HFBzZEnvoSs8JKKrW3k" width="534" height="388"></p>
Segmentation output for sigmoid become 0.5

henry_Kang: <p>Hello. I have some questions about my code. I follow this code from Kaggle loss <a href="https://www.kaggle.com/code/bigironsphere/loss-function-library-keras-pytorch/notebook" class="inline-onebox" rel="noopener nofollow ugc">Loss Function Library - Keras &amp; PyTorch | Kaggle</a></p>
<p>My data is highly class imblance in the image. For example, my object is 10 ~ 20 pixels from 4K images.<br>
For this, I can not resize the image, instead of resizing, I am cropping the image with 1024 x 512 resolution.<br>
For the input normalization, I divide by 255 about each channel. My mask is composed of 0 and 1. 1 is the target object, and my background is 0.<br>
During the training, I apply the IoU loss. When I check that most of value of sigmoid output is 0.5. For this reason, the IoU is very low since our output is 0 and 1.<br>
Second, the output pixel value becomes the minus value.<br>
Third, the loss calculation become minus sometiems.<br>
Anyone have ideo for this?<br>
I assume that try softmax and try to solve this problem as classification problem. In that cases it does not provide good result compare to Focal tversky loss or iou loss.<br>
I understand that sigmoid can provide the 0.5 but I believe that I should provide function like relu. instead of sigmoid. What do you think about this idea?<br>
Thank you.</p>
RuntimeError: unable to open shared memory object </torch_3282906_2794037818_1009> in read-write mode: Too many open files (

Salwa_Mostafa: <p>Hello</p>
<p>I am facing this error while trying to run my code</p>
<p>Traceback (most recent call last):<br>
File “/wrk1/Salwa_Directory/KhairyCode/find-goal/train.py”, line 72, in <br>
File “/wrk1/Salwa_Directory/KhairyCode/find-goal/util/shared_opt.py”, line 42, in <strong>init</strong><br>
File “/wrk1/Salwa_Directory/KhairyCode/find-goal/util/shared_opt.py”, line 49, in share_memory<br>
File "/homedir05/smostafa22/.local/lib/python3.9/site-packages/torch/<em>tensor.py", line 515, in share_memory</em><br>
File “/homedir05/smostafa22/.local/lib/python3.9/site-packages/torch/storage.py”, line 599, in share_memory_<br>
File “/homedir05/smostafa22/.local/lib/python3.9/site-packages/torch/storage.py”, line 195, in share_memory_<br>
<strong>RuntimeError: unable to open shared memory object &lt;/torch_3282906_2794037818_1009&gt; in read-write mode: Too many open files (24)</strong><br>
<strong>First:</strong><br>
I checked the shared memory size. It was<br>
$ cat /proc/sys/kernel/shmmni<br>
4096<br>
I am working on the university server so I don’t have access to increase the shared memory.</p>
<p>$ ulimit -n 16384<br>
bash: ulimit: open files: cannot modify limit: Operation not permitted<br>
<strong>Second</strong><br>
I tried to change the sharing strategy<br>
import torch.multiprocessing<br>
torch.multiprocessing.set_sharing_strategy(‘file_system’)</p>
<p>but it raises the following error instead</p>
<p>File “/homedir05/smostafa22/.local/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 689, in <br>
return self._apply(lambda t: t.cuda(device))<br>
RuntimeError: CUDA error: out of memory<br>
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.<br>
For debugging consider passing CUDA_LAUNCH_BLOCKING=1</p>
<p>Can you please help me out?</p>

ptrblck: <p>If you cannot change the system settings your second approach sounds valid and the CUDA OOM error might be unrelated. Did you check if this workload can fit onto your GPU?</p>
Split a dataset to multiple slices

ZSheikhb: <p>How can I randomly split a data and target one to n slices in pytorch? Something similar to the following example in numpy:</p>
<pre><code class="lang-auto">I=15

split_points = np.random.choice(data.shape[0] - 2, I - 1, replace=False) + 1

split_points.sort()

data = np.split(data, split_points)

target= np.split(target, split_points)
</code></pre>
Running everything on CPU

AfonsoSalgadoSousa: <p>Hi. How do I explicit setup Ignite’s configuration to run everything on CPU (like auto_model and auto_dataloader to CPU as well)?</p>

vfdev-5: <p>I think there is nothing special with that. You can just inform the script that there is no GPUs and it would take cpu as default:</p>
<pre><code class="lang-bash">CUDA_VISIBLE_DEVICES= python script.py
</code></pre>
<p>What do you think ?</p>
TypeError: cannot unpack non-iterable float object

Joshua_Wilbur: <p>I have recently started on my machine learning and deep learning journey and I was following this <a href="https://www.youtube.com/watch?v=HMAIoVQel1c&amp;ab_channel=Dataquest" rel="noopener nofollow ugc">DataQuest tutorial</a>. Everything was going fine. Each cell was running just the way it should. Until i got to the training and even then the model trained for 3.5 hours then i got this error when it finished.</p>
<code>
<p>TypeError: cannot unpack non-iterable float object</p>
</code>
<p>This is the block of code that would gave me the error</p>
<code>
<p>for epoch in range(EPOCHS):<br>
for batch, (images, labels, img_paths) in enumerate(train):<br>
optimizer.zero_grad()</p>
<pre><code>    images = images.to(device)
    pred = model(images.float())
    labels = labels.to(device)
    loss = loss_fn(pred, labels)

    loss.backward()
    optimizer.step()

loss, current = loss.item()
print(f"loss: {loss:&gt;7f} [{epoch}]")
</code></pre>
</code>
<p>This is the entire <a href="https://github.com/JoshuaWilbur/Dog_Emotion_Detector/blob/e5f605dbefc56e24d3586c9bf48d12c76ddcaa49/Dog_Emotion_Detector.ipynb" rel="noopener nofollow ugc">Code on GitHub</a>.</p>
<p>I would usually play around with the code a bit and see what i can do but 3.5 hours is a long time to wait to see if what i tried worked.</p>
<p>Any help would be appreciated.</p>
<p>Thank you</p>

ambujpawar: <p>Hi, the issue lies in the line:<br>
<code>loss, current = loss.item()</code></p>
<p>loss.item() gives you a single float value. And it cannot be packed to two values, loss, current. The correct way would be:<br>
<code>loss = loss.item()</code></p>
<p>If you want a more in depth tutorial please refer to this example over <a href="https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html#training-the-model" rel="noopener nofollow ugc">here</a>.</p>
<p>If you want to read more about <code>loss.item()</code> please refer <a href="https://discuss.pytorch.org/t/what-is-loss-item/61218">this</a>.</p>
<p>Regarding, it taking 3.5 hours. I thinking you are not using GPUs in google colab. Computations would be much faster in GPU.</p>

ambujpawar: <p>Just checked the video you linked as well, the author uses this as well over <a href="https://youtu.be/HMAIoVQel1c?t=2143" rel="noopener nofollow ugc">here</a></p>

Joshua_Wilbur: <p>Thank you for your help and for the references for loss.item(). It is extremely appreciated. I used googles colab to code in because my computer isnt the greatest its an old hp prodesk 6200 and definitely doesnt have a nvidia gpu. I changed the runtime to gpu i just figured it took so long because of the size of the data set and the amount of epochs. In the video i was following he said it took him til the next day to finish training</p>
Key error 10 while running the training code

Kamin: <p>I have tried to train the following code:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/SeungjunNah/DeepDeblur-PyTorch">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/SeungjunNah/DeepDeblur-PyTorch" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/ed944bf2478cec3167efa2944067abca0730b6d57fa596ee6ef80567b97d3d2e/SeungjunNah/DeepDeblur-PyTorch" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/SeungjunNah/DeepDeblur-PyTorch" target="_blank" rel="noopener nofollow ugc">GitHub - SeungjunNah/DeepDeblur-PyTorch: Deep Multi-scale CNN for Dynamic...</a></h3>

  <p>Deep Multi-scale CNN for Dynamic Scene Deblurring. Contribute to SeungjunNah/DeepDeblur-PyTorch development by creating an account on GitHub.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
But the training code is stopped automatically at epoch 10 due to <strong>key error 10</strong> . I’m not exactly sure why this error occurs.</p>
<pre><code class="lang-auto">File "main.py", line 67, in &lt;module&gt;
    main()
  File "main.py", line 64, in main
    main_worker(args.rank, args)
  File "main.py", line 54, in main_worker
    trainer.test(epoch)
  File "/home/DeepDeblur-PyTorch-master/src/train.py", line 196, in test
    self.evaluate(epoch, 'test')
  File "/home/DeepDeblur-PyTorch-master/src/train.py", line 182, in evaluate
    tq.set_description(self.criterion.get_loss_desc())
  File "/home/DeepDeblur-PyTorch-master/src/loss/__init__.py", line 311, in geoss_desc
    loss = self.loss_stat[self.mode]['Total'][self.epoch]
KeyError: 10 


</code></pre>
<p>Could you help me solve this error.</p>

ambujpawar: <p>This seems not be a pytorch/torchvision issue. More like a issue in the github repo.</p>
<p>Based on the initial hunch, it looks like something with epoch, maybe…</p>
How to reproduce my experiment completely?

pangpang: <p>Hello everyone,<br>
I want to reproduce my experiment completely. Previously I used a server with RTX 3090, but now it has been moved now, so I bought a new one. And I configure the pytorch environment as before, but I can’t reproduce my experiment (The accuracy is different from before). I use this function to fix the seed as the picture shows. What else can I do to reproduce my experiment? Thanks!<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/6/1/61a4a8d262022383257c1b9a5e2cd4cb44ed43d5.jpeg" alt="微信图片_20221013180818" data-base62-sha1="dVN5WDECBcSQu7n4RUGBgae0UfP" width="608" height="322"></p>

InnovArul: <p>From <a href="https://pytorch.org/docs/stable/notes/randomness.html" class="inline-onebox">Reproducibility — PyTorch 2.1 documentation</a> guide,</p>
<blockquote>
<p>Completely reproducible results are not guaranteed across PyTorch releases, individual commits, or different platforms. Furthermore, results may not be reproducible between CPU and GPU executions, even when using identical seeds.</p>
</blockquote>
<p>So, you may be unable to reproduce the exact training sequence/accuracy with a different CPU/GPU.<br>
What’s the deviation in accuracy that you observe?</p>
Network Loading Fails for LSTM

Rajesh_Karmakar: <p>Hi,</p>
<p>I am getting the below error for the code which was just working, the network is the same, and now I get this error. Any possible solution? thanks.</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "sample.py", line 70, in &lt;module&gt;
    decode(args)
  File "sample.py", line 33, in decode
    decoder.load_state_dict(torch.load(args.decoder_path))
  File "/home/rj/anaconda2/envs/dlp/lib/python3.6/site-packages/torch/nn/modules/module.py", line 769, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for DecoderRNN:
	Missing key(s) in state_dict: "embed.weight", "lstm.weight_ih_l0", "lstm.weight_hh_l0", "lstm.bias_ih_l0", "lstm.bias_hh_l0", "lstm.weight_ih_l1", "lstm.weight_hh_l1", "lstm.bias_ih_l1", "lstm.bias_hh_l1". 
	Unexpected key(s) in state_dict: "bn.weight", "bn.bias", "bn.running_mean", "bn.running_var", "bn.num_batches_tracked". 
	size mismatch for linear.weight: copying a param with shape torch.Size([256, 2048]) from checkpoint, the shape in current model is torch.Size([36881, 512]).
	size mismatch for linear.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([36881]).
</code></pre>

lanka: <p>i also getting the same…any solution?</p>
Persistent worker Error

umarkhalid: <p>I am getting the error<br>
TypeError: <strong>init</strong>() got an unexpected keyword argument ‘persistent_workers’<br>
I am using torch ==1.5.1<br>
It doesn’t matter even if I makes pin memory False.</p>

ptrblck: <p>Your PyTorch version might be too old, so update to the latest stable or nightly release and this argument should work.</p>
Should `pin_memory` always used with `non_blocking=True`?

yzs: <p>Simplest case:</p>
<pre><code class="lang-python">import torch
import torch.cuda.nvtx as nvtx
from contextlib import contextmanager

@contextmanager
def stream_wrapper(stream):
    yield torch.cuda.stream(stream=stream)

non_blocking = True
s1 = torch.cuda.current_stream()
cuda0 = torch.device('cuda:0')

a = torch.tensor([1., 2.]).pin_memory().to(device=cuda0)
for _ in range(5):
    nvtx.range_push('copy')
    with stream_wrapper(s1):
        a = a.to(device=torch.device('cpu'), non_blocking=non_blocking)
    s1.synchronize()
    with stream_wrapper(s1):
        a = a.to(device=cuda0, non_blocking=non_blocking)
    s1.synchronize()
    nvtx.range_pop()

</code></pre>
<p>If the <code>non_blocking</code> is set to <code>False</code>, the <code>a.to()</code> will only launch <code>cudaMemcpy</code> on pageable memory. If I understand the two right, I believe this is weird since the only difference between a <code>pin_memory()</code>-ed tensor and a normal one should be the virtual memory settings in CPU memory. And <code>non_blocking</code> should only affect the specific memory copy methods: <code>cudamemcpy</code> or <code>cudamemcpyasync</code>.</p>
<p>My original purpose is to use pinned memory for fast CPU-GPU swapping (synced) of tensors. Currently, I could only use cuda stream to wait for the <code>non_blocking</code> copy. I believe if <code>tensor.to(non_blocking=False)</code> could achieve high performance on pinned tensors, I will not need to explicitly synchonize the stream.</p>
Modified a input_channel using replacement with new Conv2d working but not with using model.layer.in_channel=12

xiang-zhe: <p>I have a model network and its parameters, i modified the parameters, then go to change a channel of a conv2d layer, at first, i used model.layer.in_channels = 12, and then check with model.modules, it showed that input_channel(is 12 as assigned) of the conv2d was modified, but when using model.load_state_dict(parameters), it showed errors with parameters’ shape is (16,12,3,3) but model’s shape is (16,3,3,3);<br>
but i using model.layer = conv2d() to replace original layer works fine.<br>
why model.layer.in_channels = 12 is not working? but comment code is working.<br>
THANKS for any interpretation.<br>
codes like:</p>
<pre><code class="lang-auto">m = Model()
print(m.features[0][0])
m.features[0][0].in_channels = 12
print(m.features[0][0])
print(m.modules)
# new_layer = nn.Conv2d(12, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
# m.features[0][0] = new_layer
m.load_state_dict(mv)
</code></pre>
<p>output is like:</p>
<pre><code class="lang-auto">Conv2d(3, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
Conv2d(12, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
&lt;bound method Module.modules of MobileNetV3LargeEncoder(
  (features): Sequential(
    (0): ConvBNActivation(
      (0): Conv2d(12, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
...
ERROR:size mismatch for features.0.0.weight: copying a param with shape torch.Size([16, 12, 3, 3]) from checkpoint, the shape in current model is torch.Size([16, 3, 3, 3]).

</code></pre>

ptrblck: <p>Changing the internal <code>in_channels</code> attribute won’t reinitialize the parameters. Creating a new module or reassigning new <code>nn.Parameter</code>s should work.</p>

xiang-zhe: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="163339">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>nn.Parameter</code>s</p>
</blockquote>
</aside>
<p>THANK YOU very much,</p>
<blockquote>
<p>reassigning new <code>nn.Parameter</code>s should work</p>
</blockquote>
<p>i use Parameter() reinitialize the parameters,<br>
it works using like:</p>
<pre><code class="lang-auto">m.features[0][0].in_channels = 12
m.features[0][0].weight = nn.Parameter(mv['features.0.0.weight'])
m.load_state_dict(mv)
</code></pre>
Told driver is too old, but it's the latest version (Ubuntu WSL2)

andrewjk: <p>Getting “UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 10010 …”</p>
<p>However nvidia-smi reports: " NVIDIA-SMI 520.56.05  Driver Version: 522.25  CUDA Version: 11.8" and print(torch.version.cuda) returns 11.6</p>
<p>Running 20.04.3 on 5.10.102.1-microsoft-standard-WSL2 with an RTX 3070.</p>
<p>I’ve searched but haven’t found a clear path forward. Any tips?</p>
Attn_mask implementation in torch.nn.functional

Tony_Chen: <p>in <code>_scaled_dot_product_attention(</code> of torch.nn.function, attn_mask is used with</p>
<pre><code class="lang-auto">4851     if attn_mask is not None:
4852         attn = torch.baddbmm(attn_mask, q, k.transpose(-2, -1))
</code></pre>
<p>since baddbmm is essentially adding attn_mask to q @ k.T, I was wondering how this does what the documentation says attn_musk does? (" attn_mask: If specified, a 2D or 3D mask preventing attention to certain positions. Must be of shape")</p>
<p>Thanks!</p>
Torch :: Greater and Torch:: Where functions

donparker1: <p>I am trying to consolidate torch::where and torch::greater into a single function. I have bypass the binary output from torch::where to store the indexes of into a tensor object; however, the performance is not efficient. I have included my solution:</p>
<pre><code class="lang-auto">  std::vector &lt;int&gt; idxs;
   
            for(int x = 0; x &lt; tensor.numel(); x++){
                        if (tensor[x].item&lt;float&gt;() &gt; threshold)
                                            idxs.push_back(x);
                                    
            }
</code></pre>
Dynamic Dataframe possible?

adgower: <p>I’m not entirely certain how to accomplish what I want to accomplish with pytorch yet, but I was wondering if its possible to have a base dataframe that is for example: unlimited rows, and 3 columns</p>
<p>column 1 has 1 value, column 2 is an array that can hold any amount of arrays, and column 3 is an array that can hold any amount of arrays</p>
<pre><code class="lang-auto">df = pd.DataFrame([
    ['Col 1', 'Col 2', 'Col 3'],
    [
        1,
        [
            [
                1,
                [
                    [1, 1, 1, 1],
                    [1, 1, 1, 1]
                ]
            ],
            [
                2,
                [
                    [2, 2, 2, 2],
                    [2, 2, 2, 2]
                ]
            ]
        ],
        [
            [
                1,
                [1, 1]
            ],
            [
                2,
                [2, 2]
            ]
        ]
    ]
])
</code></pre>
How to do projected gradient descent in pyTorch?

hyperactve: <p>i have a variable which is in [0,1]. So, to optimize it I was thinking of using projected gradient descent. Was wondering how to do that. Here is a skeleton I have right now:</p>
<pre><code class="lang-auto">for it in range(no_it):
    optimizer.zero_grad()
    //get data D, y
    z = model(D)
    loss = crit(z,y)
    loss.backward()
    optimizer.step()
    if model.b.detach().cpu().numpy() &gt; 1.0:
        model.b.data = torch.tensor(1.0).to(device)
    if model.b.detach().cpu().numpy() &lt;10**-5:
        model.b.data = torch.tensor(10**-5).to(device)
</code></pre>
<p>Was wondering if this is the efficient way to do it. I’m afraid it might fill up the GPU with tensor variables if I keep creating them. Is there a way to directly change the value in the underlying numpy variable?</p>
<p>note that <code>model.b.size()</code> outputs <code>torch.Size([])</code>.</p>
<p>Any help is appreciated.</p>
Warning spam then using MultiMarginLoss

hadaev8: <pre><code class="lang-auto">/usr/local/lib/python3.8/dist-packages/torch/nn/functional.py:3349: UserWarning: An output with one or more elements was resized since it had shape [], which does not match the required output shape [56].This behavior is deprecated, and in a future PyTorch release outputs will not be resized unless they have zero elements. You can explicitly reuse an out tensor t by resizing it, inplace, to zero elements with t.resize_(0). (Triggered internally at  ../aten/src/ATen/native/Resize.cpp:23.)
  return torch._C._nn.multi_margin_loss(input, target, p, margin, weight, reduction_enum)
</code></pre>
<p>Not sure what it want from me, I checked inputs and shapes is [56, 22] and [56]</p>

ptrblck: <p>Could you post a minimal, executable code snippet reproducing this warning, please?</p>

hadaev8: <pre><code class="lang-auto">import torch
from torch import nn
from torch.nn import functional as F

bs = 56
model = nn.Linear(128, 22).cuda()
loss = nn.MultiMarginLoss()
x = torch.rand((bs, 128)).cuda()
targets = torch.randint(22, (bs,)).cuda()
out = model(x)
print(targets.shape)
print(out.shape)
loss(out, targets)
</code></pre>
<p>Another observation: it is fine without cuda.</p>

eqy: <p>Thanks for finding and reporting this issue! I’ve opened a PR to address it here:<br>
<a href="https://github.com/pytorch/pytorch/pull/75000" rel="noopener nofollow ugc">Abate spurious resize warnings in <code>MultiMarginLoss</code> on CUDA by eqy · Pull Request #75000 · pytorch/pytorch (github.com)</a></p>

crissallan: <p>Hi, does this issue fixed now? or is there any way to ignore the warning during my training process</p>

ptrblck: <p><a class="mention" href="/u/eqy">@eqy</a>’s PR was merged already so the issue should be fixed.</p>
I'm getting an error while using torchvision.datasets.Kinetics

Bihy: <p>I’m trying to use  <a href="https://pytorch.org/vision/main/generated/torchvision.datasets.Kinetics.html" rel="noopener nofollow ugc">torchvision.datasets.Kinetics</a> but I’m getting this Assertion Error.</p>
<p>This is the initialization of the dataset</p>
<pre><code class="lang-auto">test_dataset = torchvision.datasets.Kinetics(root=root_path, frames_per_clip=33,
     step_between_clips=5, num_workers = 40, split='test', transform = transforms.RandomResizedCrop(size=224))
</code></pre>
<p>This is error I get:</p>
<pre><code class="lang-auto">AssertionError: Caught AssertionError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/bairouk/anaconda3/envs/deep/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/bairouk/anaconda3/envs/deep/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/bairouk/anaconda3/envs/deep/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 49, in &lt;listcomp&gt;
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/bairouk/anaconda3/envs/deep/lib/python3.10/site-packages/torchvision/datasets/kinetics.py", line 234, in __getitem__
    video, audio, info, video_idx = self.video_clips.get_clip(idx)
  File "/home/bairouk/anaconda3/envs/deep/lib/python3.10/site-packages/torchvision/datasets/video_utils.py", line 373, in get_clip
    assert len(video) == self.num_frames, f"{video.shape} x {self.num_frames}"
AssertionError: torch.Size([34, 720, 1280, 3]) x 33
</code></pre>
<p>Thank you.</p>
Too many values to unpack

semsem20051: <p>Hello all,</p>
<p>I’m currently studying aws MLE and we should make a project about image classifications database dog breeds.</p>
<p>after wrote the training script, every time i run the code it raises value error  Too many values to unpack</p>
<pre><code class="lang-auto">Failure reason
AlgorithmError: ExecuteUserScriptError: Command "/opt/conda/bin/python3.6 hpo.py --batch-size 32 --lr 0.003409797631466478" Downloading: "https://download.pytorch.org/models/resnet18-5c106cde.pth" to /root/.cache/torch/hub/checkpoints/resnet18-5c106cde.pth 0%| | 0.00/44.7M [00:00&lt;?, ?B/s] 43%|âââââ | 19.2M/44.7M [00:00&lt;00:00, 201MB/s] 87%|âââââââââ | 39.0M/44.7M [00:00&lt;00:00, 205MB/s] 100%|ââââââââââ| 44.7M/44.7M [00:00&lt;00:00, 206MB/s] Traceback (most recent call last): File "hpo.py", line 194, in &lt;module&gt; main(args) File "hpo.py", line 143, in main train(model, train_loader, criterion, optimizer, epoch, device) File "hpo.py", line 57, in train for data, target in train_loader: ValueError: too many values to unpack (expected 2), exit code: 1
</code></pre>
<p>here is my code</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/samy-moh/Project-3">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/samy-moh/Project-3" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/64ef64df07c743f7c04d911ae491d390cb861cc9269e2b72fe237a4b4acc5986/samy-moh/Project-3" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/samy-moh/Project-3" target="_blank" rel="noopener nofollow ugc">GitHub - samy-moh/Project-3</a></h3>

  <p>Contribute to samy-moh/Project-3 development by creating an account on GitHub.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>if anyone could help i will be very appreciated</p>
<p>thanks in advance</p>

ptrblck: <p>Check what your <code>train_loader</code> returns e.g. via <code>print(next(iter(train_loader)))</code> and make sure you are receiving two objects. If that’s not the cast check why the <code>Dataset</code> might not return your expected tensors.</p>

semsem20051: <p>Thanks a lot for your help sir, I will try it, and hope that I will figure it out.</p>
Finding accuracy and plotting the learning curve and plotting the losses in mask rcnn

filmonmehari: <p>I am just using <strong>maskrcnn</strong> model for my project. But i am unable to find out the accuracy of my model and plot the learning curve( train and validation Vs epoc) and loss graph (i.e train loss vs test loss). I have read previous discussions abt accuracy saying mAP is used in object detection, but my advisor is requesting me to show him the accuracy(he is familiar only with the normal accuracy metrics) , learning curve and loss graph.</p>
<p>Even i am troubled whether i have to add data for my model cuz i didn’t find the top accuracy and learning curve in mask rcnn.<br>
So please help me for the solution , i have only 3 weeks to submit with those parameters</p>

Lane: <p>Did you find accuracy, I’m also working with maskrcnn ?</p>
How to find nan in dataframe

Berny: <p>Hello. I need to find the elements of two dataframe columns that are nan and set them to zero:</p>
<pre><code>for k in mat_files:
        my_df[k]['calibratedHsCube'] = my_df[k]['calibratedHsCube'].replace(np.nan, 0)
        my_df[k]['mask'] = my_df[k]['mask'].replace(np.nan, 0)
</code></pre>
<p>But I get this:  AttributeError: ‘numpy.ndarray’ object has no attribute ‘replace’<br>
Where am I wrong?</p>

srishti-git1110: <p><code>replace</code> is for Series (<a href="https://pandas.pydata.org/docs/reference/api/pandas.Series.replace.html" class="inline-onebox" rel="noopener nofollow ugc">pandas.Series.replace — pandas 1.5.0 documentation</a>).</p>
<p>You could use <code>fillna()</code> for DataFrame objects.</p>

Berny: <p>I noticed that when I print the dataframe column there are still some nan values:</p>
<pre><code>    for k in mat_files:
        my_df[k][['calibratedHsCube', 'mask']] = my_df[k][['calibratedHsCube', 'mask']].fillna(value=0)
        print(my_df[k]['calibratedHsCube'])
</code></pre>
<p>Did I make a mistake maybe?</p>

srishti-git1110: <p>I am not sure what <code>mat_files</code> is and consequently how exactly the dataframe is being indexed; looks like <code>k</code> corresponds to row index.<br>
Apparently, you aren’t replacing nans in a full column or the full dataframe at once. Check your indexing technique to look for potential sources of error.</p>

Berny: <p>mat_files is:</p>
<pre><code> mat_files = glob(f'{Folder}/*.mat')
</code></pre>
Calling model.eval() on during validation or just testing?

Dusty: <p>Hello everyone,</p>
<p>do I call <code>model.eval()</code>, when computing the validation loss during my training? Or do I only use evaluation mode on my test set during inference, after my training is complete?</p>
<p><strong>Context</strong>: I have a train set and a test set. <em>After each epoch</em>, i run my test set through the model to obtain the validation loss at each epoch. Since the validation loss seems to be “simply scaled” by some orders of magnitutes (as in the curve looks the same for val and train loss, but are around 40x higher for validation), I wonder if this has anything to do with it.</p>

srishti-git1110: <p>Yes, you should be calling <code>model.eval()</code> whenever you are inferencing.</p>

Dusty: <p>Thank you very much.</p>
Why the grad of parameter is None?

mx10108287: <p>I want to use feature_conv * self.a to select  values in feature_conv, But the value in Parameter a is always be one.</p>
<p>I found the grad of a is None, how can I fix it?</p>
<pre><code class="lang-auto">    def _sel_CAM(self, feature_conv):

        #self.a = nn.Parameter(torch.ones((1,2048,1,1), dtype=torch.float32),requires_grad=True)
        #feature_conv is a torch tensor

        B, C, H, W = feature_conv.size()
        S = H*W

        cam = torch.sum(feature_conv*self.a, dim=1) / 2048
        cam = cam.view(B, -1)
        rank = torch.sort(cam, dim=1)[1].view(B, H, W)
        mask = rank &gt;= (S-16)
        mask = mask.view(B, H, W)
        feature_conv = feature_conv * mask.unsqueeze(1)
        
        return feature_conv
</code></pre>

srishti-git1110: <aside class="quote no-group" data-username="mx10108287" data-post="1" data-topic="163348">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mx10108287/48/53956_2.png" class="avatar"> mx10108287:</div>
<blockquote>
<p>I found the grad of a is None, how can I fix it?</p>
</blockquote>
</aside>
<p>Where are you backpropagating? Please post a minimum executable snippet that reproduces your error.</p>
Extract modules from output tensor

ConvolutionalAtom: <p>Hi,<br>
I want to know if we can see the applied modules given the output tensor.<br>
More precisely, given the following code</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

my_input = torch.rand(4,12)
my_ln = nn.Linear(12, 4)
my_output = my_ln(my_input)

</code></pre>
<p>Can we extract <code>nn.Linear</code> module just from <code>my_output</code> tensor? I think we can extract the operations applied with <code>grad_fn</code>, but I am asking about modules.</p>
Installed Pytorch 1.12.1 in the conda environment but it detects version 1.10.0+cpu

vikrant_karale: <p>Recently, I have installed pytorch 1.12.1 in the conda environment. After installation, I checked the version of pytorch using <code>print(torch.__version__)</code>, it returns <code>1.10.0+cpu</code>. I also checked the available packages in the environment. It shows pytorch version <code>1.12.1</code></p>
<p>I am unable to understand why it is detecting version <code>1.10.0+cpu</code>. I even reinstalled Anaconda python in Windows, still it shows same version <code>1.10.0+cpu</code> even in the <code>base</code> environment.</p>
<p>My system has Nvidia 1650Ti GPU and Cuda 11.7 with Nvidia driver 517.00 is installed.</p>
<p>Is Pytorch 1.10.0+cpu inbuilt in anaconda python?</p>

ptrblck: <p>No packages should be built into the anaconda base env unless you specify them during its setup. Delete the <code>torch</code> installation from the base env, install the desired version in your new virtual env and it should work. If that’s still not the case check all install logs and make sure no other of your installed packages needs the older <code>1.10</code> release and installs it additionally.</p>

vikrant_karale: <p>Thanks for the reply. I have not installed Pytorch in ‘base’ environment. I created new environment and then installed it.</p>
<p>I even uninstalled Anaconda python completely and reinstalled it.<br>
After reinstalling Anaconda python, I executed commands <code>import torch</code>, and <code>print(torch.__version__)</code>.<br>
It is very strange that even without installing Pytorch, it is able to import it and it shows version <code>1.10.0+cpu</code>.</p>
<p>Also, ‘conda list’ does not show Pytorch package in the list (Refer figure)<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/e/a/ea047e87d7029c60bfae373599eb9abcbc561b3d.png" alt="image" data-base62-sha1="xodm8humKQPMwf29P8vr21OZYhn" width="526" height="242"></p>

ptrblck: <p>In that case your conda env might detect a <code>pip install</code>ed PyTorch binary. Uninstall all PyTorch versions from conda, pip, and any other way you’ve previously installed it to create a clean environment first, then install it it the desired env only.</p>

vikrant_karale: <p>Thanks for your input. I tried uninstalling the existing pytorch version 1.10.0+cpu using pip/conda but nothing was working. I was finally able to get rid of it by deleting the old Python folder at ‘C:\Users\Vikrant\AppData\Roaming\Python’. I created a new environment and installed Pytorch 1.12.1. Now, everything is working fine. Thanks!</p>
Repeat tensor only in the last dimension

SungmanHong: <p>I have a tensor of shape [2, 2] where the first 2 is the batch size</p>
<p>I want to make this tensor to [2, 64] at the forward function. (Repeat 32 times)<br>
How can I do it? the torch’s repeat api changes every dimension… (for e.g : [64, 64] comes out)</p>

InnovArul: <p><code>torch.repeat()</code> lets us specify the number of times to repeat for each dimension.<br>
Something like below should work:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; x =  torch.randn(2,2)
&gt;&gt;&gt; y = x.repeat(1, 32)
&gt;&gt;&gt; y.shape
torch.Size([2, 64])
</code></pre>
Torch acos cast to float32

samsja: <p>Hey all.</p>
<p>I have a very weird issue that I can’t reproduce outside of my training loop so I apologize in advance for not providing a minimal reproducible example.</p>
<p>I have a tensor in float 16 on cuda, I passed it to torch.acos in my training loop and the return is a float32 …</p>
<p>I am debugging in the middle of my training step.</p>
<pre><code class="lang-python">x = torch.zeros(10).half().to('cuda')
torch.acos(x).dtype
&gt;&gt;&gt; torch.float32
</code></pre>
<p>This makes no sense to me. What could go wrong with my back-end for that to happened  ? I can’t reproduce outside of this training loop</p>
<p>my torch version: ‘1.12.1+cu102’</p>
<p>I am using pytorch lightning not sure if this can introduce some issue somewhere …</p>

samsja: <p>Okay I found the issue <img src="https://discuss.pytorch.org/images/emoji/apple/person_facepalming.png?v=12" title=":person_facepalming:" class="emoji" alt=":person_facepalming:" loading="lazy" width="20" height="20">, it is the autocast that is probably activated in pytorch lightning</p>
<pre><code class="lang-python">import torch
with torch.autocast('cuda'):
    x = torch.zeros(10).half().to('cuda')
    print(torch.acos(x).dtype)
&gt;&gt;&gt; torch.float32
</code></pre>
<p>Is it okay then to recast it automatically to half precision if I need to ? Is there a way I can ask for keeping half precision here ?</p>

ptrblck: <p>Yes, you can cast it back to <code>float16</code> if your use case doesn’t need the numerical precision which will be lost by this operation. Alternatively, you could also disable <code>autocast</code> for this operation so that no casts will be done.</p>