akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
What is the mean flatten all dimensions except batch in CNN

Waleed_Eid: <p>what is the mean flatten all dimensions except batch in CNN .<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/3/8391b5ce29853062ae8f770415f0224a1b5054ba.png" data-download-href="https://discuss.pytorch.org/uploads/default/8391b5ce29853062ae8f770415f0224a1b5054ba" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/8/3/8391b5ce29853062ae8f770415f0224a1b5054ba.png" alt="image" data-base62-sha1="iLUJkTLudObA4HGJN8Vgoox1zFU" width="676" height="499" data-dominant-color="F3F3F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">795×588 18.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

yiftach: <p>It changes the shape of <code>x</code>, from a multidimensional tensor with a shape that includes batch size, number of channels, width and height (e.g. [64, 3, 240, 480] to a tensor that has 2 dimensions where the first is batch the the second is everything else (e.g. [64, 345600]).</p>

Mah_Neh: <p>Yeah adding up to comment, you convert the remaining dimensions in a 1D array, so that you now have all red values followed by green and blue, or whichever order, but this order in which the data is flattened is deterministic / always the same, for the same arrange of input channels and dimensions, of course.</p>
Graph Neural Networks cheatsheet

zyxcab123: <p>I am going through GNN cheatsheet: [GNN Cheatsheet — pytorch_geometric documentation<br>
<a href="https://pytorch-geometric.readthedocs.io/en/latest/notes/cheatsheet.html" class="inline-onebox" rel="noopener nofollow ugc">GNN Cheatsheet — pytorch_geometric documentation</a> . There is a table in it by giving many functions and giving a tick mark for corresponding attributes.Someone please explain whats happening in the first function so that i can proceed further. thanks in advance</p>
Using vision transformers for different image resolutions

Oussama_Bouldjedri: <p>Hi, I ma working on using vision transformers not only the vanilla ViT, but different models on UMDAA2 data set, this data set has an image resolution of 128*128 would it be better to transform the images into the vit desired resolution like 224 or 256 or it is better to keep the 128 and try to update the other vision transformer parameters to this resolution like the  dim,depth,heads ?</p>
Save entire model in snapshot or not

iamexperimentingnow: <p>Hi, I’m just wondering, whether is it possible to save entire model in snapshot or not. As, I want to resume the training, I thought it would be good to save the entire model and load.</p>
<p>As of now, I’m doing the below approach</p>
<pre><code class="lang-auto">    def _save_snapshot(self, epoch):
        snapshot = {
            "MODEL_STATE": self.model.module.state_dict(),
            "EPOCHS_RUN": epoch,
        }
        torch.save(snapshot, self.snapshot_path)
        print(f"Epoch {epoch} | Training snapshot saved at {self.snapshot_path}")
</code></pre>
<p>But, I’m expecting something like save the entire model, along with epoch count, optimizer, and loss.</p>
<p>and I tired the below approach and just want to check whether this approach is correct or is there any efficient method?</p>
<pre><code class="lang-auto">torch.save({
            'epoch': 12,
            'model': model,
            'loss': 'checkloss',
            }, './checking_/folder3/testcheck.pth')

</code></pre>
<pre><code class="lang-auto">snap = torch.load('./checking_/folder3/testcheck.pth')
new_model = snap['model']
</code></pre>
<p>can you just confirm me and guide me here?</p>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a></p>

ptrblck: <p>I would not recommend using the second approach as it can easily break since you would need to recreate the same code structure when loading the model.<br>
Saving the <code>state_dict</code>s of the model and optimizer is the recommended approach as also mentioned in <a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html">the docs</a>.</p>
Do I need to install the proprietary AMD drivers to use ROCM on Ubuntu?

xeon826: <p>The directions I see most commonly for installing ROCM on Ubuntu involve doing <a href="https://github.com/RadeonOpenCompute/ROCm-docker/blob/master/quick-start.md" rel="noopener nofollow ugc">this</a></p>
<pre><code class="lang-auto">wget https://repo.radeon.com/amdgpu-install/5.3/ubuntu/focal/amdgpu-install_5.3.50300-1_all.deb 
sudo apt-get install ./amdgpu-install_5.3.50300-1_all.deb
sudo amdgpu-install --usecase=rocm
</code></pre>
<p>Won’t this install the proprietary AMD graphics drivers? I’d rather not use them because they’re not as good as their open source counterparts on Linux, if I remember correctly when I installed the proprietary AMD drivers for Ubuntu it involved commands similar to this, does the <code>--usecase=rocm</code> make it so that only ROCM is installed?</p>
<p>In a nutshell, my question is if I’m using a Radeon 5700 XT navi11 GPU on Ubuntu 22.04.3 jammy do I need to use the proprietary AMD graphics drivers in order to use ROCM or can I keep the open source drivers? I see there’s <a href="https://rocm.docs.amd.com/en/latest/deploy/linux/os-native/install.html" rel="noopener nofollow ugc">another installation method</a> using the Ubuntu repositories would this work to only install ROCM and not the proprietary drivers?</p>
Question about Temporally Fully Connected LSTM in PyTorch

AhzamEjaz: <p>I am currently working on a project involving LSTM (Long Short-Term Memory) networks in PyTorch and have a question about the connectivity of LSTM layers, specifically regarding “temporally fully connected” connections.</p>
<p>I would like to confirm if my understanding is correct: Does PyTorch’s <code>nn.LSTM</code> implementation indeed have these temporally fully connected connections, where each time step in a sequence is connected to each hidden unit in the LSTM layer?</p>
<p>If so, this would align with the standard behavior of LSTM architectures. I want to ensure that I have the correct understanding of how PyTorch handles these connections within LSTM layers.</p>
<p>Suppose I have input_size=10 and hidden_size =100 and time_stamps=10, than would input at each time stamp map to each hidden state in pytorch?</p>
<p>Please clarify and reference any sources(if possible)…</p>
Distributed evaluation with DDP

yiftach: <p>Hi,</p>
<p>When scaling <strong>training</strong> from a single worker to multiple workers (say, multiple GPUs on the same machine), DDP provides abstractions so that I do not have to think about how to best implement synchronization between the workers.</p>
<p>On <strong>evalutation</strong>, however, it seems currently no abstraction/best practice exists, and I have to resort to using lower-level distributed calls, to gather/reduce all my metrics into a single one. Is that correct, or does torch somehow provide a similar experience for evaluation?</p>
<p>I’ve read <a href="https://discuss.pytorch.org/t/ddp-evaluation-gather-output-loss-and-stuff-how-to/130593" class="inline-onebox">Ddp: evaluation, gather output, loss, and stuff. how to?</a> but wonder if things have changed ever since.</p>

iamexperimentingnow: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> <a class="mention" href="/u/suraj.pt">@suraj.pt</a> It would be really helpful you provide some guidance on this. Thanks</p>

ptrblck: <p>I would start with the <a href="https://github.com/pytorch/examples/blob/main/imagenet/main.py#L347-L406">PyTorch ImageNet example</a> and use it as a template.</p>

yiftach: <p>Thanks for the helpful link. Defining an AverageMeter class and calling reduce seems to be very common (I do it in my code, based on other multiple repos that use this idea). Since this practice is common enough to be needed in many different places including what should be the most simple code (imagenet evaluation), any plans to be incorporated into PyTorch itself? Any reason to avoid it?</p>

iamexperimentingnow: <p>thanks <a class="mention" href="/u/ptrblck">@ptrblck</a> I will work on this.</p>
Model evaluation after DDP training

hendrikl: <p>This is maybe a more general question, but I cannot find information about this anywhere.<br>
There are a lot of tutorials how to train your model in DDP, and that seems to work for me fine. However, once the training is done, how do you do the evaluation?<br>
When train on 2 nodes with 4 GPUs each, and have <code>dist.destroy_process_group()</code> after training, the evaluation is still done 8 times, with 8 different results. In my training loop I save the model using <code>torch.save(model.state_dict(),'ClassifierModel.pt')</code>, only on rank 0. Then for the testing loop I load the model using <code>model.load_state_dict(torch.load('./ClassifierModel.pt', map_location={'cuda:%d' % 0: 'cuda:%d' % device}))</code>. Additionally, for the testing I set <code>test_sampler=None</code>.<br>
If needed I could provide more details of my code but I only listed the things which I think might be relevant.</p>

aazzolini: <p>One thing that would be helpful for debugging would be to print out the parameter values for each of the GPUs (e.g. you could print the norm of each parameter of your model). If the values diverge on each GPU, then you may want to brodcast the value of each parameter to all GPUs before you destroy the process group.</p>

huahuanZ: <p>This is somehow weird.<br>
From your description, I think the all the 8 GPUs (processes) loading same parameters and data during inference. There should be no difference on results across these processes, unless your model contains some undetermined operations that related to the random seed.</p>
<p>My suggestion is to narrow down the issue: draw one data sample that would produce the difference, then print the network output layer by layer.</p>

iamexperimentingnow: <aside class="quote no-group" data-username="huahuanZ" data-post="3" data-topic="156754">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/huahuanz/48/39082_2.png" class="avatar"> huahuanZ:</div>
<blockquote>
<p>My suggestion is to narrow down the issue: draw one data sample that would produce the difference, then print the network output layer by laye</p>
</blockquote>
</aside>
<p>Even, I’m facing the same problem, I would like to know how do we evaluate the model after each epoch to see the model performance.</p>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a> <a class="mention" href="/u/suraj.pt">@suraj.pt</a></p>
<p><a class="mention" href="/u/suraj.pt">@suraj.pt</a> I saw your  DDP series tutorial firstly thanks for it. I also have a suggestion that if you could add  an evaluation part in the tutorial would really help us. Could you please us ?</p>
<p>Also, I have an additional question</p>
<ol>
<li>If I train a model on 2 node, each node contains 4 GPU, what should be my <code>num_worker</code> value in <code>DataLoader</code>?<br>
Note: Each node has 48 vCPU nodes</li>
</ol>

ptrblck: <p>Cross-post from <a href="https://discuss.pytorch.org/t/distributed-evaluation-with-ddp/188088/3">here</a> with an answer.</p>
Huge gap between cuda allocated and reserved memories

Pauul: <p>Hi,</p>
<p>I am using PyTorch for a small project of mine and I noticed weird values for CUDA allocated memory and reserved memory. Indeed, while training my model, CUDA reserved memory hits 12 GB while the allocated memory  does not exceed 1.5 GB. Even after using <code>torch.cuda.empty_cache()</code>, the gap between the two remains. I checked and no other process runs on the GPU I’m using.</p>
<p>As far as I understand, based on <a href="https://discuss.pytorch.org/t/difference-between-allocated-and-reserved-memory/87278">this topic</a>, the reserved memory includes pre-cached memory so it should be greater than the allocated memory, but I don’t understand why I obersve such a difference.</p>
<p>When getting OOM errors, PyTorch suggest setting the <code>max_split_size_mb</code> of the PYTORCH_CUDA_ALLOC_CONF variable but I have no idea what values could help resolve my issue.</p>
<p>So my questions are:</p>
<ul>
<li>why are the reserved and allocated memories so different ?</li>
<li>how could I lower the reserved memory values (in contrast to the allocated memory) to avoid OOM errors ?</li>
</ul>
<p>Thanks for your time</p>

harsanyidani: <p><a href="https://discuss.pytorch.org/t/difference-between-allocated-and-reserved-memory/87278/4">Reserved memory contains the allocated and cached memory.</a> So I think they should be roughly the same after calling <code>torch.cuda.empty_cache()</code>. But I too experienced otherwise. So I’m also looking for an answer. (Already asked in <a href="https://discuss.pytorch.org/t/difference-between-allocated-and-reserved-memory/87278/7">this</a> comment.)</p>
Dependency and injector

Ardeal: <p>Hi,</p>
<p>I am reading the code at:<br>
<code>https://github.com/Xilinx/brevitas</code>.</p>
<p>dependency and injector is used in the file:<br>
<code>https://github.com/Xilinx/brevitas/blob/master/src/brevitas/inject/__init__.py</code></p>
<p>I tried to navigate to <code>_dependencies.injector</code>, but I found the package name and information.</p>
<p>is <code>_dependencies.injector</code> the package in python?<br>
or is <code>_dependencies.injector</code> a third party package?<br>
is there any document or tutorial about how to use it?</p>

Ardeal: <aside class="quote no-group" data-username="Ardeal" data-post="1" data-topic="162007">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ardeal/48/33257_2.png" class="avatar"> Ardeal:</div>
<blockquote>
<p>is <code>_dependencies.injector</code> the package in python?</p>
</blockquote>
</aside>
<p>I eventually found out the root cause by myself. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>Just record my solution to here for me and others’ reference:<br>
<code>pip install -U dependencies==2.0.1</code></p>
<p><code>pip install denpendencies</code> will not work. Brevitas requires version 2.0.1.</p>
How to update RPN in Faster RCNN?

johnny69: <p>I want to update the loss function of the RPN in FasterRCNN. See these lines <a href="https://github.com/pytorch/vision/blob/beb4bb706b5e13009cb5d5586505c6d2896d184a/torchvision/models/detection/generalized_rcnn.py#L104-L105" rel="noopener nofollow ugc">here</a>, which refers to the loss function <a href="https://github.com/pytorch/vision/blob/main/torchvision/models/detection/rpn.py#L298" rel="noopener nofollow ugc">here</a>.</p>
<p>I’m trying to update this <code>compute_loss</code> function in my code like so:</p>
<p><code>rpn.RegionProposalNetwork.compute_loss = custom_loss</code></p>
<p>However, this is not changing anything. Any idea how to update the RPN’s loss function?</p>
Efficient scattering a high resolution point cloud of features over a low resolution voxel grid

pirazh: <p>Hi,</p>
<p>I have a point cloud of size <code>N</code> where for each point I have a feature vector of size <code>C</code> and 3D location <code>(x,y,z)</code>, so I can represent this point cloud with two tensors of size <code>N*C</code> and <code>N*3</code>. Now I am trying to scatter this point cloud over a lower resolution voxel grid of size <code>V*V*V</code> so that I can get the feature volume of size <code>V*V*V*C</code>. Since the voxel grid is of lower dimension, there will be some voxels that receive multiple points while others might get None. (I make sure to translate my point cloud coordinates to voxel coordinates). Now to compute the feature volume first I instantiate a count volume of size <code>V*V*V</code> and a feature volume of size <code>V*V*V*C</code> with zeros. A straight forward way is run a for loop over all the points, identify which voxel the point belongs to, add the point feature to the voxel feature and increase the count corresponding to that voxel by 1. Once done, to get the final feature volume, the summed feature volume is divided by the count:</p>
<blockquote>
<p>feat_vol = torch.zerors(V,V,V,C)<br>
count_vol = torch.zeros(V,V,V,1)</p>
</blockquote>
<blockquote>
<p>for i in feat_pc.shape[0]:</p>
<blockquote>
<p>feat_vol[coord_pc[i]] += feat_pc[i]<br>
count_vol[coord_pc[i]] += 1</p>
</blockquote>
</blockquote>
<blockquote>
<p>feat_vol /= (count_vol + eps)</p>
</blockquote>
<p>However this would be very inefficient as the number of points increases and I am wondering whether there is a more efficient way for this.</p>
<p>I can run the following vectorized operations:</p>
<blockquote>
<p>feature_vol[coord_pc] += feat_pc<br>
count_vol[coord_pc] += 1</p>
</blockquote>
<p>The issue here is that since there are multiple points sharing the same voxel (<code>coord_pc</code> is the same for multiple points) <code>feature_vol</code> and <code>count_vol</code> is only updated with one point which is not desired. Unless there is a similar concept such as manager in python multiprocessing, that can update the same underlying object from multiple inputs, this cannot work. I would appreciate any thoughts or suggestions.</p>
<p>Thanks</p>
Noise gate from torchaudio functional or transforms

Circuit: <p>Hi,</p>
<p>Is there a way to replicate a noise gate using either a functional or transforms filter in plain pytorch audio? As part of my pipeline I’ve been able to keep everything on the GPU except for a noise gate, where I move the tensor back to CPU and get it’s numpy handle to run it through Pedalboard’s noise gate. If I used a torch sox effect it would still be on the CPU. If anyone has worked on their own solution, I’d be glad to see.</p>
Is cubic spline available in pytorch?

Doublejelly: <p>I want to find a cubic spline function approximation given data ${x_i,y_i}$ which is equivalent to scipy.interpolate.interp1d(x,y,kind=‘cubic’). Scipy gives a numpy-type function and this will disconnect graph. I tried to find such a function but I think torch.nn.functional.intepolate is  different from what I want. Is it available in pytorch?</p>

william_ashbee1: <p>did you ever find a solution to this?</p>

googlebot: <aside class="onebox allowlistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">
      <a href="https://github.com/patrick-kidger/torchcubicspline" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars3.githubusercontent.com/u/33688385?s=400&amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/patrick-kidger/torchcubicspline" target="_blank" rel="nofollow noopener">patrick-kidger/torchcubicspline</a></h3>

<p>Interpolating natural cubic splines. Includes batching, GPU support, support for missing values, evaluating derivatives of the spline, and backpropagation. - patrick-kidger/torchcubicspline</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p><a href="https://github.com/bayesiains/nsf/blob/master/nde/transforms/splines/cubic.py" rel="nofollow noopener">NSF implementation</a></p>

dengbuqi: <p>Love you so much!<br>
That’s exactly what I want!</p>