akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Userwarning: Numpy array not writable after coverting png image loaded using Pillow and tryping to covert to tensor

peepeepoopoo: <p>Seems the issue arose when converting to numpy array, is there a walk around to this instead making a copy of the array (since this will gave a huge overhead)?</p>
<p>The exact warning: <em><strong>UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /pytorch/torch/csrc/utils/tensor_numpy.cpp:141.)</strong></em></p>
<p>PS: used the recommended method for conversion: convert_image_dtype(pil_to_tensor(img)) but again the issue isn’t here but the fact that pytorch seems to not be using numpy.array(img) for conversion.</p>

Javi_UIBK: <p>Hello! just came by your thread because I’m facing the exact same problem. In my past projects I did not have such issue but now I have it for some reason… Did you find a solution to this?</p>
How to unroll ISTA as a neural network

Nawel: <p>Hello,</p>
<p>I’m trying to unroll a proximal gradient descent/ISTA algorithm as a NN.<br>
The classical ISTA iteration can be written as :</p>
<pre><code> x(k+1) = sign(x(k)) * soft_threshold( W1 * x(k) + W2* y)
</code></pre>
<p>where W1 and W2 are known matrices.</p>
<p>I want to unroll the optimization steps as a NN where W1 and W2 are learnable linear layers.</p>
<p>This is my attempt to do so</p>
<pre><code class="lang-auto">class Lista(torch.nn.Module):

    def __init__(self, uvw, freq, npixel, n_layers, max_iter=100, verbose=False, device=None, *args,**kwargs):
        
        super().__init__()

        self.H        =  torch.tensor(forward_operator(uvw, freq, npixel))
        self.n_layers = n_layers
        self.device   =  device
        self.nvis, self.npixel2 = self.H.shape
        self.L        =        np.linalg.norm(self.H,ord=2)**2      

        self.layers_Wx  = nn.ModuleList()
        self.layers_Wy = nn.ModuleList()

      
        for _ in range(self.n_layers):
           self.layers_Wx.append(torch.nn.Linear(self.npixel2,self.npixel2, bias=False))
           self.layers_Wy.append(torch.nn.Linear(self.nvis,self.npixel2, bias=False))  #.to(torch.cdouble)

        self._init_layer_parameters()

     
    def forward(self, y, lmbd, x0):
        x_   = check_tensor(x0, device=self.device)
        step = 1/self.L
        for layer_idx in range(self.n_layers):

            z_ =  self.layers_Wx[layer_idx](x_) + self.layers_Wy[layer_idx](y)
            x_ = soft_thresholding(z_, lmbd*stp)
        
        return x_



    def _init_layer_parameters(self):
        Wx_init = torch.from_numpy( np.eye(self.npixel2) - np.matmul(self.H.T.conj(), self.H)/self.L )
        Wy_init = torch.from_numpy( self.H.T.conj()/self.L)

        for layer_idx in range(self.n_layers):
            self.layers_Wx[layer_idx].weight = nn.Parameter(Wx_init)
            self.layers_Wy[layer_idx].weight = nn.Parameter(Wy_init)

</code></pre>
<p>Everything functions correctly when executing the code without backpropagation (i.e., solely as ISTA without updating W1 and W2).<br>
However, the results deteriorate when running it with backpropagation (no convergences, loss explodes etc).</p>
<p>I would greatly appreciate any insights or guidance on this issue.<br>
Thanks</p>
Different result in training and validation

cbd: <p>The result of last training iteration and first iteration of validation is different. Which parameters are vary during validation? I have taken the same image in last iteration of training and first iteration of validation. The result is different.</p>

ptrblck: <p>Assuming you are calling <code>model.train()</code> before starting training the model and <code>model.eval()</code> during the evaluation some layers will change their behavior, such as Dropout (which will be disabled during evaluation) and BatchNorm (which will use its running stats during evaluation instead of the batch stats to normalize the input activation).</p>
How torch.fft.fft works in autograd?

trinitite: <p>Hi,<br>
In autograd, what math is used to compute the gradient of fft?<br>
<span class="hashtag-raw">#an</span> example<br>
<code>import torch</code><br>
<code>asd = torch.linspace(1,100,100).view(-1,1)</code><br>
<code>x3 = asd.repeat(1,100)</code><br>
<code>yobs3 = torch.fft.fft(x3**2,n=1024,dim=1)+2.0</code><br>
<code>yobs3</code><br>
<code>x3.requires_grad=True</code><br>
<code>y3 = torch.fft.fft(x3**2,n=1024,dim=1)</code><br>
<code>y3</code><br>
<code>loss3 = torch.sum((y3-yobs3)**2)</code><br>
<code>loss3.backward()</code><br>
<code>print(x3.grad)</code></p>

trinitite: <aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/pytorch/blob/826ab0e32d558415d5d682842417fd16b2223739/torch/csrc/autograd/FunctionsManual.cpp#L4472C12-L4472C12">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/blob/826ab0e32d558415d5d682842417fd16b2223739/torch/csrc/autograd/FunctionsManual.cpp#L4472C12-L4472C12" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/826ab0e32d558415d5d682842417fd16b2223739/torch/csrc/autograd/FunctionsManual.cpp#L4472C12-L4472C12" target="_blank" rel="noopener nofollow ugc">pytorch/pytorch/blob/826ab0e32d558415d5d682842417fd16b2223739/torch/csrc/autograd/FunctionsManual.cpp#L4472C12-L4472C12</a></h4>



    <pre class="onebox"><code class="lang-cpp">
      <ol class="start lines" start="4472" style="counter-reset: li-counter 4471 ;">
          <li>Tensor fft_c2r_backward(</li>
      </ol>
    </code></pre>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
Question about Tensor storage lifespan

yatorho: <p>I have some confusion regarding the lifecycle of PyTorch tensors. The lifecycle of a PyTorch tensor is managed by Python,  and CUDA computations in PyTorch are asynchronous to the Python interpreter(CPU). Could a situation arise where Python triggers the destruction of a tensor variable, but at the same time, a CUDA kernel is still executing? In the following example:</p>
<pre><code class="lang-auto">a = torch.randn(3, 4).cuda() + torch.randn(1).cuda()
</code></pre>
<p>PyTorch asynchronously starts a CUDA kernel for the addition operation. However, the tensor created by <code>torch.randn(3, 4).cuda()</code> is a temporary variable that is destructed when <code>__add__</code> is called. In this case, the CUDA kernel might still be in progress when the tensor is destructed, potentially leading to the kernel accessing invalid device memory.</p>
<p>However, in my tests, this situation doesn’t seem to occur. I would like to understand how PyTorch ensures that asynchronous CUDA kernels complete their computation before the memory associated with temporary tensors is reclaimed.</p>
Jitting fails on mps device

boywithacoin: <pre><code class="lang-auto">def f(x): return x.to(torch.uint8)
x = torch.tensor([-1.0, -2.0], device=torch.device("mps"))

print(f(x))
print(torch.compile(f)(x))
</code></pre>
<blockquote>
<p>torch._dynamo.exc.BackendCompilerFailed: backend=‘inductor’ raised:<br>
TypeError: ‘NoneType’ object is not callable</p>
</blockquote>
<p>The above fails when I select to compile using Metal device.<br>
However, on the cpu it passes using OpenMP.</p>
<p>device info: <code>Apple M1 Max using torch 2.1.0</code></p>

marksaroufim: <p>the default backend inductor is not supported on MPS so you’ll need to use <code>torch.compile(m, backend="aot_eager")</code></p>

boywithacoin: <p>thanks. although, it defeats the purpose of torch.compile for me, if it’s still eager. my goal was to see the output device-specific kernel.</p>
<p>is there a github tracker for mps support in compile mode? maybe i could contribute.</p>

marksaroufim: <p><a class="mention" href="/u/kulinseth">@kulinseth</a> was working on this I believe, my understanding is MPS would need a Triton backend things should just work with Inductor although the details probably matter</p>
Pytorch_jni.dll: The specified procedure could not be found ->Module.load() Please help me

TomS: <p>I have a pytorch model which I serialized through torch.jit.trace. I want to use it in an android app. When I load the model through Module module = Module.load(“model.pt”); I get this error:</p>
<p>“Exception in thread “main” java.lang.UnsatisfiedLinkError: C:\Windows\System32\pytorch_jni.dll: The specified procedure could not be found<br>
at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)<br>
at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388)<br>
at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:232)<br>
at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:174)<br>
at java.base/jdk.internal.loader.NativeLibraries.findFromPaths(NativeLibraries.java:315)<br>
at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:287)<br>
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2427)<br>
at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:818)<br>
at java.base/java.lang.System.loadLibrary(System.java:1989)<br>
at com.facebook.soloader.nativeloader.SystemDelegate.loadLibrary(SystemDelegate.java:24)<br>
at com.facebook.soloader.nativeloader.NativeLoader.loadLibrary(NativeLoader.java:52)<br>
at com.facebook.soloader.nativeloader.NativeLoader.loadLibrary(NativeLoader.java:30)<br>
at org.pytorch.NativePeer.(NativePeer.java:10)<br>
at org.pytorch.Module.load(Module.java:28)<br>
at org.pytorch.Module.load(Module.java:38)<br>
at com.example.pytorch_app.Temp.main(Temp.java:28)”<br>
All these lines point to the load function.</p>
<p>I have tried using Java/Kotlin, tried changing versions but nothing is working. I’ve been stuck on it for days. Please help <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
Has “torch.compile” been tested on deepspeed?

BobLiu20: <p>I tested the combination of torch.compile and deepspeed under the LLaMA2 model and encountered a problem:</p>
<ol>
<li>When running only on a single machine with 8 GPUs, the performance can be improved by about 30%+.</li>
<li>When running on 2 machines with 16GPU, there is no performance improvement.</li>
</ol>
<p>I don’t know how to check the problem. I searched a lot on Google and didn’t find the answer.<br>
Any suggestions? Thanks.</p>

marksaroufim: <p>We’re in early discussions with the deepspeed team to figure this out, no concrete plans yet</p>
Triton random warning

AidanGoldfarb: <p>Every execution of my function decorated with <code>@dynamo.optimize("inductor")</code> I am getting the following warning:<br>
<code>torch._inductor.utils: [WARNING] using triton random, expect difference from eager</code></p>
<p>I am ok with differences due to a faster RNG, however I would love if this warning could be suppressed. The API for warnings seems to have changed a lot since I last used Pytorch, I was hoping for some help.</p>
<p>Even</p>
<p><code>torch._inductor.config.fallback_random = True</code></p>
<p>Which I assume disables the more efficient RNG, does not work anymore.</p>

marksaroufim: <p>Ah this seems annoying could you please open an issue so we can track <a href="https://github.com/pytorch/pytorch/issues" class="inline-onebox">Issues · pytorch/pytorch · GitHub</a></p>
How can i realize this attention

Skyrax: <p>Hello, i’m trying to use linear self-attention for image, but if i apply it to image 128x128 it consume too much memory. Can i make patches (8, 8 for example) for query and key with conv layers compute attention and apply this attention to 128x128 value matrix so 1 element form attention apply to 8x8=64 element of value matrix</p>
Monotonicity and strict concavity

FedeHeinrichsCorrado: <p>Hi all,</p>
<p>I’ve got a partial differential equation that I’m trying to approximate with a physics inspired neural network approach. The issue is that it doesn’t quite come out right. I know that the solution should be monotonic and strictly concave, and I think I have to impose those conditions, and I’m wondering if anyone has any ideas about how to go about it.</p>
<p>Many thanks,<br>
Federico</p>
2 Nvidia A40,but show error "torch.cuda.OutOfMemory"Error

dongkuang: <p>when I run the LLM of int8 of 70B, when I need a 70G video memory program, the system loads about 46G video memory of an A40 graphics card, and another A40 graphics card has not been started, and an error is reported, as follows:</p>
<p>torch.empty(self.output_size,<br>
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 448.00 MiB (GPU 0; 44.42 GiB total capacity; 43.11 GiB already allocated; 426.81 MiB free; 43.11 GiB reserved in total by PyTorch) If reserved memory is &gt;&gt; allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF</p>
<p>Please help me!</p>

ptrblck: <p>Reduce the memory requirement by e.g. decreasing the batch size, via checkpointing, CPU-offloading etc.</p>

dongkuang: <p>Thank you!But why can’t we solve this problem by working together with two A40 graphics cards?And I have installed nvlink,But the result is still the same!</p>

ptrblck: <p>You can use both GPUs, but would need to implement it explicitly e.g. via pipeline parallel approaches as PyTorch won’t shard the model automatically.</p>
Font in the instructions are not sharp in Chrome browser

sisim: <p>for example when I open the link below in chrome, windows the fonts are not sharp and are blurry.<br>
It could be said, that is chrome issue, which is not supporting these fonts family or could be fixed by selecting another famous fonts, just to make it readable for everyone</p>
<p><a href="https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss</a></p>
<p>Thanks<br>
Best</p>

albanD: <p>Which font is the problem exactely? Is it the math part?</p>

sisim: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/d/ddaf33120bebcdf7f08e1c39e6651e07d07cc647.png" data-download-href="https://discuss.pytorch.org/uploads/default/ddaf33120bebcdf7f08e1c39e6651e07d07cc647" title="Capture"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/d/ddaf33120bebcdf7f08e1c39e6651e07d07cc647_2_690x462.png" alt="Capture" data-base62-sha1="vD6SOGI222SOSKzQj8n0K7GnMkD" width="690" height="462" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/d/ddaf33120bebcdf7f08e1c39e6651e07d07cc647_2_690x462.png, https://discuss.pytorch.org/uploads/default/original/3X/d/d/ddaf33120bebcdf7f08e1c39e6651e07d07cc647.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/d/d/ddaf33120bebcdf7f08e1c39e6651e07d07cc647.png 2x" data-dominant-color="F4F2E9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture</span><span class="informations">938×629 65.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

albanD: <p>Maybe we want to have someone looking into this, cc <a class="mention" href="/u/smth">@smth</a></p>

albanD: <p>I opened a github issue to track this: <a href="https://github.com/pytorch/pytorch/issues/28906" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/28906</a><br>
Feel free to post there your browser / screen size or any relevant information.</p>
<p>Thanks for the report !</p>

Volt: <p>I’m having the same issue, fonts looks awfully. Looks like I’ve found what causes it: I usually increase the text size in W11 by using the “Text Size” slider on the “Accessibility” &gt; “Text Size” page in the Windows settings, and this makes the font look terrible on the Documentation pages on the Pytorch site (BTW, the only site I’ve seen that has such an issue). If I reduce the text size to 100%, this problem goes away. Hope to see this issue fixed, it’s quite annoying, to be honest.</p>
How i realise this?

yupeng_chou: <p>In practice, we define a subset S of all template images for each training iteration. Before training begins, we sequentially examine each label in the training batch and remove the duplicated labels,then accumulate unique label from a shuffled label set until the number of unique label reaches the predefined batch size N. This sampling and accumulating strategy ensures that the template images corresponding to the training samples are totally covered in the subset S.<br>
.</p>
DDP training fast initially, then slows to a crawl but does not crash

jhaas: <p>Hello,</p>
<p>I’m using DDP to train using the torch.distributed.launch command.</p>
<p>Training is normal for about 700 batches, then it consistently completely slows down at this point and does not recover (training continues, just very slowly).</p>
<p>Memory usage remains constant after the slowdown. htop reveals that CPU usage drops off heavily – at the beginning, all 40 cores are highly utilized, but at the point of slowdown, only 2 or 3 processes continue with high utilization (close to 100%) and the rest of them fall to around 1-5%.</p>
<p>Nvidia-smi reveals that GPU utilization locks at 100% for a subset of GPUs, and then 0% for the others-- this is the case whether training with 2 or 4 GPUs.</p>
<p>It looks to me like there is some kind of synchronization problem, but I’m not sure how to confirm that, or how to address it, if true.</p>
<p>I’ve tried different settings for OMP_NUM_THREADS without any change.</p>
<p>I’ve also tried KMP_AFFINITY=granularity=fine,compact,1,0</p>
<p>CPU: Intel(R) Xeon(R) Silver 4210 CPU @ 2.20GHz<br>
GPUs: 4 x TeslaV100 23GB<br>
CentOS: centos-release-7-9.2009.1.el7.centos.x86_64<br>
PyTorch: 1.12.1<br>
CUDA: 10.1</p>
<p>Using the compact transformers training script from:</p><aside class="onebox githubfolder" data-onebox-src="https://github.com/SHI-Labs/Compact-Transformers/tree/main">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/SHI-Labs/Compact-Transformers/tree/main" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h3><a href="https://github.com/SHI-Labs/Compact-Transformers/tree/main" target="_blank" rel="noopener nofollow ugc">GitHub - SHI-Labs/Compact-Transformers: Escaping the Big Data Paradigm with...</a></h3>

  <p><a href="https://github.com/SHI-Labs/Compact-Transformers/tree/main" target="_blank" rel="noopener nofollow ugc">main</a></p>

  <p><span class="label1">Escaping the Big Data Paradigm with Compact Transformers, 2021 (Train your Vision Transformers in 30 mins on CIFAR-10 with a single GPU!) - GitHub - SHI-Labs/Compact-Transformers: Escaping the Big ...</span></p>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
Verbose warning in `torch.logical_xor`

hzhangxyz: <p>Here is an example which causes pytorch throw warning.</p>
<pre><code class="lang-python">import torch

a = torch.tensor([False, True]).reshape([1, 2])
b = torch.tensor([False, True]).reshape([2, 1])
c = torch.logical_xor(a, b)
</code></pre>
<p>The warning detail is</p>
<pre><code class="lang-auto">/home/hzhangxyz/Downloads/tmp.py:5: UserWarning: An output with one or more elements was resized since it had shape [1, 2], which does not match the required output shape [2, 2]. This behavior is deprecated, and in a future PyTorch release outputs will not be resized unless they have zero elements. You can explicitly reuse an out tensor t by resizing it, inplace, to zero elements with t.resize_(0). (Triggered internally at /build/python-pytorch/src/pytorch-opt/aten/src/ATen/native/Resize.cpp:28.)
  c = torch.logical_xor(a, b)
</code></pre>
<p>Pytorch throws such warning in version 2.1.0 but did not throw such warning in version 2.0.1.</p>