akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Loading dataset from .pt file using torch.load

Arij-Aladel: <p>Hello !<br>
<span class="mention">@ptrblck_de</span><br>
I have preprocessed a adataset and stored the preprocessed dataset as .pt file.<br>
From a while I have deleted the .cache folder totally</p>
<p>Now loading the data set using torch.load(“filename.pt”)<br>
gives me this error<br>
/home/arij/.cache/huggingface/datasets/hotpot_qa/distractor/1.0.0/133b9501f892e5193babbad937bee3b4899deb4691ef4d791e6ac0111c875bb5/cache-be3bdbe7ba4c6f40.arrow’. Detail: [errno 2] No such file or directory</p>
<p>I just can not understand if I am loading the file from physical location why torch searching the cache???  Do I need to preprocess the dataset again or there is a solution???</p>

Megh_Bhalerao: <p>I have the same question, did you find a solution?</p>

Arij-Aladel: <p>no, I could not find reasonable linking for this error. It is just not reasonable after processing ans saving all in a file to load from cache. I have preprocessed the data again, but this is not reasinable solution for me</p>

JuanFMontesinos: <p>Sincerely you should be using numpy, not torch.</p>
<p>I don’t know the details but in the end pytorch pt files are pickle objects which store any sort of info for which all the dependencies are required during the serialization.<br>
Saving np arrays in  a npy file just requires numpy and allows you to use mmap for efficient loading.<br>
If you aim to save more complex structures then you should prob go for hdf5</p>

Megh_Bhalerao: <p><a class="mention" href="/u/arij-aladel">@Arij-Aladel</a> , as <a class="mention" href="/u/juanfmontesinos">@JuanFMontesinos</a>  mentioned while loading you need to import any python files that might be used by the dataset class which is being saved (or the data loader class). I was able to get mine working after importing some .py files which I was not importing earlier.</p>

Arij-Aladel: <p>I really can not understand your answers I did not mentiomed the numpy or any other library. I just used torch to save processed dataset and I wnated to load it and I could not.</p>

InnovArul: <aside class="quote no-group" data-username="Arij-Aladel" data-post="1" data-topic="151102">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/arij-aladel/48/38195_2.png" class="avatar"> Arij-Aladel:</div>
<blockquote>
<p>/home/arij/.cache/huggingface/datasets/hotpot_qa/distractor/</p>
</blockquote>
</aside>
<p>I doubt that the error is not related to your code, but it is related to <code>huggingface</code> library. It expects <code>hotspotQA</code> dataset to be present in the <code>.cache</code> folder it seems.<br>
Can you try the following code to see if it downloads the dataset?</p>
<pre><code class="lang-auto">from datasets import load_dataset
dataset = load_dataset("hotpot_qa", "distractor")
</code></pre>
<p>(or)<br>
possibly eliminate or comment on the code that might try to use this dataset?</p>

Arij-Aladel: <p>I have already tried before asking any way I preprocessed the dataset again becuase I did not find a solution</p>
Processing sequential autoregressive model outputs in parallel on single gpu?

jayz: <p>Hello,</p>
<p>My model predicts sequentially on input samples X, where the output of the previous timestep “out” is always fed back into the model to get the next output:<br>
out_1 = model(X_1,out_0)<br>
out_2 = model(X_2,out_1)<br>
…<br>
out_t = model(X_t,out_t-1)</p>
<p>Currently, I feed the model sequences [X_1,X_2,…,X_8] of length 8 (and initial value out_0) to compute [out_1,…,out_8] and do the backprop. A single sequence is what I call here a batch. I.e. the length of the sequence = batch_size.</p>
<p>For a single batch, just a small fraction of my GPUs memory is utilized. Therefore, I’d like to process N batches in parallel (given the same state of the model) and then do the backprop every N batches. This would yield to much better memory usage of the GPU while the timing to process the batches (roughly) stays the same as for a single batch. A note: as the sequence length plays a role as a hyperparameter in setting, it is not simply sufficient to increase the batch_size / sequence length of a single batch such that the GPU mem is fully utilized.</p>
<p>I have not dealt with parallelization in pytorch, yet. I often find descriptions how to parallelize models and computations across multiple cuda devices (seriously, how many graphics cards do people have? XD), but not how to efficiently fill the memory of a single cuda device and compute stuff in parallel.</p>
<p>Thanks!</p>
<p>Best, JZ</p>

ptrblck: <aside class="quote no-group" data-username="jayz" data-post="1" data-topic="152495">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jayz/48/46742_2.png" class="avatar"> jayz:</div>
<blockquote>
<p>but not how to efficiently fill the memory of a single cuda device and compute stuff in parallel.</p>
</blockquote>
</aside>
<p>The ability to run workloads in parallel depends on the available compute resources and the stream usage.<br>
Usually kernels are written to occupy the GPU and a potential parallelism of different kernels is not taken into consideration. I.e. even if you schedule parallel kernels using different Streams no SMs could be free to actually launch the overlapping workload, but you could certainly try it out for your use case.</p>
How to efficiently assign features from a 3x3 window in another matrix

BraveDistribution: <p>Hello,</p>
<p>I have a point cloud - node having their own coordinates (x, y). So, for instance we have a point cloud:</p>
<pre><code class="lang-auto">X =    [[1,1], 
        [2,2], 
        [3,4]]
</code></pre>
<p>Then, I have a large matrix that has some features, let’s say a matrix 8,8</p>
<pre><code class="lang-auto">M =     [[1,2,3,2,5,6,7,8],
        [4,5,6,1,8,1,2,4],
        [4,5,H,3,8,1,2,4],
        [1,5,4,4,1,1,2,3],
        [2,5,2,7,1,1,4,3],
        [3,5,4,8,5,1,4,3],
        [4,4,3,6,4,1,1,3],
        [4,4,6,5,8,1,3,5]]
</code></pre>
<p>From the task I know that features around the coordinates are important for the point cloud (let’s say the “kernel size is 3” and i want to get flattened features from those positions. So, for point [2,2] we’d get 5,6,1,5,3,5,4. - Then I want to concatenate them to point cloud features - so for point [2,2] we’d get [2, 2, 5, 6, 1, 5, 3, 5, 4].</p>
<p>I know probably how to do that via CPU, but is there any good algorithm how to do that efficiently? It would be done on-the-fly during training on GPU.</p>
<p>thanks!</p>

eqy: <p>I think this achieves the spirit of what you’re trying to accomplish but some tweaks might be needed depending on how you want to handle the <code>H</code>/center of each patch, etc…</p>
<pre><code class="lang-auto">import torch

X =    [[1,1],
        [2,2],
        [3,4]]

M =     [[1,2,3,2,5,6,7,8],
        [4,5,6,1,8,1,2,4],
        [4,5,0,3,8,1,2,4],
        [1,5,4,4,1,1,2,3],
        [2,5,2,7,1,1,4,3],
        [3,5,4,8,5,1,4,3],
        [4,4,3,6,4,1,1,3],
        [4,4,6,5,8,1,3,5]]

X = torch.tensor(X, device='cuda', dtype=torch.float)
M = torch.tensor(M, device='cuda', dtype=torch.float)

unfold = torch.nn.Unfold(3, padding=1)
M2 = unfold(M.reshape(1, 1, M.size(0), M.size(1)))

X_flat = X[:,0]*M.size(1) + X[:,1]
M2 = M2.permute(0, 2, 1)
out = M2[:,X_flat.long(),:]
print(out.reshape(X.size(0), -1))
</code></pre>
<pre><code class="lang-auto">tensor([[1., 2., 3., 4., 5., 6., 4., 5., 0.],
        [5., 6., 1., 5., 0., 3., 5., 4., 4.],
        [3., 8., 1., 4., 1., 1., 7., 1., 1.]], device='cuda:0')
</code></pre>

BraveDistribution: <aside class="quote no-group" data-username="eqy" data-post="2" data-topic="152135">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/eqy/48/46233_2.png" class="avatar"> eqy:</div>
<blockquote>
<p><code>[1,1]</code></p>
</blockquote>
</aside>
<p>Thank you this is exactly what I wanted. I cannot get my head around the unfold operation… By the way, is this easily changed into M with more channels? (concatenation even over channels).</p>
<p>Thanks.</p>

eqy: <p>There should be no restriction on the number of channels as unfold is basically doing a “convolution” without the reduction.</p>
RuntimeError: unexpected EOF

solargravity: <p>I was trying to run the Python code from this page:<br>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://pytorch.org/favicon.ico?" class="site-icon" width="32" height="32">
      <a href="https://pytorch.org/hub/nvidia_deeplearningexamples_tacotron2/" target="_blank" rel="nofollow noopener">pytorch.org</a>
  </header>
  <article class="onebox-body">
    <img src="https://pytorch.org/assets/images/pytorch-logo.png" class="thumbnail onebox-avatar" width="500" height="500">

<h3><a href="https://pytorch.org/hub/nvidia_deeplearningexamples_tacotron2/" target="_blank" rel="nofollow noopener">PyTorch</a></h3>

<p>An open source deep learning platform that provides a seamless path from research prototyping to production deployment.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>But when the Python interpreter gets to the line “waveglow = torch.hub.load(‘nvidia/DeepLearningExamples:torchhub’, ‘nvidia_waveglow’)”</p>
<p>I get the following error:</p>
<p>Traceback (most recent call last):<br>
File “C:/Users/Public/Documents/Python/PycharmProjects/PyTorchTacotron2/TacotronTest.py”, line 10, in <br>
waveglow = torch.hub.load(‘nvidia/DeepLearningExamples:torchhub’, ‘nvidia_waveglow’)<br>
File “C:\Users\solargravity\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\hub.py”, line 363, in load<br>
model = entry(*args, **kwargs)<br>
File “C:\Users\solargravity/.cache\torch\hub\nvidia_DeepLearningExamples_torchhub/hubconf.py”, line 197, in nvidia_waveglow<br>
ckpt = torch.load(ckpt_file)<br>
File “C:\Users\solargravity\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\serialization.py”, line 386, in load<br>
return _load(f, map_location, pickle_module, **pickle_load_args)<br>
File “C:\Users\solargravity\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\serialization.py”, line 580, in _load<br>
deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)<br>
RuntimeError: unexpected EOF, expected 1439144 more bytes. The file might be corrupted.</p>
<p>Process finished with exit code -1073740791 (0xC0000409)</p>
<p>I tried deleting the nvidia_DeepLearningExamples_torchhub directory which the code downloaded so that a new download is initiated but I still keep getting the same error.  Has anyone had this issue before?  I’m running Windows 10 and Python 3.7.</p>

prasadheeramani: <p>If you are using Window 10 machine, deleting <code>vgg16-something</code> in folder <code>C:\Users\UserName\.cache\torch\checkpoints</code> would solve probelm.</p>

DiogoSnows: <p>Thanks! My notebook crashed while downloading and then I started having the same issue.</p>
<p>Didn’t realise this folder existed (I was clearing the hub cache only).<br>
Funny enough, <code>force_reload=True</code> parameter didn’t help. I would expect it to do the right thing.</p>