akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
About torch.fx: Is there a way to convert Proxy to base type？

username: MartinZhang
cooked: <p>Is there a way to convert Proxy to base type？<br>
such as :</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from torch.fx import symbolic_trace

class M(nn.Module):
  def __init__(self):
    super().__init__()
  def forward(self, x):
    x = x.size(1)
    a = torch.randn(x)
    return a
m = M()

symbolic_trace(m)
</code></pre>
<p>while the type of x is Proxy, and the function randn can’t use Proxy.</p>
<p>So, I need to convert Proxy to int type.</p>
<p>please help me, thanks a lot.</p>

username: ash_gamma
cooked: <p>Hi, did you find a fix for this? Running into similar isse</p>

username: pvalois
cooked: <p>Any possible solution? I am also having problems with torch.eye</p>

username: MartinZhang
cooked: <pre><code class="lang-auto">import torch
import torch.nn as nn
from torch.fx import symbolic_trace

class M(nn.Module):
  def __init__(self):
    super().__init__()
    self.n = 0

  def forward(self, x):
    if not isinstance(x, torch.fx.Proxy):
        self.n = x.size(1)
    # x = x.size(1)
    a = torch.randn(self.n)
    return a

m = M()
data = torch.randn(1, 3, 224 ,224)
m(data)  # need forward this module first.

symbolic_trace(m)
</code></pre>
<p>I hope this can help you!</p>
One-hot encoding with label 0 all 0

username: qiminchen
cooked: <p>Suppose I have a tensor <code>a</code> with size of <code>(1, 1, 6, 6)</code></p>
<pre><code class="lang-auto">print(a)
tensor([[[[0., 0., 0., 0., 0., 0.],
          [0., 4., 4., 5., 5., 0.],
          [0., 4., 4., 5., 5., 0.],
          [0., 2., 2., 1., 1., 0.],
          [0., 2., 2., 1., 1., 0.],
          [0., 0., 0., 0., 0., 0.]]]])
</code></pre>
<p>and I have 5 classes in totall, is it possible to have one-hot encoding <code>b</code> with size of <code>(1, 5, 6, 6)</code> (without for loop if possible)</p>
<p>such that <code>0</code> for empty so one-hot for label <code>0</code> would be <code>[0, 0, 0, 0, 0]</code>, and <code>1</code> would be <code>[1, 0, 0, 0, 0]</code> … <code>5</code> would be <code>[0, 0, 0, 0, 1]</code>, I tried to use <code>F.one_hot</code> but cant get what I need. Thanks.</p>
<ul>
<li>the label should be <code>0-4</code> since 5 classes, but I have 0 specifically for empty space so I change the label to <code>1-5</code>.</li>
</ul>

username: ptrblck
cooked: <p>You could zero out the first row:</p>
<pre><code class="lang-python">x = torch.arange(5)
print(x)
# tensor([0, 1, 2, 3, 4])
y = F.one_hot(x, num_classes=5)
print(y)
# tensor([[1, 0, 0, 0, 0],
#         [0, 1, 0, 0, 0],
#         [0, 0, 1, 0, 0],
#         [0, 0, 0, 1, 0],
#         [0, 0, 0, 0, 1]])
y[:, 0] = 0
print(y)
# tensor([[0, 0, 0, 0, 0],
#         [0, 1, 0, 0, 0],
#         [0, 0, 1, 0, 0],
#         [0, 0, 0, 1, 0],
#         [0, 0, 0, 0, 1]])
</code></pre>

username: qiminchen
cooked: <p>so instead of using <code>num_classes=5</code>, I can use <code>num_classes=6</code> and just discard the first channel by something like <code>y[:, 1:]</code></p>

username: ptrblck
cooked: <p>This could also work assuming you want to use an indexing starting at <code>1</code> (which would make index 5 the 6th class in the default use case).</p>
CUDA out of memory even when memory is free?

username: Atharva_Pandey_B20CS
cooked: <p>Hello guys,<br>
I have been running a github repository that started showing an abrupt behaviour. Some times the code worked without any error and other times it threw the below error.<br>
<code>RuntimeError: CUDA out of memory. Tried to allocate 2.26 GiB (GPU 0; 6.00 GiB total capacity; 53.78 MiB already allocated; 4.41 GiB free; 8.22 MiB cached)</code><br>
I am running the code from the following repository:</p>
<p>Repo:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/czq142857/IM-NET-pytorch">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/czq142857/IM-NET-pytorch" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/f654b994e71e61c3441b0e95257803260a7bf4344926b685ead73691ea8c5db0/czq142857/IM-NET-pytorch" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/czq142857/IM-NET-pytorch" target="_blank" rel="noopener nofollow ugc">GitHub - czq142857/IM-NET-pytorch: PyTorch 1.2 implementation of IM-NET.</a></h3>

  <p>PyTorch 1.2 implementation of IM-NET. Contribute to czq142857/IM-NET-pytorch development by creating an account on GitHub.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Command I run:<br>
python main.py --ae --getz</p>
Conditional DataLoader / variable number of DataLoaders

username: bornexmachina
cooked: <p>Hi, I try to describe my problem / my desired solution.</p>
<p>I have a large dataset, where the samples have varying conditions. I would like to create DataLoaders which load only samples of particular condition for further modeling.</p>
<p>Now the condition is coded via filename and I can extract a list of indices for condition 1 to condition M.<br>
For the next file (different type of samples) I have condition 1 to condition N.</p>
<p>Thus if I work with Subset() I need to find a way how to design a variable number of DataLoaders per type of samples.</p>
<p>What I have in mind is to create a dictionary with something like type_of_sample : num_of_conditions and then append DataLoaders with selection via Subset to a list, which I would iterate in some later stage.</p>
<p>Is there any more elegant way, which does not feel like whacky tinkering?</p>
Model not Predicting

username: rodrigoappinto
cooked: <p>Greetings everyone,</p>
<p>Currently, I am working on an architecture to enhance semi-supervised problems by utilizing two models simultaneously. These models are the UNet 3D and the Swin-Unet 3D, which is a relatively new model. When training this architecture, I have discovered that using the UNet 3D with any other model works seamlessly. However, when I use the UNet 3D with the Swin-Unet 3D, it only produces the background class from the first iteration, resulting in a zero score during validation testing. As the Swin 3D model is transformer-based, my assumption is that I require pre-training, but no pre-trained models are available yet due to its novelty. Is there any way to improve the prediction quality?</p>
<p>Thank you for your help.</p>
How to fix parameter in CNN (vgg19)

username: SutthidaMint
cooked: <p>I use VGG19 for train model but I’m curious that I should use which one and what is different</p>
<ol>
<li>model.features.eval()</li>
<li>for param in model.features.parameters():<br>
param.requires_grad = False</li>
<li>with torch.no_grad():</li>
</ol>
<p>example</p>
<pre><code class="lang-auto"># fix the weight of convolution layers
model.features.eval()

# modify classifier
model.classifier = torch.nn.Sequential(
  nn.Linear(25088, 4096),
  nn.ReLU(inplace=True),
  nn.Dropout(p=0.5, inplace=False),
  nn.Linear(4096, 4096),
  nn.ReLU(inplace=True),
  nn.Dropout(p=0.5, inplace=False),
  torch.nn.Linear(4096, 2))
</code></pre>

username: ptrblck
cooked: <ol>
<li>
<p>This sets the model in evaluation mode, which changes the behavior of some layers but does not freeze trainable parameters. After calling <code>model.eval()</code> e.g. dropout layers will be disabled and batchnorm layers will use their running stats to normalize the input activation.</p>
</li>
<li>
<p>These lines of code will freeze all properly registered trainable parameters of the <code>.features</code> submodule.</p>
</li>
<li>
<p>This guard will disable gradient calculation and will not store intermediate forward activations thus reducing the memory usage.</p>
</li>
</ol>
Reference about RLE ASCII coding

username: Sylvain_Ard
cooked: <p>Hello,<br>
the masks in soloV2 prediction are coded in ASCII RLE, have you a reference explaining how it works please ?<br>
Thank you<br>
Best regards</p>

username: Sylvain_Ard
cooked: <p>Apparently it uses the pycocotools mask encode/decode but I am curious, how it works please ?</p>
How to select a part of image and zoom it

username: DaLa
cooked: <p>Hello,</p>
<p>I know this has nothing to do with pytorch but I want to select a small region of an image and then zoom it like in the image attached. How to do that? Is it done in python or photo edting tools?</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/b/1b319dca43a61e025789f913f79ff4f59af6d143.png" alt="forum" data-base62-sha1="3Szc2Od14dUv0Ce59PZRGRRhHTt" width="545" height="394"></p>

username: ptrblck
cooked: <p><a href="https://matplotlib.org/stable/gallery/subplots_axes_and_figures/zoom_inset_axes.html">This <code>matplotlib</code> example</a> might be useful.</p>

username: DaLa
cooked: <p>Thanks a lot!<br>
I appreciate your help.</p>
Is it impossible to pickle model with hook?

username: malioboro
cooked: <p>Hi, I am facing a similar situation with these questions from different years:</p>
<ul>
<li><a href="https://discuss.pytorch.org/t/torch-save-cant-pickle-local-object/70291" class="inline-onebox">Torch.save Can't pickle local object</a></li>
<li><a href="https://discuss.pytorch.org/t/save-model-when-the-hooks-are-used/154311" class="inline-onebox">Save model when the hooks are used</a></li>
<li><a href="https://discuss.pytorch.org/t/saving-the-model-while-using-another-class-for-the-layers-of-that-model/55088" class="inline-onebox">Saving the model while using another class for the layers of that model</a></li>
</ul>
<p>And all of them don’t have an answer yet. So, Is it impossible to pickle a PyTorch model that contains hook?</p>

username: marksaroufim
cooked: <p>Interesting, the easiest way to solve this is to <code>torch.save(model.state_dict())</code></p>
<p>It might be possible to also pickle hooks if you wrap them in classes as a nice to have cc <a class="mention" href="/u/alband">@albanD</a> but regardless in the meantime <code>state_dict()</code> is the way to go</p>

username: albanD
cooked: <p>Serializing only the state_dict is the way around pickle’s limitations yes!</p>

username: malioboro
cooked: <p>I see… thank you for the clarification!</p>
Different result with the same input data in eval and no_grad model

username: oneriver
cooked: <p>Hi, I trained a model, and want to use it to predict in a eval and no_grad model. Strangely, I got different results if I feed the same data to the model twice. Here is the code snipper:</p>
<pre><code class="lang-python">    with torch.no_grad():
        for data in  generator:
            left = data['left']
            right = data['right']
            print('is training': self.model.training)
            data1 = self.model(left, right)
            data2 = self.model(left, right)
            data2 = self.predict(data2)[0]
            print(abs((data1-data2) &gt;= 0.00001).double().sum())
</code></pre>
<p>and the output is like:<br>
is training: False<br>
tensor(10878., device=‘cuda:0’, dtype=torch.float64)</p>
<p>is there any idea to debug  the problem ?</p>

username: ptrblck
cooked: <p>Small numerical mismatches are expected due to the limited numerical precision.<br>
In your code it also seems the <code>abs</code> should be applied on the subtraction, not the comparison, so you might want to fix it.</p>

username: oneriver
cooked: <p>Hi, ptrblck<br>
Thank you for your time.<br>
Yes, you are right, the abs should be applied on the subtraction.<br>
I traced the code and find that the difference came from the ConvTransposed2d module:</p>
<pre data-code-wrap="python"><code class="lang-python">        x1 = copy.deepcopy(x)
        x1 = self.conv(x1)
        x = self.conv(x)
        print('baseicconv:', (x!=x1).sum())
</code></pre>
<p>and</p>
<pre data-code-wrap="python"><code class="lang-python"> self.conv = nn.ConvTranspose2d(in_channels, out_channels, bias=False, **kwargs)
</code></pre>
<p>and the output looks like:<br>
baseicconv: tensor(158545, device=‘cuda:0’)<br>
If, as you said, the small numerical mismatches are expected, it should be always non-zero. However, as far as I know, the difference happens occasionaly:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/a/b/ab3d4b57a18786adda697070ca3347d2d6bbdea1.png" alt="image" data-base62-sha1="oqQY8giPZ4I6g43xbmU4UlTNZ1D" width="358" height="181"></p>
<p>PS: I found when set the batch-size to 1, there is no such a problem. <img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=12" title=":joy:" class="emoji" alt=":joy:" loading="lazy" width="20" height="20"></p>

username: Jiaxin_Li
cooked: <p>Hello, ptrblck. If I understand correctly, the output has a small numerical mismatch for the same input under the eval and no_grad model. But I don’t understand why the limited numerical precision will lead to this phenomenon. From my understanding, the calculation will keep the same decimal part for the same input if it is out of limited precision. Can you give a detailed explanation about this? Is there any other way to avoid this?</p>

username: ptrblck
cooked: <p>Non-deterministic algorithms could be used as described in the <a href="https://pytorch.org/docs/stable/notes/randomness.html">Reproducibility docs</a>, which would cause these small numerical errors.</p>
Import and use CC100 with torchtext

username: MarcoSteinke
cooked: <p>Hello,<br>
I am currently trying to load the CC100 dataset with torchtext.</p>
<p>I am using the following line to load it:</p>
<pre><code class="lang-auto">dataset = CC100(root="data", language_code='en')
</code></pre>
<p>and it runs and returns an object of the type “ShardingFilterIterDataPipe”. I have googled a lot, queried github and read tons of documentations, but I can not figure how to correctly load the dataset and access its elements.</p>
How to handle variable sequence input data for conv1d (TCN) model? torch.nn.utils.rnn.pack_padded_sequence does not work for non-RNN based models

username: yanzhou01
cooked: <p>Input:<br>
<code>(Batch, T, L)</code></p>
<ul>
<li>
<code>L</code> stands for <code>the length of features</code> and <strong>L holds the same</strong>.</li>
<li>
<code>T</code> stands for <code>the length of the sequence</code> and  <strong>T changes from 2 to 60</strong>.</li>
</ul>
<p>Currently, I am passing <strong>the padded inputs</strong> into the TCN model so that T is aligned with the max length 60.<br>
However, there are many zeros in the such case as of <code>T</code> was 2 before padded.</p>
<p>I tried to use the function (<code>torch.nn.utils.rnn.pack_padded_sequence</code>) before passing the data into <code>Conv1d</code> models.<br>
However, it didn’t work with the error:</p>
<pre><code class="lang-auto">TypeError: conv1d() received an invalid combination of arguments - got (PackedSequence, Tensor, Parameter, tuple, tuple, tuple, int), but expected one of:
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, tuple 
of ints padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (PackedSequence, Tensor, Parameter, tuple, tuple, tuple, int)
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, str padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (PackedSequence, Tensor, Parameter, tuple, tuple, tuple, int)
</code></pre>
<p><strong>So how can I input variable length data in a batch to the Conv1D TCN model, meanwhile, remove all the padded zeros?</strong></p>
<p>It would be much appreciated if you can give me some hints.</p>
Backpropagation with pre- and postprocessing inside model

username: ockaro
cooked: <p>Hi there! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"><br>
I wanted to integrate the pre- and postprocessing inside my pytorch model (following this thread <a href="https://discuss.pytorch.org/t/how-to-add-preprocess-into-the-model/66154/3" class="inline-onebox">How to add preprocess into the model - #3 by MarcSteven</a>) such that I have exactly the same processing steps during training and serving time.</p>
<p>For this purpose I apply a list of torch functions before and after applying the ‘actual’ processing steps of the model in the forward method:</p>
<pre><code class="lang-auto">    def forward(self, x: torch.Tensor) -&gt; torch.Tensor:
        # apply preprocessing functions
        for function in self.preprocessing_funcs:
            x = function(x)

        for layer_idx, layer in enumerate(self.layers):
            x = layer(x)
            if layer_idx == len(self.layers) - 1:
                # apply last layer activation function after last layer
                x = self.last_activation_fn(x)
            else:
                # apply hidden layer activation function after input and each hidden layer
                x = self.hidden_activation_fn(x)

        # apply postprocessing functions
        for function in self.postprocessing_funcs:
            x = function(x)

        return x
</code></pre>
<p>Of course I set all parameters used in these functions to <code>requires_grad = False</code> such that they are not trained. So as a result of this ‘in model processing’ I would need to perform the training on completely raw input and output data and this feels kind of wrong to me…<br>
Therefore my question is: Does the training work that way? Or am I missing something related to the torch autograd logic?</p>
Data to vector not transposed

username: DejNea
cooked: <p>When I transpose 2D array it works fine. but when I want to put transposed data to vector, then it’s the same as vector before transpose function.</p>
<p>CODE:</p>
<p>torch::Tensor data = torch::rand({5,10}).to(torch::kFloat32);</p>
<p>std::cout &lt;&lt; data &lt;&lt; std::endl;<br>
std::vector vectorData(data.data_ptr(), data.data_ptr() + data.numel());<br>
std::cout &lt;&lt; vectorData &lt;&lt; std::endl;</p>
<p>data = data.transpose(0, -1);</p>
<p>std::cout &lt;&lt; data &lt;&lt; std::endl;<br>
std::vector vectorData1(data.data_ptr(), data.data_ptr() + data.numel());<br>
std::cout &lt;&lt; vectorData1 &lt;&lt; std::endl;</p>

username: ptrblck
cooked: <p><code>data.transpose</code> changes the metadata of the tensor and doesn’t trigger a copy so you might want to call <code>.coniguous()</code> on the tensor:</p>
<pre><code class="lang-python">x = torch.randn(3, 4)
print(x.shape)
# torch.Size([3, 4])
print(x.stride())
# (4, 1)
print(x.data_ptr())

y = x.transpose(1, 0)
print(y.shape)
# torch.Size([4, 3])
print(y.stride())
# (1, 4)

z = y.contiguous()
print(z.shape)
# torch.Size([4, 3])
print(z.stride())
# (3, 1)
</code></pre>

username: DejNea
cooked: <p>Thank you for the explanation.</p>
Empty_cpu return TensorBase as Tensor

username: Ryan.L
cooked: <p>I’m a fresher on PyTorch and I found the following code in aten/src/ATen/native/TensorFactories.cpp:254</p>
<pre><code class="lang-auto">Tensor empty_cpu(IntArrayRef size, c10::optional&lt;ScalarType&gt; dtype_opt, c10::optional&lt;Layout&gt; layout_opt,
                 c10::optional&lt;Device&gt; device_opt, c10::optional&lt;bool&gt; pin_memory_opt, c10::optional&lt;c10::MemoryFormat&gt; memory_format_opt) {
  return at::detail::empty_cpu(size, dtype_opt, layout_opt, device_opt, pin_memory_opt, memory_format_opt);
}
</code></pre>
<p>My question is that: This function returns a Tensor object and it seems that the return value of “at::detail::empty_cpu()” is TensorBase which is base class of Tensor. (e.g. at::detail::empty_cpu() returns a base-class object and pass it to a subclass variable)</p>
<pre><code class="lang-auto">aten/src/ATen/EmptyTensor.cpp:239
TensorBase empty_cpu(
    IntArrayRef size,
    c10::optional&lt;ScalarType&gt; dtype_opt,
    c10::optional&lt;Layout&gt; layout_opt,
    c10::optional&lt;Device&gt; device_opt,
    c10::optional&lt;bool&gt; pin_memory_opt,
    c10::optional&lt;c10::MemoryFormat&gt; memory_format_opt) {
  TORCH_INTERNAL_ASSERT_DEBUG_ONLY(device_or_default(device_opt).type() == DeviceType::CPU);
  TORCH_INTERNAL_ASSERT_DEBUG_ONLY(layout_or_default(layout_opt) == Layout::Strided);

  auto pin_memory = pinned_memory_or_default(pin_memory_opt);
  auto dtype = dtype_or_default(dtype_opt);
  return empty_cpu(size, dtype, pin_memory, memory_format_opt);
}
</code></pre>
<p>Why that works? Any suggestions would be appreciated  <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=12" title=":sweat_smile:" class="emoji" alt=":sweat_smile:" loading="lazy" width="20" height="20"></p>

username: albanD
cooked: <p>Hey!</p>
<p>The Tensor and TensorBase objects are pretty much the same thing. TensorBase is just a trick we use to speed up compilation by removing the dependency of the whole library (that depends on Tensor) to all the methods on Tensor.<br>
So TensorBase is the Tensor object with all the methods defined in regular cpp files while Tensor is the same with all the methods from native_functions.yaml (and our codegen in general) added to it.</p>
<p>To make this smooth, you can trivially convert from one object to the other.</p>

username: Ryan.L
cooked: <p>I get it! Thank you very much!!! <img src="https://discuss.pytorch.org/images/emoji/apple/grin.png?v=12" title=":grin:" class="emoji" alt=":grin:" loading="lazy" width="20" height="20"></p>
Return a SimpleITK.SimpleITK.Image from dataloader

username: kabbas570
cooked: <p>Hello PyTorch Community!<br>
I want to return a SimpleITK.SimpleITK.Image from my data loader as my model needs it at some stage, I am getting following error.<br>
TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found &lt;class ‘SimpleITK.SimpleITK.Image’&gt;</p>
<p>with this code</p>
<pre><code>def __getitem__(self, index):
    img_path = os.path.join(self.images_folder, str(self.images_name[index]).zfill(3),str(self.images_name[index]).zfill(3))
  
    img_SA_path = img_path+'_SA_ES.nii.gz'
    img_SA_ES_itk = sitk.ReadImage(img_SA_path)  
 
    return img_SA_ES_itk 
</code></pre>
<p>Thanks</p>

username: ptrblck
cooked: <p>As the error message describes you would need to transform your <code>SimpleITK.Image</code> to a tensor or another data type PyTorch understands. You could alternatively also create a custom <code>collate_fn</code>, which could deal with your data type, but would eventually still need to transform it to a tensor before feeding it into a model.</p>

username: kabbas570
cooked: <p>Thanks for the reply. I cannot convert because the formation related to spacing, angles, and directions, and some related parameters are lost, and my model uses this information at a later stage.<br>
As an alternative, I am returning the path of SimpleITK.SimpleITK.Image and later will read them during the training/inference stage.</p>
<p>Cheers<br>
Abbas</p>

username: ptrblck
cooked: <p>Wouldn’t it be possible to return the transformed image as a tensor with all needed metadata together?<br>
At one point you would still need to transform the image, so I’m unsure why not in the <code>__getitem__</code> method to avoid the error.</p>

username: kabbas570
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="180090">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>return</p>
</blockquote>
</aside>
<p>Thanks, yes, this is also possible.</p>
Pytorch Lightening prediction output length does not make sense

username: chess2021
cooked: <p>Hi everyone,<br>
I have trained an NBeats model from PyTorch forecasting using [<a href="https://pytorch-forecasting.readthedocs.io/en/stable/api/pytorch_forecasting.data.timeseries.TimeSeriesDataSet.html" class="inline-onebox" rel="noopener nofollow ugc">TimeSeriesDataSet — pytorch-forecasting documentation</a>](<a href="https://Pythorch" rel="noopener nofollow ugc">https://Pythorch</a> Forecasting)<br>
using <strong>TimeSeriesDataSet</strong> method.</p>
<p>Here is the configuration of model and data:</p>
<pre><code class="lang-auto">training = TimeSeriesDataSet(
   train_data,
    time_idx="time",
    target="target",
    group_ids=["group"],
    time_varying_unknown_reals=["target"],
    max_encoder_length=100,
    max_prediction_length=100
)
</code></pre>
<pre><code class="lang-auto">train_load= training.to_dataloader(train=True, batch_size=128)

test_load= testing.to_dataloader(train=False, batch_size=128)
</code></pre>
<p><strong>My goal is to look back the past data points and forecast the next 100 points in future</strong>, and this is why I set the <strong>max_prediction</strong> = 100.</p>
<p>The test data has 234 samples.</p>
<p>I tried to look at the predicted value but the size of prediction is 35 and not 100 and this is super strange since I want to forecast the next 100 (the <strong>prediction_horizon</strong>).<br>
The prediction is:</p>
<pre><code class="lang-auto">Pred = mymodel.predict(dataloaders=test_load)

len(Pred)

35
</code></pre>
<p>Also if you look at the</p>
<pre><code class="lang-auto">print(test_load.dataset)

TimeSeriesDataSet[length=35](...)
</code></pre>
<p>You see the output also shows the length is 35 and not 100.<br>
Can someone please explain why I get 35 as the length and not 100:</p>

username: eqy
cooked: <p>The length of a tensor is generally just the extent of the first dimension:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; a = torch.empty(2, 3, 1)
&gt;&gt;&gt; len(a)
2
&gt;&gt;&gt; a = torch.empty(3, 2,  1)
&gt;&gt;&gt; len(a)
3
&gt;&gt;&gt; a = torch.empty(1, 2)
&gt;&gt;&gt; len(a)
1
</code></pre>
<p>so <code>len(Pred)</code> is not a useful indicator for the number of forecasted steps in your case unless it was the first dimension. Instead it looks like the batch size is the first dimension, as it matches the “length” of your dataset. You can check which dimension corresponds to the “steps” by printing out the shape of the prediction rather than just the length. I would also double check the number of samples in your dataset, as it having length 35 suggests that there are only 35 samples.</p>
How to use the UVA feature in PyTorch?

username: BearBiscuit05
cooked: <p>Hello, it’s my first time encountering the UVA (Unified Virtual Addressing) feature in PyTorch. I would like to understand whether, in PyTorch, we only need to use <code>torch.Tensor</code> to define a variable, and PyTorch will automatically set it as a UVA type for easy GPU access. Or do I need to set other parameters to meet this requirement?</p>

username: albanD
cooked: <p>Hi,</p>
<p>There is no UVA support in PyTorch. You should explicitly handle the move of your Tensors to and from the GPU. This more explicit control avoid subtle performance cliffs when using GPU.</p>

username: ptrblck
cooked: <p><a class="mention" href="/u/alband">@albanD</a> is right, but it might also be worth checking the <a href="https://pytorch.org/docs/stable/generated/torch.cuda.CUDAPluggableAllocator.html#torch.cuda.CUDAPluggableAllocator"><code> torch.cuda.CUDAPluggableAllocator</code></a> which might allow you to experiment with a custom allocator.</p>

username: BearBiscuit05
cooked: <p>I understand now. Thank you very much for your response.</p>
torch.distributed.reduce(op=torch.distributed.ReduceOp.AVG) behavior when few gpus have input batches while others do not

username: amoha107
cooked: <p>I am new to DDP. I am collecting the training losses from all the GPUs and reducing them at gpu_id=0 with op=torch.distributed.ReduceOp.AVG. What happens in the last iteration where only a few GPUs (not all of them) have batches. How will the torch.distributed.ReduceOp.AVG behave?</p>
<p>Ex: Assume in the last iteration of the epoch, only 2 (out of 8 GPUs) receive data from the DataLoader. Will torch.distributed.ReduceOp.AVG average the losses tensors in the 2 GPUs only as it should? Or will it assume the other 6 GPUs have zero losses and divide the sum by 8? Remember that those 6 GPUs should not have batch in the last iteration from the DistributedSampler as there is simply no more training data.</p>
<p>The same question could be made for the validation DataLoader.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="amoha107" data-post="1" data-topic="180927">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/e5b9ba/48.png" class="avatar"> amoha107:</div>
<blockquote>
<p>Remember that those 6 GPUs should not have batch in the last iteration from the DistributedSampler as there is simply no more training data.</p>
</blockquote>
</aside>
<p>I don’t think this is a valid use case as the <code>DistributedSampler</code> explicitly ensures each rank receives the same amount of data as seen <a href="https://github.com/pytorch/pytorch/blob/c6d9a0b9dd45092708be2d60a37140b7760abc75/torch/utils/data/distributed.py#L82-L93">here</a> and <a href="https://github.com/pytorch/pytorch/blob/c6d9a0b9dd45092708be2d60a37140b7760abc75/torch/utils/data/distributed.py#L106-L116">here</a>.</p>
<p>If you are somehow forcing this uneven behavior I would expect to see a hang.</p>

username: amoha107
cooked: <p>Thank you. I am beginning to understand <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
How can I use DenseNet, XceptionNet, and ResNet

username: nima_pw
cooked: <p>Hi,<br>
I have a CNN model and I want to compare its accuracy with DenseNet, XceptionNet, and ResNet.<br>
I need to train these nets with my own dataset with 30 classes.<br>
I have not had any experience with these models.<br>
I would be grateful for the guidance.</p>

username: ptrblck
cooked: <p>The <a href="https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html">Finetuning tutorial</a> might be a good resource as it walk you though the process of finetuning <code>torchvision</code> classification models.</p>
Best model to handle multi labeled data?

username: volesteni
cooked: <p>I have some data that is segmented, and some data that has bbox labels. I want to be able to leverage both label types, and my goal is to create a really strong deep classifier. what model can I use?</p>

username: ptrblck
cooked: <p>The <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html">Object Detection tutorial</a> might be a good starter which uses <code>Mask R-CNN</code> for both target types.</p>
RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x65536 and 32x20)' in my implementation of resnet50

username: peony
cooked: <p>I have defined my Resnet50 following a tutorial:</p>
<pre><code class="lang-auto">class ResNet(nn.Module):
    def __init__(self):
        super(ResNet, self).__init__()
   
        self.resnet = resnet50(pretrained=True)
        # isolate the feature blocks
        self.features = nn.Sequential(self.resnet.conv1,
                              self.resnet.bn1,
                              nn.ReLU(),
                              nn.MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False),
                              self.resnet.layer1, 
                              self.resnet.layer2, 
                              self.resnet.layer3, 
                              self.resnet.layer4)

       # average pooling layer
        self.avgpool = self.resnet.avgpool
        self.classifier = self.resnet.fc
       
   

    # gradient placeholder
        self.gradient = None
        
    def activations_hook(self, grad):
        self.gradients = grad
        
    def get_gradient(self):
        return self.gradient
    
    def get_activations(self, x):
        return self.features(x)
    
    def forward(self, x):
        x = self.features(x)
        h = x.register_hook(self.activations_hook)
        
        # complete the forward pass
        x = self.avgpool(x)
        x = x.view((1, -1))
        x = self.classifier(x)
        return x
</code></pre>
<p>However, I am getting the error</p>
<p><code>RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x65536 and 32x20)</code></p>
<p>I tried defining the classifier like the following:</p>
<pre><code class="lang-auto">
self.classifier = nn.Sequential(nn.Linear(65536, 1000), nn.Dropout())
</code></pre>
<p>but now I am getting the error <code>ValueError: Target size (torch.Size([32, 20])) must be the same as input size (torch.Size([1, 1000]))</code></p>
<p>Any suggestions for me? Thank you in advance</p>

username: ptrblck
cooked: <p>Double post from <a href="https://discuss.pytorch.org/t/how-to-register-backward-hook-at-the-last-convolutional-layer-of-resnet50/180880/3">here</a> with a follow-up.</p>
OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB

username: Basil_K_Raju
cooked: <p>OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 79.35 GiB total capacity; 32.78 GiB already allocated; 19.19 MiB free; 33.10 GiB reserved in total by PyTorch) If reserved memory is &gt;&gt; allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF</p>
<p>I am trying to fine tune whisper model.Its throwing this error.</p>
<p>I tried with small number of dataset.In collab its works in pro version but this is not working on DGX A100 server.</p>
<p>Need help on this.I am new to this.</p>
<p>I tried setting os.environ[‘PYTORCH_CUDA_ALLOC_CONF’] = ‘max_split_size_mb=100’<br>
But no chnage in the error</p>

username: ptrblck
cooked: <p>Based on the error message it seems another process is using the GPU memory and is thus not allowing PyTorch to allocate more, so check if something else is being executed or if e.g. a dead process might use the memory.</p>