akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
SSDlite annotation format

TsukeBurrel: <p>Hello, I am fine-tuning pretrained SSDlite model from torchvision.<br>
The training somehow goes well when I feed bounding boxes coordinates in absolute coordinates to the fed image size. However, the bb coordinates “theoretically” should be normalized.<br>
I’d like to know that In what format does the SSDlite in torchvision expect the annotation?<br>
In addition to it, what is the reason that the absolute coordinates make training going well while relative coordinates do not.</p>
<p>just in case, absolute coordinates look like this (162, 280, 185, 300). relative coordinates look like this (0.52, 0.8, 0.6, 0.9).</p>

ptrblck: <p>I don’t know how exactly you are feeding the bounding box coordinates to the training, but based on the <a href="https://github.com/pytorch/vision/blob/af3077e3d0c3537476ccc2ed3f29a45e56ed30ee/torchvision/models/detection/ssd.py#L138-L139">model definition</a> the output coordinates don’t seem to be normalized:</p>
<pre><code class="lang-python">        - boxes (``FloatTensor[N, 4]``): the ground-truth boxes in ``[x1, y1, x2, y2]`` format, with
          ``0 &lt;= x1 &lt; x2 &lt;= W`` and ``0 &lt;= y1 &lt; y2 &lt;= H``.
</code></pre>

TsukeBurrel: <p>thank you for the answer. I could confirm the reason.<br>
by the way, the classification loss is not decreasing very well after making annotations to absolute coordinates. Is this because of loss functions in SSDlite or something else?</p>
Error in TimeSeies Dataset PyTorch forecasting

sdg91: <p>PyTorch TimeSeries Dataset compaints about NaN; when there is no NaN in the data.</p>
<p>To confirm, created a sample dataset with no NaN values in the data of 1000 rows.</p>
<pre data-code-wrap="training"><code class="lang-plaintext">    df[lambda x: x.index &lt;= training_cutoff],
    time_idx='index',
    target='Target',
    group_ids=['Region'],
    min_encoder_length=1,
    max_encoder_length=max_encoder_length,
    min_prediction_length=1,
    max_prediction_length=max_prediction_length,
    static_categoricals=['categorical_feature'],
    time_varying_unknown_reals=['numerical_feature1'],
    target_normalizer=GroupNormalizer(groups=['Target']),
    add_relative_time_idx=True,
    add_target_scales=True,
    add_encoder_length=True)
</code></pre>
<p>It raises following error:</p>
<pre><code class="lang-auto">ValueError: 995 (100.00%) of Target_scale values were found to be NA or infinite (even after encoding). NA values are not allowed `allow_missing_timesteps` refers to missing rows, not to missing values. Possible strategies to fix the issue are (a) dropping the variable Target_scale, (b) using `NaNLabelEncoder(add_nan=True)` for categorical variables, (c) filling missing values and/or (d) optionally adding a variable indicating filled values
</code></pre>
<p>What is causing this issue and what is the possible fix?</p>
On disk writing permanently increasing

Thibaud_LE_GALL: <p>Hi here,</p>
<p>We are using Pytorch 1.10 and Cuda 11.x</p>
<p>my question would be related to the topic <a href="https://discuss.pytorch.org/t/disk-ssd-or-hdd-space-increase-decrease-ing-during-training/137107" class="inline-onebox">Disk(SSD or HDD) space increase(decrease)ing during training</a></p>
<p>We are training a model on-prem and on GCP. We are observing that the space available on disk is constantly diminishing during training. We have observed this behavior with our own algorithm and with a random basic algorithm found on the net (that i can provide if required).</p>
<p>Given the previously quoted article, we understand that it could be because we run out RAM, which would be understandable with our algorithm but not with the one pick-up from internet.</p>
<p>The writing on disk get to a problematic level with our in-house algorithm getting to 4Gb per epoch, which is something we will sort out, but the behavior of a constant increase in use of disk space is something we can’t find any explanation about.</p>
<p>We are posting here to check if this behavior is consistent with Pytorch expected behavior, and if anyone else has had any trouble with it before ?</p>
<p>Regards</p>
<p>PS: We do have all sorts of logs from shutils or du commands in different environments (mostly on GCP), We’ll provide the relevant files/lines of logs in line with the discussion</p>
ImageNet Directory: /datasets01/imagenet_full_size/061417/

samin_hamidi: <p>I am trying to run the code in this repository <a href="https://github.com/facebookresearch/deepcluster" rel="nofollow noopener">https://github.com/facebookresearch/deepcluster</a>. I am using windows 10. When I try to run main.sh, nothing happens. I opened the file and there is this line DIR="/datasets01/imagenet_full_size/061417/train". I know that they are using ImageNet but what I don’t understand is that whether the dataset is downloaded by the code and this directory is created or whether I need to download it myself? I checked other codes and they also used the same path. I appreciate any help and guidance regarding this and also running the code in general.</p>
<p>Thanks</p>

thisisnotahuman: <p>same question，have you solved this now? <img src="https://discuss.pytorch.org/images/emoji/apple/smiling_face_with_tear.png?v=12" title=":smiling_face_with_tear:" class="emoji" alt=":smiling_face_with_tear:" loading="lazy" width="20" height="20"></p>

rammalali: <p>Hello<br>
I found out that you can use it without the [/datasets01/imagenet_full_size/061417/] dataset<br>
just create your own directory : /dataset/imagenet_full_size/061417/<br>
then add 2 folders (train and test) from any dataset on kaggle</p>
<pre><code class="lang-auto">.
└── 061417
    ├── test
    │   ├── cats
    │   │   ├── cat_106.jpg
    │   │   ├── cat_109.jpg
    │   │   ├── .........
    │   └── dogs
    │       ├── dog_114.jpg
    │       ├── dog_123.jpg
    │       ├── ..........

    └── train
        ├── cats
        │   ├── cat_100.jpg
        │   ├── ........
        └── dogs
            ├── dog_0.jpg
            ├── .........
</code></pre>
<p>check this out <a href="https://pytorch.org/vision/main/generated/torchvision.datasets.ImageFolder.html" rel="noopener nofollow ugc">IMAGEFOLDER - Pytorch configuration</a></p>
Why use PyTorchBot?

scruel: <p>I wonder why your team decided to develop and use PyTorchBot to manage GitHub pull requests?<br>
Are there any advantages to using this bot? Rather than just use GitHub built-in features?</p>
<p>Consider that some people might be confused about the bot’s action, like:</p><aside class="quote" data-post="3" data-topic="174567">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/b5a626/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/pull-request-closed-after-approval/174567/3">Pull Request closed after approval</a> 
  </div>
  <blockquote>
    Thanks for the prompt response, but then why is the PR status ‘closed’ and not ‘merged’?
  </blockquote>
</aside>


ptrblck: <p>Which built-in features are you referring to?<br>
The bot allows you to easily rebase, merge when all tests pass, force a merge and explain why a failing test can be ignored, revert PRs etc. with minimal effort.</p>

marksaroufim: <p>One of the main reasons was that Github didn’t support a merge queue until recently <a href="https://github.blog/2023-07-12-github-merge-queue-is-generally-available/" class="inline-onebox">GitHub merge queue is generally available - The GitHub Blog</a> and with the volume of PRs that get merged to Github it would be impossible to merge anything without it</p>
<p>Other nice features rebases and reverts which the team uses quite a bit and of course ignoring tests which you typically can’t do unless you’re an admin</p>
Automatically figure out dimension of linear layer

Gears_Gears: <p>Hi!</p>
<p>I have a sequential layer with convolutions and max pooling, followed by a linear layer<br>
The problem is, if I change the settings of convolutions or max pooling, the incoming dimension of the linear layer will change</p>
<p>What would be a good way to automatically check and find the right dimension, so the linear layer always works?</p>
<p>Thanks very much!<br>
P.S. I tried LazyLinear but because the weights are uninitialized, I can’t initialize them with my initialize-weights function. So I’m trying to use a nn.Linear, which requires me to know the input dimension before creating it</p>

ptrblck: <p>You could still use the lazy layer and initialize it afterwards once the parameters were created.</p>

Gears_Gears: <p>Thanks a lot for your help ptrblck! I see this in the error message as well. I think I will have an initialize method like this: (<em>I’m not sure about the call to forward()</em> since it’s supposed to be done by proxy of the nn.Module as a callable)</p>
<pre><code class="lang-auto"># inside the class hosting the model
initialize(self, dummy_input):
    self.forward(model) # perhaps I should have a compute() where I have all the computation? And then for my forward() I just call the compute()?
    self.network_1.apply(fn) # I have a bunch of networks that I can't put under the same nn.Sequential because I'm inserting extra calculation steps between their usage
    self.network_2.apply(fn)
    ...
</code></pre>

ptrblck: <p>Yes, having and internal <code>initialize</code> method should work, but I assume you want to pass the <code>dummy_input</code> to the model instead of the undefined <code>model</code> variable.<br>
So something like this?</p>
<pre><code class="lang-python">def initialize(self, dummy_input):
    self(dummy_input)
    ...
</code></pre>
<p>You don’t need to call <code>self.forward</code> and can just pass the input to <code>self</code> in the same way you would use <code>model(input)</code> instead of <code>model.forward(input)</code>, although it will most likely not matter here.</p>

Gears_Gears: <p>Thank you! I know what to do now!</p>
(some form of) Pattern Matching for Tensors

Gears_Gears: <p>Hi!</p>
<p>I’m trying to check the shape of tensors (N, C, H, W) for my code<br>
I know N will change, but I want to check that C, H, W are correct<br>
What would be a good way to do this?</p>
<p>Thanks a lot!</p>

ptrblck: <p><code>tensor.size()</code> or <code>tensor.shape</code> will return a <code>torch.Size</code> object which you can then use to compare against a reference as seen here:</p>
<pre><code class="lang-python">x = torch.randn(2, 3, 4, 5)

print(x.size() ==  torch.Size([3, 4, 5]))
# False
print(x.size()[1:] == torch.Size([3, 4, 5]))
# True
</code></pre>

Gears_Gears: <p>Thanks very much for your help ptrblck! This method will help me catch many errors!</p>
Problems with the combination of torch.jit.trace and torch.complie

MartinZhang: <pre><code class="lang-auto">import torch
from torchvision import models

net = models.resnet18(pretrained=True)
cnet = torch.compile(net)
data = torch.randn(1, 3, 224, 224)
tnet = torch.jit.trace(cnet, data)
</code></pre>
<pre><code class="lang-auto">ERROR:Graphs differed across invocations!
</code></pre>
<p><strong>Is this normal？Why?</strong></p>

ptrblck: <p>I don’t think tracing a compiled model is supported and you should use <code>torch.compile</code> only in recent PyTorch releases.</p>

marksaroufim: <p>Yeah <code>torch.compile()</code> and <code>torch.jit()</code> do not compose, if you’re looking for serialization I’d suggest you take a look at <code>torch.export.export()</code> instead</p>

spiegelball: <p>Can a torch.export.export() saved model be loaded on the C++ side now?</p>
PyTorch tensor assignment

bltzmann: <p>Hi everyone, I’m trying to assign elements from a 6D tensor x, to a 4D tensor y. I would like to optimize this process. I write here the code with for cycles:<br>
for r in range(R):<br>
for k in range(K):<br>
y[r,k,:,:] = x[r,r,k,k,:,:]</p>
<p>the dimensions of tensor are:<br>
x.size()=(R,R,K,K, N, M)<br>
y.size()=(R,K,N,M)<br>
How can I do this avoiding explicit for cycles?</p>

ptrblck: <p>Direct indexing should work:</p>
<pre><code class="lang-python">R, K, N, M = 2, 3, 4, 5

x = torch.randn(R, R, K, K, N, M)
y = torch.zeros(R, K, N, M)

# reference
for r in range(R):
    for k in range(K):
        y[r,k,:,:] = x[r,r,k,k,:,:]

# direct indexing
z = x[torch.arange(R)[:, None], torch.arange(R)[:, None], torch.arange(K), torch.arange(K)]

# comapre
print((z == y).all())
# tensor(True)
</code></pre>
Cannot install Pytorch 1.0.1

atharva_deore: <p>I am working on project that requires pytorch 1.0.1 and python 3.6.5 . So I have downloaded the wheel file but when I install using pip install torch-1.0.1-cp36-cp36m-linux_x86_64.whl but i am getting the following  error: ERROR: torch-1.0.1-cp36-cp36m-linux_x86_64.whl is not supported wheel on this platform .I have created conda environment that has python 3.6.5 .When I check python --version I get 3.6.5 but when I check with python3 --version I am getting 3.8.10 .Can somebody help me with this problem please?</p>
Model distillation with mixed-precision training

pepper8362: <p>Hi,</p>
<p>I am doing model distillation with mixed-precision training. But my code gives me some strange results: the results seem very different from training without distillation even if the distillation weight is set to zero. I wonder if I did something wrong in my use of mixed-precision training. The code is as follows</p>
<pre><code class="lang-auto">with torch.cuda.amp.autocast(enabled=True):
    outputs_student = model_student(inputs, targets)
    with torch.no_grad():
        outputs_teacher = model_teacher(inputs, targets)
    loss_distillation = distill_loss(outputs_student, outputs_teacher)
    loss_student = some_loss(outputs_student)    
    loss = loss_student + weight*loss_distillation 
optimizer.zero_grad()
scaler.scale(loss).backward()
scaler.step(self.optimizer)
scaler.update()
</code></pre>
<p>where <code>scaler = torch.cuda.amp.GradScaler(enabled=True)</code>. As mentioned above, I tried to set <code>weight</code> to 0, but the result was still very different from training using <code>loss_student</code> only. Is there anything wrong in my use of mixed-precision training?</p>

pepper8362: <p>Just a follow-up. I find the gradient of some layer weights is nan after <code>scaler.scale(loss).backward()</code>, although the weights look find after <code>scaler.step(self.optimizer)</code>. This seems only happen for mixed precision training. Is this normal?</p>

ptrblck: <p>Yes, since invalid gradients can be created by the scaler, which will trigger the optimizer to skip this parameter update as described in the docs.</p>

pepper8362: <p>Thanks for your reply. Just another question relevant to this one. In the code below:</p>
<pre><code class="lang-auto">outputs_student = model_student(inputs, targets)
with torch.no_grad():
    outputs_teacher = model_teacher(inputs, targets)
loss_distillation = distill_loss(outputs_student, outputs_teacher)
</code></pre>
<p><code>model_student</code> is actually wrapped by <code>torch.nn.parallel.DistributedDataParallel</code> for multi-GPU training. Given that the teacher model is in eval mode and does not require gradient during training, is it ok to not wrap <code>model_teacher</code> with DDP?</p>

ptrblck: <p>Yes, I believe this should be fine since you are computing a static target with it used to compute the loss. <code>DDP</code> won’t need to synchronize any data.</p>
How to let the two input nodes of `add` op share the same quantization params(scale and zero point)?

Wenlong_Shi: <p>If the two input nodes of <code>add</code> op share the same quantization params(scale and zero point), the calculation of <code>quantized_add</code> will be simpler. Can multiple nodes share the same observer in FX Graph Mode Quantization?</p>

jerryzh168: <p>we don’t support this in fx, but the new quant flow will support this: <a href="https://pytorch.org/tutorials/prototype/pt2e_quantizer.html#annotate-operators-that-shares-quantization-params" class="inline-onebox" rel="noopener nofollow ugc">How to Write a Quantizer for PyTorch 2 Export Quantization — PyTorch Tutorials 2.1.0+cu121 documentation</a></p>

Wenlong_Shi: <p>Thanks for your reply.</p>
Writer.add_graph(model)

Rakshith_V: <p>I’m trying to pass the model to tensorboard.<br>
Model takes multiple inputs such as input_ids,attention,etc.</p>
<p>So, how exactly I should use writer.add_graph in my case to make it work</p>

Chris_Ding: <p>I have the same issue.</p>

Rakshith_V: <p>You just need to call your model with inputs (could be a random input to the model)<br>
It’s equivalent of calling forward function</p>
Whats the limit of data output by output node?

gaurabdhg: <p>My polynomial regression model is supposed to output the coordinates of vertices of polygons from images. Since there can be multiple polygons on the same image and the number of vertices in a polygon can vary, I’ve got 2 output nodes, to output the x and y coordinates.<br>
I’ve padded the labels to max _size of the labels, to create uniform size labels.</p>
<p>The model outputs for x and y, but they’re of different shape than the label and requires padding for loss calculation.<br>
Will this (after the model is fully trained)  limit output size to max_size?</p>
<p>I’m working with a CNN-polynomial regression model.</p>

J_Johnson: <p>To clarify, you are feeding the model an image, and want the coordinates of any vertices. Correct?</p>
<p>Suppose there are 20 vertices in one image, 15 in the second and 2 in the third. In that case, I would make the output size of the model 30% greater than the largest size needed to display the coordinates for all vertices. In the example above, this would be (20*2)*1.3 = 52 outputs. And then make sure the label coordinates are ordered according to their x, y position, and fill in any not present with -1(Or some value never present in the scaled labels, otherwise).</p>
<p>Alternatively, you could make the output of the image size width and height and treat it as an object detection, such that you have 0s where there are no vertices and 1s where there are.</p>
Sine activation missing in pytorch.nn

robinho: <p>The sine activation function, which becomes useful recently, is missing in pytorch.nn. How to remedy this?</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/robinho">@robinho</a>,</p>
<p>You can find the sine function here: <a href="https://pytorch.org/docs/stable/generated/torch.sin.html" class="inline-onebox" rel="noopener nofollow ugc">torch.sin — PyTorch 2.1 documentation</a></p>

robinho: <p>Hi <a class="mention" href="/u/alphabetagamma96">@AlphaBetaGamma96</a>  thanks a lot, but can this be used as an activation function directly by writing torch.sin in torch.nn?</p>

AlphaBetaGamma96: <p>You can just do,</p>
<pre><code class="lang-auto">torch.sin(x)
</code></pre>
<p>you don’t have to initialize from <code>torch.nn</code>, autograd will track the gradient.</p>

robinho: <aside class="quote no-group" data-username="AlphaBetaGamma96" data-post="4" data-topic="189470">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/3da27b/48.png" class="avatar"> AlphaBetaGamma96:</div>
<blockquote>
<p><code>torch.sin(x)</code></p>
</blockquote>
</aside>
<p>but in the following code:</p>
<p>import torch.nn as nn<br>
nn.Sequential(nn.Linear(1,10),nn.Sigmoid(),nn.Linear(10,1,bias=False))</p>
<p>it doesn’t seem possible to replace nn.Sigmoid() with torch.sin(x) or sth similar directly?</p>

AlphaBetaGamma96: <p>If you want to define your model within a <code>nn.Sequential</code> object, you can either</p>
<ol>
<li>re-define your model as an <code>nn.Module</code> object and you can use <code>torch.sin(x)</code> like I stated above</li>
<li>Define an <code>nn.Module</code> object for <code>torch.sin</code> and pass it to the <code>nn.Sequential</code> object.</li>
</ol>
<pre><code class="lang-auto">class Sin(nn.Module):
  def __init__(self):
    pass
  def forward(self, x):
    return torch.sin(x)
</code></pre>
<p>Then replace <code>nn.Sigmoid()</code> with <code>Sin()</code></p>

robinho: <p>many thanks! I tried the 2nd method writing the same class only to get the following error message:<br>
‘Sin’ object has no attribute ‘_parameters’</p>

J_Johnson: <p>Try deleting the <code>__init__</code>.</p>
<pre><code class="lang-auto">class Sin(Module):
    def forward(self, input: Tensor) -&gt; Tensor:
        return torch.sin(input)
</code></pre>
Pytorch register full backward pre hook can not change the grad of the module

Qizhou: <p>The simplest code is follows:</p>
<pre><code class="lang-auto">import torch
from torch import nn

def hook(module, grad_output):
    grad_output = [grad_output[0] + 9999999]  
    return grad_output

l1 = nn.Linear(2,3)
l1.register_full_backward_pre_hook(hook)

x = torch.rand(5,2)
# x.requires_grad_(True)
y = l1(x)
loss = y.sum()
loss.backward()

print(l1.weight.grad)
</code></pre>
<p>pytorch version is 2.0.1+cu118. Is there a bug? If I dont annotate the <code>x.requires_grad_(True)</code>, it  would be right.</p>

ptrblck: <p>From the docs:</p>
<blockquote>
<p>The hook should not modify its arguments, but it can optionally return a new gradient with respect to the output that will be used in place of grad_output in subsequent computations.</p>
</blockquote>

Qizhou: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="189533">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>subsequent</p>
</blockquote>
</aside>
<p>Thank you for your reply!</p>
<p>I think the output of <code>l1.weight.grad</code> in this code should be a large tensor since I have added a big number into the grad of  <code>l1</code> layer’s output at the backward_pre stage.</p>
<p>However, when I run the code:</p>
<pre><code class="lang-auto">import torch
from torch import nn

def hook(module, grad_output):
    a = [grad_output[0] + 9999999]  
    return a

l1 = nn.Linear(2,3)
l1.register_full_backward_pre_hook(hook)

x = torch.ones(5,2)
# x.requires_grad_(True)
y = l1(x)
loss = y.sum()
loss.backward()

print(l1.weight.grad)
</code></pre>
<p>It output:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/0/9/098a841d76cdb217fed0dfcb4bcebb26afa76e61.png" alt="image" data-base62-sha1="1mp47PXmiVZndcsZUNFDbfGuHjb" width="179" height="75"></p>
<p>When I run:</p>
<pre><code class="lang-auto">import torch
from torch import nn

def hook(module, grad_output):
    a = [grad_output[0] + 9999999]  
    return a

l1 = nn.Linear(2,3)
l1.register_full_backward_pre_hook(hook)

x = torch.ones(5,2)
x.requires_grad_(True)
y = l1(x)
loss = y.sum()
loss.backward()

print(l1.weight.grad)
</code></pre>
<p>It output:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/0/5/054a9a5e0e0321c7b34c8a9aaf58f33a0c02f8c3.png" alt="image" data-base62-sha1="KOdRhgmxFNTziUQpUgwyXv3FSj" width="316" height="86"></p>
<p>The only difference between the two codes is in <code>x.requires_grad_(True)</code>. I think that the results of two runs should both be tensors of 50000000, or, at least the results of both should be the same.</p>
SeLU Activation Function Implementation In GRUCell

Tahseen_Abbas_Hyder: <p>The actual task is to replace the tanh_() at line#799 with SeLU activation function in new_gate of gru_cell. The following code block is the RNN.cpp file from PyTorch’s Official Repo.</p>
<pre><code class="lang-auto">template &lt;typename cell_params&gt;
struct GRUCell : Cell&lt;Tensor, cell_params&gt; {
  using hidden_type = Tensor;

  hidden_type operator()(
      const Tensor&amp; input,
      const hidden_type&amp; hidden,
      const cell_params&amp; params,
      bool pre_compute_input = false) const override {
    if (input.is_cuda() || input.is_xpu()) {
      TORCH_CHECK(!pre_compute_input);
      auto igates = params.matmul_ih(input);
      auto hgates = params.matmul_hh(hidden);
      auto result = at::_thnn_fused_gru_cell(
          igates, hgates, hidden, params.b_ih(), params.b_hh());
      // Slice off the workspace argument (it's needed only for AD).
      return std::move(std::get&lt;0&gt;(result));
    }
    const auto chunked_igates = pre_compute_input
        ? input.unsafe_chunk(3, 1)
        : params.linear_ih(input).unsafe_chunk(3, 1);
    auto chunked_hgates = params.linear_hh(hidden).unsafe_chunk(3, 1);
    const auto reset_gate =
        chunked_hgates[0].add_(chunked_igates[0]).sigmoid_();
    const auto input_gate =
        chunked_hgates[1].add_(chunked_igates[1]).sigmoid_();
    **const auto new_gate =
        chunked_igates[2].add(chunked_hgates[2].mul_(reset_gate)).tanh_();**
    return (hidden - new_gate).mul_(input_gate).add_(new_gate);
  }
};

</code></pre>
<p><strong>The <strong>new_gate</strong> is the Tensor. How we can implement a custom function (if it’s not already implemented officially yet) to iterate over the Tensor and apply the Selu activation function on them ??</strong></p>
Why test cases can't be all passed successfully?

scruel: <p>Following the guide of <a>https://github.com/pytorch/pytorch/blob/main/CONTRIBUTING.md#python-unit-testing</a> documentation, I ran the command <code>python test/run_test.py</code>, during the run period, I can see failures frequently…<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/f/af8be60e6483852216a344e0164ca231be0f7eec.png" data-download-href="https://discuss.pytorch.org/uploads/default/af8be60e6483852216a344e0164ca231be0f7eec" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/f/af8be60e6483852216a344e0164ca231be0f7eec.png" alt="image" data-base62-sha1="p2XhUdG87IS1HXL9RdbRtK5fdIU" width="690" height="306" data-dominant-color="212121"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">884×393 17.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Also, for testing the specific test case, I tried the demo commands (like <code>python test/test_jit.py TestJit.test_Sequential</code>) in documentation, those are works fine, however according to the following section:</p>
<blockquote>
<p>You can narrow down what you’re testing even further by specifying the name of an individual test with <code>TESTCLASSNAME.TESTNAME</code>. Here, <code>TESTNAME</code> is the name of the test you want to run, and <code>TESTCLASSNAME</code> is the name of the class in which it is defined.<br>
I tried to execute the command <code>python test/test_linalg.py TestLinalg.test_linalg_matrix_exp_boundary_cases</code>, and the error occurred:</p>
</blockquote>
<pre><code class="lang-auto">E
======================================================================
ERROR: TestLinalg (unittest.loader._FailedTest)
----------------------------------------------------------------------
AttributeError: module '__main__' has no attribute 'TestLinalg'

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)
</code></pre>
<p>The problem here is that, when I tried to run all tests, I will always get the failures, and I don’t think that I should go through to fix those failed tests, because even for the unchanged clone repo, the results are the same.</p>
<p>For the single test case, still don’t know about how to fix to make it runnable, so can anyone give me some helps? Thanks!</p>
<p>Version info:<br>
torch                         2.0.0</p>
How should I use Scikit-Learn Scalers inside my pytorch dataset

adiv5: <p>Hello,<br>
I am having trouble understanding how could I use scikit-learn scalers inside my pytorch dataset.</p>
<p>So, I have a working training loop where for each epoch, we run through all batches in train_loader and then use the feature vector as input to the model (‘Resnet50’) . Followed by loss calculation backward pass and optimizer updates. Pretty standard training biolerplate code.</p>
<p>However now i am interested to pass scaled version of my feature vector. for that I tried introducing MinMaxScaler from sklearn inside my custom dataset. It does transform the data as needed, however i couldnt return the trained MinMaxScaler object as Pytorch Dataset dont allow to return objects.</p>
<p>So now If i wish to perform any kind of scaling , how do i proceed, should i return the feature vector as is for train and validation, and use scaler inside my training loop, or should i somehow try to do it inside dataset class??</p>

ptrblck: <aside class="quote no-group" data-username="adiv5" data-post="1" data-topic="189530">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/adiv5/48/63308_2.png" class="avatar"> adiv5:</div>
<blockquote>
<p>for that I tried introducing MinMaxScaler from sklearn inside my custom dataset. It does transform the data as needed, however i couldnt return the trained MinMaxScaler object as Pytorch Dataset dont allow to return objects.</p>
</blockquote>
</aside>
<p>Why do you want to return the scaler object? Wouldn’t you apply the scaling in the <code>__getitrm__</code> as any other transformation and return the transformed tensor?</p>
TorchLib inference works slower than ONNX and JIT

ifedorov: <p>Hi, I’m trying to implement C++ inference for huggingface bert (mini). Here’s are my code snippets to measure elapsed time:</p>
<pre><code class="lang-auto">#include &lt;torch/script.h&gt; // One-stop header.
#include &lt;iostream&gt;
#include &lt;memory&gt;

using namespace std::chrono;
using namespace std;

int main(int argc, const char* argv[]) {
  if (argc != 2) {
    std::cerr &lt;&lt; "usage: example-app &lt;path-to-exported-script-module&gt;\n";
    return -1;
  }


  torch::jit::script::Module module;
  try {
    // Deserialize the ScriptModule from a file using torch::jit::load().
    module = torch::jit::load(argv[1]);
  }
  catch (const c10::Error&amp; e) {
    std::cerr &lt;&lt; "error loading the model\n";
    return -1;
  }

  std::cout &lt;&lt; "ok\n";

  std::vector&lt;torch::jit::IValue&gt; inputs;
  inputs.push_back(torch::ones({1, 150}, at::kLong));
  torch::NoGradGuard no_grad;
  auto start = high_resolution_clock::now();
  auto output = module.forward(inputs);
  auto stop = high_resolution_clock::now();
  auto duration = duration_cast&lt;milliseconds&gt;(stop - start);

  cout &lt;&lt; duration.count() &lt;&lt; std::endl;
  std::cout &lt;&lt; output;
}
</code></pre>
<pre><code class="lang-auto">import torch
import numpy as np
from time import time
import onnxruntime as onnx_rt

onnx_rt = onnx_rt.InferenceSession('clf_mini.onnx', providers=['CPUExecutionProvider'])
model = torch.jit.load('jit_mini.pt')
model.eval()
inp = torch.zeros((1, 150), dtype=torch.int64)

N = 1000
times = []
for i in range(N):
    t0 = time()
    r = onnx_rt.run(None, {'input_values': np.zeros((1, 150), dtype=np.int64)})[0]
    dt = time()-t0
    times.append(dt)
print('ONNX Runtime:', 1000 * sum(times) / N, 'ms')

N = 1000
times = []
for i in range(N):
    t0 = time()
    r = model(torch.zeros((1, 150), dtype=torch.int64))
    dt = time()-t0
    times.append(dt)
print('Torch JIT Runtime:', 1000 * sum(times) / N, 'ms')

</code></pre>
<p>Results: ONNX and JIT ~5 ms, TorchLib ~70 ms.</p>
<p>Is this expected behavior or I did something wrong?</p>

ifedorov: <p>I found the problem: it seems that at the first 1-2 forwards in C++ torchlib performs some kind of warmup or benchmarking, so it takes longer. After that, it has the same execution time as python.</p>
PretrainedConfig with non-serializable JSON attribute

Garvey: <p>I am building a custom model that consist of multiple pre-trained models (e.g. BERT). I want to have a global <code>PretrainedConfig</code> that control the setup these group of pre-trained models in a higher level.</p>
<p>Examples such as:</p>
<pre><code class="lang-auto">from transformers import PretrainedConfig, BertConfig, GPT2Config
class GlobalConfig(PretrainedConfig):
    def __init__(self, model_a_cfg=None, model_b_cfg=None, *inputs, **kwargs):
        super().__init__()
        self.model_a_cfg = model_a_cfg
        self.model_b_cfg = model_b_cfg

class Config1(BertConfig):
    def __init__(self, *inputs, **kwargs):
        super().__init__()

class Config2(GPT2Config):
    def __init__(self, *inputs, **kwargs):
        super().__init__()
</code></pre>
<p>This results in:</p>
<pre><code class="lang-auto">bert_config = Config1()
gpt_config = Config2()
GlobalConfig(bert_config, gpt_config)

# TypeError: Object of type Config1 is not JSON serializable
</code></pre>
<p>I expect the <code>GlobalConfig</code> work out like this</p>
<pre><code class="lang-auto">from transformers import BertModel, GPT2Model

class GlobalModel(PreTrainedModel):
    def __init__(self, config):
        super().__init__(config)
        self.config = config
        self.model_a = BertModel(config.model_a_cfg)
        self.model_b = GPT2Model(config.model_b_cfg)

config = GlobalConfig(bert_config, gpt_config)
global_model = GlobalModel(config)
</code></pre>
<p>I’d like to avoid managing multiple configurations for a large combined model as it can be cumbersome. What would be the most effective solution to this issue?</p>
How to make the RegionProposalNetwork generate more proposals in FasterRCNN?

johnny69: <p>I’m trying to update the proposal losses function of MaskRCNN to increase the recall. I’m trying to do this by adding a positive weight to the BCE function</p>
<p>How I create my proposal losses function:</p>
<pre><code class="lang-auto">CLASS_WEIGHTS = torch.tensor([50])

def compute_loss(
        objectness: Tensor, pred_bbox_deltas: Tensor, labels: List[Tensor], regression_targets: List[Tensor]
    ) -&gt; Tuple[Tensor, Tensor]:
    """
    Args:
        objectness (Tensor)
        pred_bbox_deltas (Tensor)
        labels (List[Tensor])
        regression_targets (List[Tensor])

    Returns:
        objectness_loss (Tensor)
        box_loss (Tensor)
    """

    sampled_pos_inds, sampled_neg_inds = model.rpn.fg_bg_sampler(labels)
    sampled_pos_inds = torch.where(torch.cat(sampled_pos_inds, dim=0))[0]
    sampled_neg_inds = torch.where(torch.cat(sampled_neg_inds, dim=0))[0]

    sampled_inds = torch.cat([sampled_pos_inds, sampled_neg_inds], dim=0)

    objectness = objectness.flatten()

    labels = torch.cat(labels, dim=0)
    regression_targets = torch.cat(regression_targets, dim=0)

    box_loss = F.smooth_l1_loss(
        pred_bbox_deltas[sampled_pos_inds],
        regression_targets[sampled_pos_inds],
        beta=1 / 9,
        reduction="sum",
    ) / (sampled_inds.numel())

    objectness_loss = F.binary_cross_entropy_with_logits(objectness[sampled_inds], labels[sampled_inds],
                                                        pos_weight=CLASS_WEIGHTS # USE CLASS WEIGHT HERE
                                                        )
    return objectness_loss, box_loss
</code></pre>
<p>Then how I set the model to use this proposal losses function:</p>
<pre><code class="lang-auto">model = maskrcnn_resnet50_fpn(weights=MaskRCNN_ResNet50_FPN_Weights.DEFAULT)
model.rpn.compute_loss = compute_loss
</code></pre>
<p>When I train the model now:</p>
<ul>
<li>the <code>compute_loss</code> function is called, as expected</li>
<li>the <strong>loss</strong> increases significantly (e.g. before it was 1, now it is like 50), as expected</li>
<li>BUT the <strong>recall</strong> and precision stay around the same (e.g. recall stagnates around 0.55 after training for several epochs)</li>
</ul>
<p>Why is this the case? How do I get the recall to improve (i.e. how do I generate more proposals)?</p>
<p><em>FYI: I already tried setting the score threshold to 0, this didn’t do anything either…</em></p>
Fast shape inference

anwang2009: <p>Hi, I have a use case where I’d like to get the output shape of torch operations.</p>
<p>I could use Fake Mode to do this, but it’s quite slow (the overhead with fake mode for 1000 torch.ones ops is about 200-250ms), which makes this not an option for me. I’ve heard that meta shape inference is also an option, but also that it doesn’t have complete coverage.</p>
<p>Linking against libtorch is also not desired, but if there’s a consistent shape inference story there I’m willing to consider it. It looks like this exists <a href="https://github.com/pytorch/pytorch/issues/54982" class="inline-onebox" rel="noopener nofollow ugc">[RFC] A PyTorch Tensor Shape DSL For Symbolic Shape Inference · Issue #54982 · pytorch/pytorch · GitHub</a> and I would love to be able to compute with symbolic shapes for shape inference, but it looks like this work has been dropped? I also see a bunch of shape inference routines in <code>ts_native_functions.cpp</code>, but I don’t see them used anywhere – and it looks like part of the LazyTensor backend. What’s the story there?</p>
<p>So I’m wondering what my options are:</p>
<ul>
<li>write my own shape inference in C++ for the ops I care about. A lot of overhead but I’d only have to do this once</li>
<li>bite the bullet and link against libtorch, use the LazyTensor shape inference implementations somehow</li>
</ul>
<p>Any thoughts? Thanks!</p>
Pytorch compilation error on high sierra

nee_go: <p>I am trying to compile pytorch on macos 10.13.6 , xcode 10.1 , cuda 10.1, python 3.11.5. I have tried to compile pytorch branch v1.4.0 but got some errors at 99% :(.<br>
I tried branch lts/release/1.8 and again got errors at 99% -</p>
<p><code>**/pytorch/torch/csrc/Stream.cpp:108:27:** **error:** **expression is not assignable**</code></p>
<p><code>Py_TYPE(&amp;THPStreamType) = &amp;PyType_Type;</code></p>
<p><code>**~~~~~~~~~~~~~~~~~~~~~~~ ^**</code></p>
<p><code>[ 97%] Building CXX object </code>modules/detectron/CMakeFiles/caffe2_detectron_ops_gpu.dir/roi_pool_f_op.cc.o<br>
<code>[ 97%] Building CXX object test_api/CMakeFiles/test_api.dir/tensor.cpp.o </code>1 warning and 1 error generated.<br>
<code>make[2]: *** [caffe2/torch/CMakeFiles/torch_python.dir/csrc/Stream.cpp.o] Error 1 </code>make[2]: *** Waiting for unfinished jobs…<br>
<code>[ 97%] Building CXX object modules/detectron/CMakeFiles/caffe2_detectron_ops_gpu.dir/sample_as_op.cc.o </code>[ 98%] Building CXX object test_api/CMakeFiles/test_api.dir/torch_include.cpp.o<br>
<code>[ 98%] Building CXX object modules/detectron/CMakeFiles/caffe2_detectron_ops_gpu.dir/select_smooth_l1_loss_op.cc.o </code>[ 98%] Building CXX object modules/detectron/CMakeFiles/caffe2_detectron_ops_gpu.dir/sigmoid_cross_entropy_loss_op.cc.o<br>
<code>[ 98%] Building CXX object modules/detectron/CMakeFiles/caffe2_detectron_ops_gpu.dir/sigmoid_focal_loss_op.cc.o </code>[ 98%] Building CXX object test_api/CMakeFiles/test_api.dir/parallel.cpp.o<br>
<code>[ 98%] Building CXX object modules/detectron/CMakeFiles/caffe2_detectron_ops_gpu.dir/smooth_l1_loss_op.cc.o </code>[ 98%] Building CXX object modules/detectron/CMakeFiles/caffe2_detectron_ops_gpu.dir/softmax_focal_loss_op.cc.o</p>
<p>114 warnings generated.</p>
<p><code>make[1]: *** [caffe2/torch/CMakeFiles/torch_python.dir/all] Error 2 </code>make[1]: *** Waiting for unfinished jobs…<br>
<code>[ 98%] Building CXX object modules/detectron/CMakeFiles/caffe2_detectron_ops_gpu.dir/spatial_narrow_as_op.cc.o </code>[ 98%] Building CXX object modules/detectron/CMakeFiles/caffe2_detectron_ops_gpu.dir/upsample_nearest_op.cc.o<br>
<code>[ 98%] **Linking CXX executable ../bin/test_api** </code>[ 98%] Built target test_api<br>
<code>[ 98%] **Linking CXX shared library ../../lib/libcaffe2_detectron_ops_gpu.dylib** </code>[ 98%] Built target caffe2_detectron_ops_gpu</p>
<p><code>make: *** [all] Error 2</code></p>
<p>Could you help me to compile and build pytorch on my set up?</p>
<p>Thank you</p>
Wheel packages are gone from download.pytorch.org

bfc: <p>Hi!</p>
<p>At work we noticed that the wheel package for PyTorch 1.11.0 with CUDA 10.2 is gone. Until yesterday we were using <a href="https://download.pytorch.org/whl/cu102/torch-1.11.0%2Bcu102-cp39-cp39-linux_x86_64.whl" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu102/torch-1.11.0%2Bcu102-cp39-cp39-linux_x86_64.whl</a> to directly install the package but from today it is not available any more.</p>
<p>Another option was to use</p>
<pre><code class="lang-auto">pip install torch==1.11.0+cu102 torchvision==0.12.0+cu102 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu102
</code></pre>
<p>but <a href="https://download.pytorch.org/whl/cu102" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu102</a> is gone too.</p>
<p>We use pip to manage the requirements and for various reasons using conda is not an option.</p>
<p>This is causing us a lot of trouble so we would appreciate some information how to manage this without compiling PyTorch from source.</p>
<p>Thank you!</p>

bfc: <p>It seems they are up again. I don’t know what happened but it would be good to give some notice if there is any maintenance work going on.</p>

ptrblck: <p>The binaries were deleted by mistake and <a class="mention" href="/u/malfet">@malfet</a> fixed it already <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
Cannot import modules form torchrl

Hermes_Morales: <p>I am not able to import any torchrl module even if I have successfully installed it. I am using Google Collaboratory.<br>
After a web search I have correctly working tensordict, that is indicated as an error source.<br>
I have discovered that is specifically associated with the Multiagent Reinforcement Learning Tutorial.</p>
<pre><code>https://pytorch.org/rl/tutorials/multiagent_ppo.html
</code></pre>
<p>It works correctly in other tutorials. Misteriously for me.<br>
Thank for any comment to fix this problem</p>

vmoens: <p>Thanks for reporting this! Do you mind giving the error message that you’re getting?</p>

Hermes_Morales: <p>Thank you for your rapid answer.</p>
<hr>
<pre><code>  ImportError                               Traceback (most recent call last)
   &lt;ipython-input-2-eaa873b138d0&gt; in &lt;cell line: 20&gt;()
   18 
   19 # Multi-agent network
---&gt;   20 from torchrl.modules import MultiAgentMLP, ProbabilisticActor, TanhNormal
       21 
       22 # Loss

ImportError: cannot import name 'MultiAgentMLP' from 'torchrl.modules' 
(/usr/local/lib/python3.10/dist-packages/torchrl/modules/__init__.py)
</code></pre>
<p><a href="https://pytorch.org/rl/tutorials/multiagent_ppo.html" rel="noopener nofollow ugc">MARL pytorch tutorial</a></p>
<p>Thank in advance for help for fixing that issue</p>

vmoens: <p>The tutorial is written with the nightlies. It will work seamlessly with the new release later this week!</p>

Kevin_Fu: <p>Hi, I am running into the same import issue on the same tutorial. I have already installed the nightly build of torchRL with <code>pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu118</code>. Is there anything else I can do?</p>
<p>Specific error:</p>
<pre><code class="lang-auto">❯ python3 tutorial.py

Traceback (most recent call last):
  File "tutorial.py", line 20, in &lt;module&gt;
    from torchrl.modules import MultiAgentMLP, ProbabilisticActor, TanhNormal
ImportError: cannot import name 'MultiAgentMLP' from 'torchrl.modules' (/home/kfu38/.local/lib/python3.8/site-packages/torchrl/modules/__init__.py)
</code></pre>

Kevin_Fu: <p>Never mind, the latest version of torchRL (v0.2.0) fixed my issues, thank you!</p>
Automatically numbered variable names declared as nonlocal

robinho: <p>To create variables x1, x2, …, x10, use:<br>
for i in range(1,10):<br>
locals()[‘x’+str(i)]<br>
Then how to declare these as nonlocal ones to use them as global variables in functions?</p>

robinho: <p>To rephrase this question, I need to define variables x1, x2, …, xn over a for loop where n is a control parameter, that is, n is given a value elsewhere. Afterwards, these variables need to function normally as those defined in usual ways. What’s the proper way of defining them?</p>

ptrblck: <p>Define a function and return them to the global scope.</p>

robinho: <p>hi <a class="mention" href="/u/ptrblck">@ptrblck</a> many thanks. I don’t know exactly what to do. How does this code look?</p>
<p>def function(n):<br>
for i in range(1,n):<br>
locals()[‘x’+str(i)]<br>
return global locals()[‘x’+str(i)]</p>

ptrblck: <p>Use a <code>dict</code> instead of trying to create variables in different scopes:</p>
<pre><code class="lang-python">def fun():
    ret = {}
    for i in range(10):
        ret["x"+str(i)] = i
    return ret

x = fun()
print(x)
# {'x0': 0, 'x1': 1, 'x2': 2, 'x3': 3, 'x4': 4, 'x5': 5, 'x6': 6, 'x7': 7, 'x8': 8, 'x9': 9}
</code></pre>
Sharing a module between two workers

fournierlouis: <p>I am wondering if it is possible to implement the following distributed training case.<br>
From my understanding of the torch.distributed and RPC frameworks, it seems difficult, but I may have misunderstood.</p>
<p>Let’s say I have two workers (0 and 1) each with their GPU. I have a module on the worker 0, loaded on the GPU 0.<br>
Of course, I can do forward and backward passes on this module from the worker 0. However, I would like to “share” this module with the worker 1, such that I can easily also forward and backward (with input loaded on the GPU 1) through the same module (loaded on GPU 0 with the same parameters) through the worker 1.</p>
<p>As far as I understand it, the distributed RPC framework allows me to create a Remote Module on the worker 0 that I can use from the worker 1, however that doesn’t answer my usecase, since the worker 0 can not also use this module. I do not want to duplicate the module on the GPU, but access it from two different workers.</p>
<p>Thank you for your help.</p>
Resnet 50 inference speed

Snehashis_Chatterjee: <p>resnet50 fp32 pytorch inference speed : 6.5ms in a 3060 rtx with 64x64x3<br>
resnet50 fp32 pytorch inference speed : 6.5ms in a 3060 rtx with 224x224x3<br>
also when benchmark vgg16 224x224x3, got same speed, where vgg16 is almost 4x - 5x bigger model<br>
why both are the same?</p>

ptrblck: <p>Your workload might be CPU-limited, e.g. caused by a generally slow CPU, or heavy CPU processing, data loading etc., which would starve your GPU. Without any details it’s pure speculation.</p>

Snehashis_Chatterjee: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="189485">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>your</p>
</blockquote>
</aside>
<pre><code class="lang-auto">def benchmark(model, input_shape=(1024, 1, 32, 32), dtype='fp32', nwarmup=50, nruns=1000):
    input_data = torch.randn(input_shape)
    input_data = input_data.to("cuda")
    if dtype=='fp16':
        input_data = input_data.half()

    print("Warm up ...")
    with torch.no_grad():
        for _ in range(nwarmup):
            features = model(input_data)
    torch.cuda.synchronize()
    print("Start timing ...")
    timings = []
    with torch.no_grad():
        for i in range(1, nruns+1):
            start_time = time.time()
            output = model(input_data)
            torch.cuda.synchronize()
            end_time = time.time()
            
            
            timings.append(end_time - start_time)
            if i%100==0:
                print('Iteration %d/%d, avg batch time %.2f ms'%(i, nruns, np.mean(timings)*1000))

    print("Input shape:", input_data.size())
    print("Output shape:", output.shape)
    print('Average batch time: %.2f ms'%(np.mean(timings)*1000))
</code></pre>
<p>code I am using for benchmarking</p>

ptrblck: <p>Use a visual profiler to compare the timelines of all runs to check the kernel execution times, their launches, as well as CPU bottlenecks.</p>