akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Pytorch compatability with Cuda

RF_SIT: <p>Hi I have NVIDIA Quadro RTX 3000 GPU in my laptop. Cuda version is 12.2. Pytorch 2.0.1. It is not detecting GPU in VS code.</p>

ptrblck: <p>PyTorch 2.0.1 is compatible with all GPUs between sm_37 to sm_89 (using the binaries shipping with CUDA 11.7) and sm_90 (using the binaries shipping with CUDA 11.8). Your RTX 3000 mobile GPU should be a Turing GPU and is thus also supported.</p>
MobileNetv2 slow on phone

Johnny_Cash: <p>Hi, I am trying to run MobileNetv2 on an Android Phone (Fairphone 3) and the best I get speedwise is about 200 ms. I tried both quantized and not quantized. The original MobileNetv2 papers states 75ms on a Pixel 1.</p>
<p>Any clues on what could be the problem here?</p>

kimishpatel: <p>on your flow for generating model to be run on mobile, did you apply optimize_for_mobile? <a href="https://pytorch.org/tutorials/recipes/script_optimized.html#optimize-a-torchscript-model" class="inline-onebox" rel="noopener nofollow ugc">Script and Optimize for Mobile Recipe — PyTorch Tutorials 2.0.1+cu117 documentation</a></p>
Creating smaller model from original LLaMA models

Kason: <p>Hi, I’d like to create a smaller llama model using LLaMA2-7b-hf model. In particular, I’d like to reduce the number layers and add skip connections in between certain layers. Can anyone help me to do this properly?</p>

tom: <p>Do you have a dataset on which the output should be similar?<br>
Then, you can train your model to match the original one’s output and (likely also) intermediate results.</p>
<p>Best regards</p>
<p>Thomas</p>

Kason: <p>Sorry if the original question isn’t clear. I don’t want similar outputs from the smaller model. I just want to create a smaller model that has the same structure as llama2-7b except that it has fewer layers and additional skip connections. Other than that the two models should be the same (e.g. embedding layer, lm_head, embedding dimension, tokenizer etc.)</p>

tom: <p>The easiest would likely be to take a very simple implementation and modify that to your needs.<br>
The ones I’m most aware of are <a href="https://github.com/Lightning-AI/lit-gpt" rel="noopener nofollow ugc">Lit-GPT</a> (disclaimer: I contributed a patch or two there) and A. Karpathy’s <a href="https://github.com/karpathy/llama2.c" rel="noopener nofollow ugc">llama2.c</a> (which has a Python model for training and a training example on smaller configurations).</p>
<p>Best regards</p>
<p>Thomas</p>
Masking\Setting Attentions in Vision Transformers

rc999: <p>Hello,</p>
<p>I’m looking for a generic way to set all my attention maps to some constant matrix A (instead of calculating the inner products) throughout an entire model, any suggestions?<br>
One of the main issues is that I’m attempting to do this over many different pretrained models from different libraries - timm \ huggingface \ torchvision.</p>
<p>Thanks!</p>
Failing build mps locally

davidradl: <p>I followed the instructions for Mac to build mps by putting them in a script</p>
<pre><code class="lang-auto">LDFLAGS="-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib"
PATH="/usr/local/opt/llvm/bin:$PATH"
BUILD_CAFFE2=0
echo $PATH
python setup.py develop
</code></pre>
<p>I get errors as follows that implies I have not generated the mps files. Did I miss a step / flag?</p>
<pre><code class="lang-auto">RegisterMPS.cpp.o -c /Users/davidradley/pytorch/build/aten/src/ATen/RegisterMPS.cpp

**/Users/davidradley/pytorch/build/aten/src/ATen/RegisterMPS.cpp:635:22:** **error:** **no member named 'abs_out_mps' in namespace 'at::native'**

return at::native::abs_out_mps(self, out);

**~~~~~~~~~~~~^**

**/Users/davidradley/pytorch/build/aten/src/ATen/RegisterMPS.cpp:645:70:** **error:** **unknown class name 'structured_acos_out_mps'; did you mean 'structured_acos_out'?**

struct structured_acos_out_mps_functional final : public at::native::structured_acos_out_mps {
</code></pre>
<p>Any thoughts?</p>

davidradl: <p>sorry I have fixed it - I had not cleaned my environment properly</p>

sujantkumarkv: <p>Can you expand more on how exactly you fixed it?</p>
<p>please say more, assume a noob here.</p>
<p>thanks <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"></p>

davidradl: <p>I cant remember exactly - I suspect I had not run</p>
<p>cd ~/pytorch</p>
<p>git submodule deinit -f .</p>
<p>git clean -xdf</p>
<p>python setup.py clean</p>
<p>git submodule update --init --recursive</p>
Question on saving checkpoint asynchronously

shinytang6: <p>Currently, saving checkpoints synchronously will block training greatly in LLM situations. We’re in need of an asynchronous checkpoint saving feature.</p>
<p>Projects like JAX(<a href="https://flax.readthedocs.io/en/latest/guides/use_checkpointing.html#asynchronized-checkpointing" class="inline-onebox" rel="noopener nofollow ugc">Save and load checkpoints</a>), PyTorch Lightning(<a href="https://lightning.ai/docs/pytorch/stable/common/checkpointing_expert.html" class="inline-onebox" rel="noopener nofollow ugc">Distributed checkpoints (expert) — PyTorch Lightning 2.0.7 documentation</a>), and Microsoft Nebula have already implemented such feature.</p>
<p>It doesn’t seem overly complex, and I’m wondering if the PyTorch community has any plans to introduce this feature. Could you provide some insights, or let me know if I’m missing something?</p>
Utilization of one GPU suddently drops

barakb: <p>Hi, Im training on 4 GPUs in a ddp manner (bot pytorch ddp and accelerate package).<br>
During training I see peculiar phenomena after N training steps, one of the gpus utilization is dropping, out of no where, look at the bottom left plot<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/9/f94d7e4a94ec1982fd3485771c2b9eb3d69d1890.png" data-download-href="https://discuss.pytorch.org/uploads/default/f94d7e4a94ec1982fd3485771c2b9eb3d69d1890" title="wandb_util_drop"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/9/f94d7e4a94ec1982fd3485771c2b9eb3d69d1890_2_690x329.png" alt="wandb_util_drop" data-base62-sha1="zzqUXXYhyvzOBlTiWyQVTbXdf4Q" width="690" height="329" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/9/f94d7e4a94ec1982fd3485771c2b9eb3d69d1890_2_690x329.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/9/f94d7e4a94ec1982fd3485771c2b9eb3d69d1890_2_1035x493.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/9/f94d7e4a94ec1982fd3485771c2b9eb3d69d1890_2_1380x658.png 2x" data-dominant-color="FCFCFB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">wandb_util_drop</span><span class="informations">1810×865 141 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>It always happens around the same step of the training, but there is nothing unique in that step, or before.<br>
I did some more experiments, and its seems that the data time is the same, the forward time is the same, but the backward time is getting longer.<br>
But the weird thing is that Im not seeing any incremental increase or decrease, but a sudden one.</p>
<p>Offcourse that the utilization drop is effecting the training time from fast to un-trainable.</p>
<p>Any clue how can I start investigate this issue?</p>

tom: <p>You probably checked, but could it be heat?<br>
Old story, when we wrote the book, one of us wondered why his training would be slower with 2 GPU than with one. Turned out, with 2 crunching data, the thermal throttling would kick in.</p>
<p>Best regards</p>
<p>Thomas</p>
Pytorch Unable to download Dataset

stanleygeorge: <p>I am currently trying to use WikiTest103 dataset from the pytorch torchtext module.<br>
When I try to run the below code, I get the error as:</p>
<pre><code>from torchtext.datasets import WikiText103
X_train= WikiText103()
print(X_train.shape)
</code></pre>
<p>Error:</p>
<pre><code>Traceback (most recent call last):
  File "D:/Freiburg/MasterProject/TestWikiText103.py", line 2, in &lt;module&gt;
    X_train= WikiText103()
TypeError: __init__() missing 2 required positional arguments: 'path' and 'text_field'
</code></pre>
<p>However,<br>
When I run the same piece of code on Google Colab, it works perfectly.<br>
My torchtext version is 0.8.1 and pytorch version is 1.7.1</p>

Flock1: <p>Have you tried this way?<br>
<a href="http://man.hubwiz.com/docset/torchtext.docset/Contents/Resources/Documents/experimental_datasets.html#wikitext103" class="onebox" target="_blank" rel="noopener nofollow ugc">http://man.hubwiz.com/docset/torchtext.docset/Contents/Resources/Documents/experimental_datasets.html#wikitext103</a></p>

Codophile1: <p>I had the same issue, I solved it by upgrading <code>torchtext</code> to version <code>0.11.0</code> with:</p>
<pre><code> pip install torchtext --upgrade</code></pre>

Ke_Jiang: <aside class="quote no-group" data-username="Codophile1" data-post="3" data-topic="123912">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/9fc29f/48.png" class="avatar"> Codophile1:</div>
<blockquote>
<p>11.0</p>
</blockquote>
</aside>
<p>It works. Thank you.</p>
Loss in Self supervised learning with Simclr

Hanyang_Chen: <p>Hi friends,</p>
<p>I am using pytorch implementation of SIMCLR, training it on my own dataset.</p>
<p>The problem is, after 100 epoch, <strong>the loss dropped from 5.6 to 5.0</strong>, and it cease to decrease anymore.  I wonder what might be the problem of it, or is it a</p>
<p>The learning rate i set is is 0.2 and I wrap the optimizer with LARS with eeta=0.001, and the batch size is 512. (around 200000 small image with resnet18)</p>
<p>As you guys could see, the learning rate is rather moderate, right? (not unappropriately large). So what do think might be the problem~</p>
<p>Thanks!</p>

Hanyang_Chen: <p>or is it a normal situation~</p>
Creating a GraphModule from FX Graph

Lalithnarayan_C: <p>I have a sample model defined.</p>
<pre><code class="lang-py">class MyModule(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.lin = torch.nn.Linear(100, 10)

    def forward(self, x):
        return torch.nn.functional.relu(self.lin(x))

mod = MyModule()
</code></pre>
<p>I create an fx graph using the make_fx function from from torch.fx.experimental.proxy_tensor</p>
<pre><code class="lang-py">fx_graph_sample =  make_fx(mod)(torch.rand(100,100))
</code></pre>
<p>The make_fx command generates an fx_graph, stored in <code>fx_graph_sample</code>.</p>
<p>Using the original model, and the fx_graph, the idea is to create a GraphModule.</p>
<pre><code class="lang-py">torch.fx.GraphModule(mod, fx_graph_sample.graph)
</code></pre>
<p>The error I face is</p>
<p>AttributeError: ‘MyModule’ object has no attribute ‘_param_constant0’</p>
<p>When the fx_graph is generated using the make_fx function, iterating through the named_parameters of the model gives different parameter names than the ones in the original MyModule subclass. How do I solve the problem of generating the GraphModule?</p>
<p>PS: The same approach, when using symbolic_trace to generate the fx_graph, works perfectly.</p>
How to modify and save batches online

wasabi: <p>I want to modify the content within a batch at every training iteration (consider training RBM with PCD, at every step, you want to start with the saved inputs from the last step), what is the best way to do this? If I have lots of data, I would imagine saving in the memory is not possible. Is there a package that allows me to save the updated data to the storage efficient?</p>
It is possible to make a concat layer with other simplest?

sergiomh499: <p>Hi, I’m now working in a project where the concat layer in low level implementation is a big problem, but I need it to add this operation on my model architecture.</p>
<p>I’m trying to implement this concat layer with other primitive operation using PyTorch, but it’s a bit tricky for me. Anyone had some possible solution to my problem?</p>
<p>Thank you in advance <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

ptrblck: <p>Could you describe a bit more what exactly this concat layer should perform and why <code>torch.cat</code> wouldn’t work?</p>
Using torch.trapz for integrating 0 to \infinity

deepsana: <p>Consider the following code</p>
<pre><code class="lang-auto"># Choose number of layers
n_input = 1
n_hidden = 10
n_output = 2
n_bias = 1

def NN_s(q,w1,w2s,b):
    sigma = torch.nn.functional.softplus(q * w1 + b)
    outputs = torch.matmul(w2s,sigma)

def target_func_s(q):
    return q**0*np.exp(-1.5**2*q**2/2)

def K_s(w1, w2, b1):
    x = torch.linspace(0, float('inf'), steps=1000)
    #numerator part
    f =  x**2 * NN_s(x, w1, w2, b) * target_func_s(x)
    num = torch.trapz(f, x)
    # denominator part 1
    g = x**2 * NN_s(x, w1, w2, b)**2
    denom1=  torch.trapz(g, x)
    # denominator part 2
    h = x**2 * target_func_s(x)**2
    denom2 = torch.trapz(h, x)
    return (num**2) / (denom1*denom2)

#### cost function
def cost(w1, w2, b):
    w2s = w2[0,:]
    c_s = (K_s(w1, w2s, b)-1)**2
    return c_s 
</code></pre>
<p>When I run the following code,</p>
<pre><code class="lang-auto">epochs = 10**5
beta = 0.9
learning_rate = 0.02
costs = []
epsilon = 1e-6

w1 = torch.distributions.uniform.Uniform(-1,0).sample([n_hidden,n_input]).requires_grad_(True)
w2 = torch.distributions.uniform.Uniform(0,1).sample([n_output, n_hidden]).requires_grad_(True)
b = torch.distributions.uniform.Uniform(-1,1).sample([n_hidden, 1]).requires_grad_(True)

optimizer = optim.RMSprop([w1, w2, b], lr=learning_rate)

for epoch in range(epochs):
    optimizer.zero_grad()
    loss = cost(w1, w2, b)
    loss.backward()
    optimizer.step()
    costs.append(loss.item())

print("costs = {costs}");

</code></pre>
<p>I have problem in my function <code>K_s(w1, w2, b1)</code> when using<br>
<code>x = torch.linspace(0, float('inf'), steps=1000)</code></p>
<p>Is there a way to handle <code>\int_0^{\inf}</code> in torch? Any suggestions would help. Thank you !!!</p>

whichone: <p>Hi!<br>
assuming that your integral converges (which is probably true as you want to compute it), than \int_0^\inf may be approximated with \int_0^L for some large number L. so the option for you is to choose satisfactory numerical limit instead of ‘float(“inf”)’, i.e. ‘1e-9’</p>

deepsana: <p>Thank you! I have a follow up question on <code>torch.optim.RMSprop</code>:</p>
<p>When I do:</p>
<pre><code class="lang-auto">optimizer = optim.RMSprop([w1, w2, b], lr=learning_rate,  alpha=0.9, eps=1e-08)

for epoch in range(epochs):
    optimizer.zero_grad()
    loss = cost(w1, w2, b)
    print(loss)
    print(loss.backward())
    optimizer.step()
    costs.append(loss.item())
</code></pre>
<p>I get values for <code>loss</code>, but <code> loss.backward()</code> gives None:</p>
<p>example of prints</p>
<pre><code class="lang-auto">tensor(0.0413, grad_fn=&lt;AddBackward0&gt;)
None
tensor(0.0403, grad_fn=&lt;AddBackward0&gt;)
None
tensor(0.0398, grad_fn=&lt;AddBackward0&gt;)
None
tensor(0.0394, grad_fn=&lt;AddBackward0&gt;)
None
tensor(0.0391, grad_fn=&lt;AddBackward0&gt;)
None
tensor(0.0388, grad_fn=&lt;AddBackward0&gt;)
None
tensor(0.0385, grad_fn=&lt;AddBackward0&gt;)
None
tensor(0.0383, grad_fn=&lt;AddBackward0&gt;)
None

</code></pre>
<p>Any suggestions would help! Thank you!!!</p>
Expected INT8 Accuracies on ImageNet-1K (ResNet QAT)

CCL: <p>I am trying to compare the post-training quantization performance of a normally trained and QAT-trained network (ResNet-18 and 50) on ILSVRC-2012, using the torchvision codebase: <a href="https://github.com/pytorch/vision/tree/main/references/classification" rel="noopener nofollow ugc">https://github.com/pytorch/vision/tree/main/references/classification</a>. However, I cannot find any references on QAT FP or INT8 (standard &amp; QAT) expected accuracies and am also getting the following surprising results:</p>
<ol>
<li><strong><code>train_quantization.py</code> training poorly:</strong> I’m getting below 40% top-1 accuracy even after 45 epochs for both ResNet-18 and 50</li>
<li><strong><code>train.py</code> standard-trained models do not degrade in accuracy post-quant:</strong> I trained an FP32 ResNet-18 to <code>Acc@1 67.610 Acc@5 88.102</code> and the INT8 post-quant accuracy is  <code>Acc@1 67.136 Acc@5 87.768</code>.</li>
</ol>
<p>This leads to the following questions about ILSVRC-2012 training (for R18 &amp; 50):</p>
<ol>
<li>What are their expected QAT FP32 accuracies after training with <code>train_quantization.py</code>?</li>
<li>What are their expected QAT INT8 accuracies after training with <code>train_quantization.py</code>?</li>
<li>What are their expected INT8 accuracies after training with the standard <code>train.py</code>? Do they correspond to the accuracies listed here <a href="https://pytorch.org/vision/main/models.html#table-of-all-available-quantized-classification-weights" rel="noopener nofollow ugc">https://pytorch.org/vision/main/models.html#table-of-all-available-quantized-classification-weights</a>?</li>
<li>Does anyone have references for the above Q1-3 for PreActResNet-18 &amp; CIFAR-10 training?</li>
</ol>
<p>It seems like there are no references for QAT + ResNet on ILSVRC-2012; there are only QAT from scratch references for MobileNetv2 &amp; 3 (<a href="https://github.com/pytorch/vision/tree/main/references/classification#qat-mobilenetv2" rel="noopener nofollow ugc">https://github.com/pytorch/vision/tree/main/references/classification#qat-mobilenetv2</a>). Do ResNets simply not train well with QAT? Thanks a lot!</p>

HDCharles: <p>i can’t really answer your questions 1-4 but its not expected that resnet’s would respond poorly to QAT.</p>
<p>one question is if you’ve ran 45 epochs of QAT and get 40% accuracy, but get almost no accuracy drop for PTQ, what’s you’re qat accuracy for epoch one? The models should be approximately identical at the start. The only difference is how QAT and PTQ pick the quantization parameters and if that’s having such an outsized effect, that would be really weird.</p>

CCL: <p>For ResNet-18, after 1 epoch, the QAT accuracies are <code>Test: QAT Acc@1 1.272 Acc@5 4.960</code> whereas the accuracies for standard training are <code>Test:  Acc@1 23.414 Acc@5 47.392</code>.</p>
Is it possible to not save metadata when exporting?

Mah_Neh: <p>The onnx model exported by <code>torch.onnx.export</code> is populated with metadata (like the path to several python modules.)</p>
<p>I do not want this information when distributing the model.</p>
<p>There are no details about this in the torch.onnx.export documentation at all.</p>
<hr>
<p>To be clear this is what I am talking about, inside the <code>onnx</code> model (which is checked and works fine.) there are many lines that I do not want shared with the model:</p>
<pre><code class="lang-auto">/Users/username/miniconda3/envs/spotRotated/lib/python3.11/site-packages/torch/nn/modules/module.py(1488): _slow_forward
/Users/username/miniconda3/envs/spotRotated/lib/python3.11/site-packages/torch/nn/modules/module.py(1501): _call_impl
/Users/username/Documents/spotRotated/nn_arch/classifierV2.py(46): forward
/Users/username/miniconda3/envs/spotRotated/lib/python3.11/site-packages/torch/nn/modules/module.py(1488): _slow_forward
/Users/username/miniconda3/envs/spotRotated/lib/python3.11/site-packages/torch/nn/modules/module.py(1501): _call_impl
/Users/username/miniconda3/envs/spotRotated/lib/python3.11/site-packages/torch/jit/_trace.py(118): wrapper
/Users/username/miniconda3/envs/spotRotated/lib/python3.11/site-packages/torch/jit/_trace.py(127): forward
/Users/username/miniconda3/envs/spotRotated/lib/python3.11/site-packages/torch/nn/modules/module.py(1501): _call_impl
/Users/username/miniconda3/envs/spotRotated/lib/python3.11/site-
...
/Users/username/miniconda3/envs/spotRotated/lib/python3.11/site-packages/torch/onnx/utils.py(1548): _export
/Users/username/miniconda3/envs/spotRotated/lib/python3.11/site-packages/torch/onnx/utils.py(506): export
/Users/username/Documents/spotRotated/utils/onnx_conversion.py(12): to_onnx
</code></pre>
<p>Do you have any idea maybe <a class="mention" href="/u/ptrblck">@ptrblck</a> ?</p>
Two similar networks, completely different training evolution

Mah_Neh: <p>So, I’m trying to remove the MaxPool replaced by strides in Convolutions as suggested in many papers (including the Resnet paper.)</p>
<p>This is my net (the backbone, but it’s the only relevant piece.):</p>
<pre><code class="lang-python">
class Backbone(nn.Module):
   # just a set of convolutions

    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 16, kernel_size=2, stride=1, padding=2) # stride=2 in version2
        self.batch1 = nn.BatchNorm2d(16)
        self.conv2 = nn.Conv2d(16, 32, kernel_size=2, stride=1, padding=2) # stride=2 in v2
        self.batch2 = nn.BatchNorm2d(32)
        self.conv3 = nn.Conv2d(32, 64, kernel_size=2, stride=1, padding=2) # ditto
        self.batch3 = nn.BatchNorm2d(64)
        self.conv4 = nn.Conv2d(64, 128, kernel_size=2, stride=1, padding=2) # ditto
        self.batch4 = nn.BatchNorm2d(128)
        self.conv5 = nn.Conv2d(128, 256, kernel_size=2, stride=1, padding=2) # stride 1 or 2
        self.batch5 = nn.BatchNorm2d(256)
        self.relu = nn.ReLU()
        self.maxpool = nn.MaxPool2d(kernel_size=(3, 3))

    def forward(self, x: torch.Tensor) -&gt; torch.Tensor:
        x = self.conv1(x)
        x = self.batch1(x)
        x = self.relu(x)
        x = self.maxpool(x)
        x = self.conv2(x)
        x = self.relu(x)
        x = self.conv3(x)
        x = self.relu(x)
        x = self.maxpool(x)
        x = self.conv4(x)
        x = self.batch4(x)
        x = self.relu(x)
        x = self.conv5(x)
        x = self.relu(x)
        return x
</code></pre>
<p>As you can see this is just a series of convolutions followed by batch normalization and in some cases max pool.</p>
<p>If I change this to strides, the loss locks at a 10X larger value and the model does not improve anymore.</p>
<p>Is there an obvious reason why this would be a mistake ?</p>

Mah_Neh: <p>I seem to have found the reason in this post <a href="https://stats.stackexchange.com/questions/387482/pooling-vs-stride-for-downsampling" class="inline-onebox" rel="noopener nofollow ugc">deep learning - Pooling vs. stride for downsampling - Cross Validated</a></p>
<p>It seems that strides=2 won’t be selecting for max signal, and this in turn ends up slowing down the learning (I am not currently checking gradients as I am a beginner but probably the gradient was small even thought the loss was not that small ?)</p>
<p>Maybe some one else can shed light</p>
When training a custom transformer translation model i get IndexError: index out of range in self

Raphael_Lorenzo_Pana: <p>I’m currently trying to understand transformer models more and i’m doing that by training a transformer model from the pytorch from scratch implementations. the tutorials i’ve been using are the ones from the text section of the tutorials in pytorch, specifically the LANGUAGE TRANSLATION WITH <code>NN.TRANSFORMER</code> AND TORCHTEXT and PREPROCESS CUSTOM TEXT DATASET USING TORCHTEXT</p>
<p>I’ve modified the code a bit so that it would work but it seems that I’ve still missed some parts as i get the error:</p>
<p>RuntimeError: CUDA error: device-side assert triggered<br>
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.For debugging consider passing CUDA_LAUNCH_BLOCKING=1.<br>
Compile with <code>TORCH_USE_CUDA_DSA</code> to enable device-side assertions.</p>
<p>to try to debug it i set CUDA_LAUNCH_BLOCKING=1<br>
and TORCH_USE_CUDA_DSA=1 as well as make the device it uses to be the cpu.</p>
<p>I also made sure that the amount of columns for each target and source pair is the same by printing them after padding, I also checked the numericalized tokens and made sure that they aren’t above the length of the vocabulary.</p>
<p>below is the link of the google colab file where i run the code. the data is the eng-fr from the tatoeba project. <a href="https://www.manythings.org/anki/" class="inline-onebox" rel="noopener nofollow ugc">Tab-delimited Bilingual Sentence Pairs from the Tatoeba Project (Good for Anki and Similar Flashcard Applications)</a></p>
<p>it would be so nice if someone were to tell me what is causing the error and how i could fix this</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://colab.research.google.com/drive/17v4LkhiLAr4SycUdyKjkH6nTW-Psn5DL?usp=sharing">
  <header class="source">
      <img src="https://ssl.gstatic.com/colaboratory-static/common/9022f8700786be2c4c60c9d4128ed8e8/img/favicon.ico" class="site-icon" width="128" height="128">

      <a href="https://colab.research.google.com/drive/17v4LkhiLAr4SycUdyKjkH6nTW-Psn5DL?usp=sharing" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>

  <article class="onebox-body">
    <img width="260" height="260" src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar">

<h3><a href="https://colab.research.google.com/drive/17v4LkhiLAr4SycUdyKjkH6nTW-Psn5DL?usp=sharing" target="_blank" rel="noopener nofollow ugc">Google Colaboratory</a></h3>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


ptrblck: <p>Try to isolate the failing line of code. Most likely an embedding layer is failing using an input containing indices which are out of range.</p>
Can't understand NCCL error

KUSHAL_JAIN: <p>I am finetuning a llama-2-7b using FSDP on 2 A100 80GBs. My pytorch version is 2.0.1+cu118. My model runs for 15 batches/steps and then crashes with the following stack trace:</p>
<pre><code class="lang-auto">[E ProcessGroupNCCL.cpp:828] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=388923, OpType=_ALLGATHER_BASE, Timeout(ms)=1800000) ran for 1804615 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:828] [Rank 1] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=388923, OpType=_ALLGATHER_BASE, Timeout(ms)=1800000) ran for 1804360 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:455] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data.
[E ProcessGroupNCCL.cpp:460] To avoid data inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 1] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=388923, OpType=_ALLGATHER_BASE, Timeout(ms)=1800000) ran for 1804360 milliseconds before timing out.
Fatal Python error: Aborted

Thread 0x00002ac6175f7700 (most recent call first):
  File "/cluster/project/sachan/kushal/math/lib64/python3.8/site-packages/torch/distributed/distributed_c10d.py", line 2532 in all_gather_into_tensor
  File "/cluster/project/sachan/kushal/math/lib64/python3.8/site-packages/torch/distributed/distributed_c10d.py", line 1451 in wrapper
  File "/cluster/project/sachan/kushal/math/lib64/python3.8/site-packages/torch/distributed/fsdp/flat_param.py", line 987 in _all_gather_flat_param
  File "/cluster/project/sachan/kushal/math/lib64/python3.8/site-packages/torch/distributed/fsdp/flat_param.py", line 919 in unshard
  File "/cluster/project/sachan/kushal/math/lib64/python3.8/site-packages/torch/distributed/fsdp/_runtime_utils.py", line 329 in _unshard
  File "/cluster/project/sachan/kushal/math/lib64/python3.8/site-packages/torch/distributed/fsdp/_runtime_utils.py", line 1007 in _prefetch_handles
  File "/cluster/project/sachan/kushal/math/lib64/python3.8/site-packages/torch/distributed/fsdp/_runtime_utils.py", line 643 in _pre_backward_hook

Thread 0x00002ac43d076700 (most recent call first):
&lt;no Python frame&gt;

</code></pre>
<p>It seems like there’s a race condition or synchronization issue happening here. How should I further debug this?<br>
Another concern that I have is FSDP seems to running the forward pass in parallel on the 2 GPUs. My understanding was that FSDP shards the model across the GPUs and runs a single forward/backward pass across both processes. Is my understanding incorrect?<br>
PS: <code>compute-sanitizer</code> and <code>CUDA_LAUNCH_BLOCKING</code> do not help in tracking down this error.</p>

ptrblck: <p>NCCL might be the victim here only re-raising errors from your <a href="https://discuss.pytorch.org/t/runtimeerror-setstorage-sizes-4096-4096-strides-1-4096-storage-offset-0-and-itemsize-2-requiring-a-storage-size-of-33554432-are-out-of-bounds-for-storage-of-size-0/187208">other post</a>.</p>
How to Change Input Size from Pretrained Model?

Skylar_White: <p>Hello, maybe this my first question looks like dump <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> (i hope not haha)</p>
<p>Currently im working on some Computer vision problem.<br>
then i want to use various model, like Resnet, EfficientNet, ViT, etc.</p>
<p>then come up some problem when <code>input_size</code> doesnt match up with weight transforms.<br>
for example. in <a href="https://pytorch.org/vision/main/_modules/torchvision/models/vision_transformer.html#ViT_H_14_Weights" rel="noopener nofollow ugc">ViT Weights</a> has 2 weight: <em><strong>IMAGENET1K_SWAG_E2E_V1</strong></em> and <em><strong>IMAGENET1K_SWAG_LINEAR_V1</strong></em>.</p>
<p>these 2 works on 518 and 224 minimum image size.</p>
<p>so let say i want to use <em><strong>IMAGENET1K_SWAG_E2E_V1</strong></em>, how i can change input image from 518 to (like) 150x150?</p>
<p>thanks in advanced.<br>
Any help would be appreciated.</p>

ptrblck: <p>I’m unsure if you want to resize the images to a self-defined size or if you want to use the transformation used during the pretraining of the model. In the latter case you can create the <code>weights</code> and use <code>weights.transforms()</code> as seen <a href="https://pytorch.org/vision/stable/models.html#classification">here</a>.</p>
Basic question about meaning of input and output sizes for torch.nn.Linear

seberino: <p>Consider torch.nn.Linear(1600, 128).</p>
<p>If previous layer has dimensions H X W X D, does that mean there are 1600 <strong>sets</strong> of connections<br>
where each <strong>set</strong> consists of connections to each of the H X W X D values?  In other words, if<br>
I was drawing a diagram, I would need to draw 1600 X H X W X D lines?</p>
<p>Likewise, does the 128 mean that there are 128 sets of connections where each set consists<br>
of connections to the 1600 values?  In other words, would a diagram for this part require<br>
drawing 128 x 1600 lines?</p>
<p>Thanks.</p>

seberino: <p>This post was flagged by the community and is temporarily hidden.</p>
DETR: i/o and keys in output of inference

somedays: <p>Hi,</p>
<p>Sorry if this topic is not appropriate for community.<br>
I develop preprocessing tool kit now;</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/what-itis/IP-Toolkit">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">

      <a href="https://github.com/what-itis/IP-Toolkit" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/344;"><img src="https://opengraph.githubassets.com/d9d959ac19ab8c91e01ecbaeff683e9877021f4c671e94d2064b51a211db45de/what-itis/IP-Toolkit" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/what-itis/IP-Toolkit" target="_blank" rel="noopener nofollow ugc">GitHub - what-itis/IP-Toolkit: Image Preprocessing Tool for Stable Diffusion</a></h3>

  <p>Image Preprocessing Tool for Stable Diffusion. Contribute to what-itis/IP-Toolkit development by creating an account on GitHub.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I use DETR for panopic segmentation (/funcs/pnpcseg.py does). The inference’s output have some problem that key (ex. pred_masks) to access makes an error of like a string must not be indexed by integer, or like an error key 0.</p>
<p>I checked source code of the DETR model, it shows used keys, but I could not understand list’s structure. I do not also know expected list structure, so debug is stopped now.</p>
<p>Does someone point out where is possible making such the errors?</p>
What does spatial_size do in datapoints.BoundingBox()?

ido0: <p>I had thought that using <code>spatial_size</code> in <code>BoundingBox()</code> would cause clipping, but this code:</p>
<pre><code class="lang-auto">r = TVF2.convert_format_bounding_box(
        datapoints.BoundingBox(ltwh, format=datapoints.BoundingBoxFormat.XYWH, spatial_size=img_hw, ),
        new_format=datapoints.BoundingBoxFormat.XYXY,
    )
</code></pre>
<p>converts <code>ltwh</code> to <code>xyxy</code> regardless of the spatial size I give it. Does it only store it for usage down the pipeline?</p>
<p>Also, I see in the documentation that <code>spatial_size</code> is <code>(h, w)</code> whereas <code>XYWH</code> implies the reverse order, why is this not consistent? perhaps I’m missing something?</p>
<p>Thanks.</p>
Is there a standard procedure to check the consistency of environment across all nodes in PyTorch DDP training?

Quincy: <p>In a distributed environment, maintaining consistency across all nodes (e.g., driver versions, NCCL versions) can help reduce some training issues caused by the environment, such as performance or compatibility problems, and make troubleshooting easier. I am wondering if PyTorch provides any standard hooks or other methods to support this kind of check?</p>
<p>In my current setup, I have multiple nodes with different hardware and software configurations, and I want to ensure that the environment is consistent across all nodes before starting the DDP training. Is there any built-in functionality in PyTorch to perform these checks, or do I need to implement a custom solution to compare the environments?</p>
<p>Any suggestions or best practices on how to handle this situation would be greatly appreciated. Thank you in advance for your help!</p>
Assign value to a tensor based on an index tensor?

ThangLD201: <p>Hi, I’m trying to use index tensor to assign value to another tensor. In particular, I have tensor A of shape [a,b,c] and index tensor B of shape [a,b].</p>
<p>I have a value tensor C of shape [b]<br>
I want to assign values such that  A[i,j,B[i,j]] = C[j].</p>
<p>I tried using a for loop but it’s too slow. Is there a proper approach to do this ? (I also checked some tutorial on tensor indexing e.g. <a href="https://stackoverflow.com/questions/64590830/how-to-index-intermediate-dimension-with-an-index-tensor-in-pytorch" rel="noopener nofollow ugc">here</a> and <a href="https://stackoverflow.com/questions/52092230/indexing-a-multi-dimensional-tensor-with-a-tensor-in-pytorch" rel="noopener nofollow ugc">here</a> but they aren’t relatable)</p>

ptrblck: <p>Could you post the (slow) reference implementation, please?</p>

ThangLD201: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="187185">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Could you post the (slow) reference implementation, please?</p>
</blockquote>
</aside>
<p>Thank for the response. I came up with the code</p>
<pre><code class="lang-auto">import torch
a = torch.rand((3,3,3))
b = torch.randint(0,3,(3,3)).unsqueeze(-1)
c = torch.full_like(b,fill_value=5,dtype=a.dtype)
print(a)
print(b)
print(a.scatter_(dim=2,index=b,src=c))
</code></pre>
<p>which did what I wanted</p>
Why doesn't llama use FSDP?

Nova-Rift: <p>Instead they are using:</p>
<p>from fairscale.nn.model_parallel.layers import (<br>
ParallelEmbedding,<br>
RowParallelLinear,<br>
ColumnParallelLinear,<br>
)</p>
<p>I’m guessing they used FSDP only for training and didn’t think it was necessary for inference.<br>
Any other ideas?</p>
Error in iterating through dataloader

asha97: <pre><code>for data, transf_data,target in data_bar:
</code></pre>
<p>File “/root/anaconda3/envs/gcn/lib/python3.11/site-packages/tqdm/std.py”, line 1178, in <strong>iter</strong><br>
for obj in iterable:<br>
File “/root/anaconda3/envs/gcn/lib/python3.11/site-packages/torch/utils/data/dataloader.py”, line 633, in <strong>next</strong><br>
data = self._next_data()<br>
^^^^^^^^^^^^^^^^^<br>
File “/root/anaconda3/envs/gcn/lib/python3.11/site-packages/torch/utils/data/dataloader.py”, line 1345, in _next_data<br>
return self._process_data(data)<br>
^^^^^^^^^^^^^^^^^^^^^^^^<br>
File “/root/anaconda3/envs/gcn/lib/python3.11/site-packages/torch/utils/data/dataloader.py”, line 1371, in _process_data<br>
data.reraise()<br>
File “/root/anaconda3/envs/gcn/lib/python3.11/site-packages/torch/_utils.py”, line 644, in reraise<br>
raise exception<br>
RuntimeError: Caught RuntimeError in DataLoader worker process 0.<br>
Original Traceback (most recent call last):<br>
File “/root/anaconda3/envs/gcn/lib/python3.11/site-packages/torch/utils/data/_utils/worker.py”, line 308, in _worker_loop<br>
data = fetcher.fetch(index)<br>
^^^^^^^^^^^^^^^^^^^^<br>
File “/root/anaconda3/envs/gcn/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py”, line 51, in fetch<br>
data = [self.dataset[idx] for idx in possibly_batched_index]<br>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
File “/root/anaconda3/envs/gcn/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py”, line 51, in <br>
data = [self.dataset[idx] for idx in possibly_batched_index]<br>
~~~~~~~~~~~~^^^^^<br>
File “/home/rani1/graph_mv/nn_linear/dataset.py”, line 14, in <strong>getitem</strong><br>
feature = self.features[idx]<br>
~~~~~~~~~~~~~^^^^^<br>
RuntimeError: CUDA error: initialization error<br>
Compile with <code>TORCH_USE_CUDA_DSA</code> to enable device-side assertions.</p>

ptrblck: <p>You didn’t describe your use case and what exactly triggers the error, so I speculate you are using <code>CUDATensor</code>s inside your <code>Dataset</code> and are then using multiple workers in your <code>DataLoader</code>, which will try to re-initialize the CUDA context.<br>
If so, use <code>CPUTensor</code>s or the <code>spawn</code> start method in multiprocessing.</p>

asha97: <p>Yes , just solved it, by num_worker = 0</p>
How determine class weights for 1 output

John_Deatherage: <p>I am creating a NN for classification with 1 binary output. I’m using BCELoss for my loss function.  My dataset is unbalanced: 97% for 0/false &amp; 3% for 1/true.  So I’d like to adjust the class weights during the loss function (nn.BCELoss(weights= ???)).  I don’t know what the function is asking for???</p>
<p>If I send/use (weights=(.97, .03)) I get an error message telling me (I think) that I’m sending two values when pytorch is expecting 1 (one output class).</p>
<p>Please help me understand. What is pytorch expecting? Is it expecting 1 number? is so, how do I calculate it?</p>
<p>Thank you for taking the time to read and consider my problem!!!</p>

ptrblck: <p>Use <a href="https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html#torch.nn.BCEWithLogitsLoss"><code>nn.BCEWithLogitsLoss</code></a> and provide the <code>pos_weight</code> argument as described in the docs:</p>
<blockquote>
<p>For example, if a dataset contains 100 positive and 300 negative examples of a single class, then pos_weight for the class should be equal to <code>300/100=3</code>. The loss would act as if the dataset contains <code>3×100=300</code> positive examples.</p>
</blockquote>
Different batchsize for training and validation

lisyuan: <p>Hi, could I set the different batchsize for training and validation? The gpu ram is not enough.<br>
For example,</p>
<ul>
<li>Training: batchsize 128</li>
<li>Validation: batchsize 1</li>
</ul>
<p>I think the batchsize of validation will not affect on the validation loss and accurancy, right?</p>

ptrblck: <p>Yes, you can use different batch sizes and the batch size during evaluation (after calling <code>model.eval()</code>) will not affect the validation results.</p>
<p>Are you using larger inputs during the validation or why do you have to reduce the batch size by 128x?</p>

lisyuan: <p>Now I am using batch size 128 for both training and validation but the gpu ram (2080Ti 11G) is full.<br>
By the way, my task is to combine image model and language model to classify. I am not sure whether my model is too large or not.</p>
<p>There are 443,757 question for training 214,354 for validation. I think the batch size 128 is a little bit small. The training time is almost 2.5 hours per epoch. It really drives me crazy…</p>

ptrblck: <p>You can reduce the memory usage during validation by wrapping the validation loop in a <code>with torch.no_grad()</code> block, which will make sure to not store the intermediate activations, which would be needed to calculate the gradients. If you aren’t using it already, you might be able to increase the batch size during validation further and speed up this loop.</p>

lisyuan: <p>I have already used <code>torch.no_grad()</code> to reduce the memory usage</p>

ndvbd: <p>Maybe this is relevant: <a href="https://openreview.net/forum?id=9MDjKb9lGi" class="inline-onebox" rel="noopener nofollow ugc">The batch size can affect inference results | OpenReview</a></p>
Why only batched inputs for epipolar metrics?

Joon: <p>Metrics for epipolar geometry in kornia force the inputs to be batched like below.</p>
<pre><code class="lang-python">if (len(Fm.shape) &lt; 3) or not Fm.shape[-2:] == (3, 3):
        raise ValueError(f"Fm must be a (*, 3, 3) tensor. Got {Fm.shape}")
</code></pre>
<p>However, I think we can get both batched and unbatched inputs and handle them separately.<br>
Is there any reason behind this logic, e.g. for safety?</p>
Assert all(tensors[0].size(0) == tensor.size(0) for tensor in tensors) AssertionError

Nayan: <p>Hi, I am using pytorch to build CNN model. But I am facing some problem with data generation. Here is my data shape:<br>
X_train = (-1, 1, 169999, 200)<br>
X_test = (-1, 1, 3000, 200)<br>
y_train = (169999, 200)<br>
y_test =  (3000, 200)<br>
train = torch.utils.data.TensorDataset(X_train, y_train)<br>
test = torch.utils.data.TensorDataset(X_test, y_test)</p>
<p>But after that I am getting this error:<br>
assert all(tensors[0].size(0) == tensor.size(0) for tensor in tensors)<br>
AssertionError</p>

albanD: <p>I guess your problem is that the tensors for <code>y_*</code> are missing the batch dimension? each training sample should have it’s corresponding label.</p>

Nayan: <p>should I need to change the dimension of y_train and y_test?</p>

albanD: <p>It depends what your task is.<br>
Pytorch always assumeses that the first dimension is the batch dimension.<br>
If you do regression, then X_train and y_train should have exactly the same size.<br>
If you are doing classification, y_train will be a 1D tensor of size <code>X_train.size(0)</code> (the batch size).</p>

Nayan: <p>I am doing classification task. I want my X as four dimension and my y as 1 dimension but getting error while I am trying to stack it together.<br>
train = torch.utils.data.TensorDataset(X_train, y_train), the error is coming from this line.</p>

albanD: <p>You need to check that <code>X_train.size(0)</code> is the same as <code>y_train.size(0)</code>.</p>

kludge397: <p>Excuse me. What is the meaning of .size(0)??<br>
I learned tensor.size() is just checking the size of tensor. but, what is the 0 meaning?</p>
Pytorch profiler's with_stack is not working

Kevin1: <p>Hi,</p>
<p>I am trying to use pytorch profiler to profile a RGCN sample code in DGL official git repo that is written in pytorch. Following the link (<a href="https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html" class="inline-onebox" rel="noopener nofollow ugc">PyTorch Profiler — PyTorch Tutorials 2.2.0+cu121 documentation</a>), I want to inspect the calling stack by setting with_stack=True. However, it does not give me the calling stack.</p>
<p>I also tried this pytorch official link (<a href="https://pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">PyTorch Profiler With TensorBoard — PyTorch Tutorials 2.2.0+cu121 documentation</a>), I ran exactly the same code (setting with_stack=True). However, the view stack is gray out. I attached the picture to show this.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/c/0c9133c1681daba383d21505c35a74dd0cc5a7d9.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/0c9133c1681daba383d21505c35a74dd0cc5a7d9" title="Screen Shot"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/c/0c9133c1681daba383d21505c35a74dd0cc5a7d9_2_690x395.jpeg" alt="Screen Shot" data-base62-sha1="1NaP4UP2ONsLGTCqLnM032bV7q9" width="690" height="395" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/c/0c9133c1681daba383d21505c35a74dd0cc5a7d9_2_690x395.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/0/c/0c9133c1681daba383d21505c35a74dd0cc5a7d9_2_1035x592.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/0/c/0c9133c1681daba383d21505c35a74dd0cc5a7d9_2_1380x790.jpeg 2x" data-dominant-color="F5F3F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot</span><span class="informations">2680×1538 288 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So, I want to know how to solve this issue to view the stack successfully?</p>
<p>I installed pytorch through the official command line and version is 2.0.</p>
<p>Other system config:<br>
OS: Ubuntu 18.04<br>
CUDA: 11.3 (Titan Xp)</p>
<p>Thanks!</p>

wjaskowski: <p>I have the same problem <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=12" title=":confused:" class="emoji" alt=":confused:" loading="lazy" width="20" height="20"></p>

qi_shen: <p>same here! <img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=12" title=":joy:" class="emoji" alt=":joy:" loading="lazy" width="20" height="20">, any ideas?</p>
What's the meaning of torch::jit::CreateFunctionalGraphs pass?

Its-astonishing: <p>Hey,<br>
I’ve been lately doing some experiments with torch’s JIT compiler and found no documentation on torch jit passes. The pass I’m interested in is <code>torch::jit::CreateFunctionalGraphs</code> (<a href="https://github.com/pytorch/pytorch/blob/558a9501fa178f47c488a608dcc79d6a5a6159af/torch/csrc/jit/passes/create_functional_graphs.cpp#L215" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/blob/558a9501fa178f47c488a608dcc79d6a5a6159af/torch/csrc/jit/passes/create_functional_graphs.cpp#L215</a>). Could someone please help me to understand the meaning of this pass, what’s the use of it? How it works?<br>
Thank you</p>