akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Use Pytorch for RTX 4070 cuda 12.2

WZN: <p>I purchased an RTX4070 for model training. According to nvidia-smi and nvcc -V, its CUDA version should be 12.2, so I tried to install the latest cu118 to attempt GPU-based training.<br>
However, after installation torch.cuda.is_available() did not return True or False but instead froze completely. Could this be because the current version of cu118 is not yet compatible with CUDA 12.2?</p>

ptrblck: <p>No, since your locally installed CUDA toolkit won’t be used unless you build PyTorch from source or a custom CUDA extension.<br>
To execute the PyTorch binaries you would need to properly install an NVIDIA driver, which seems to cause the error in your environment.</p>

WZN: <p>Thank you for checking. I will reinstall the driver and related environment to troubleshoot the issue.</p>
Error when doing backwards with custom loss function

axciss: <p>I’m trying to make my own batch loss function like below</p>
<pre><code class="lang-auto">class CustomLoss(torch.nn.Module):
    def __init__(self):
        super(CustomLoss, self).__init__()

    def forward(self, input, target, x_hat, U, sf, Tnu, Tnm, Tcn, x1, x2, acc_prob):
        diff = input - target
        all_pairs = torch.cat((x1, x2), dim=1)
        sf_tensor = torch.tensor(sf)
        acc_prob_tensor = torch.tensor(acc_prob)
        
        num_data = len(all_pairs)
        n = k

        item_list = all_pairs[:, 1]
        valid_indices = torch.arange(num_local)[torch.arange(num_local) != n]
        loss = 0
        global x_init
        x = x_init.clone()  
        with torch.no_grad():
            x[n, item_list] = x_hat.squeeze()
            x_init[n, item_list] = x[n, item_list]
        
        with torch.no_grad():
            usage = torch.sum(x[n] * sf_tensor)

        term3 = 0.0001 * d * (-torch.sum(input * sf_tensor[item_list] * torch.log(x_hat)) / num_data)
        term4 = 0.00000001 * a * (-torch.sum(torch.pow(torch.maximum(torch.tensor(0), x_hat * capacity_limit - x_hat * usage), 2)) / num_data)
        x_valid = x[valid_indices][: , item_list]
        
        u = all_pairs[:, 0]
        f = all_pairs[:, 1]
        term1 = 0
        for m in range(num_local):
            if m != n:
                term1 = term1 + U[n] * sf_tensor[f] * acc_prob_tensor[f] * (Tnu[n, u] * x[n, item_list] + \
                        (Tnm[n, valid_indices].unsqueeze(1) + Tnu[n, u]).unsqueeze(1) * \
                        (1 - x[n, f]) * x[m, f] + (Tnu[n, u] + Tcn[n]) * (1 - x[n, f]) * \
                        (1 - x[m, f]))
        loss = b * (torch.sum(term1) / (num_data * (num_local - 1)))
        k_loss2.append(term3.item())
        k_loss3.append(term4.item())
        loss = loss + term3 
        loss = loss + torch.mean(torch.pow(diff, 2))
        k_loss123.append(loss.item())
        
        return loss
</code></pre>
<p>n just an integer, x_init is a tensor that I want to update by x_hat<br>
input and x_hat is my model’s output<br>
I use relu activation function to get input and softmax function to get x_hat in my network model</p>
<p>it can work for two rounds, but in the third round, I got the error image when doing the backwards</p>
<pre><code class="lang-auto">RuntimeError: Function 'SoftmaxBackward0' returned nan values in its 0th output.
</code></pre>
<p>and here is the loss function I’m trying to implement<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/6/7/67ab0e9d66c9f604b03de203505dc756fd67d7f4.png" alt="image" data-base62-sha1="eN5FoaWPJd0aR1WSp2KLnhAfdwo" width="428" height="346"></p>
Model Loss functions becomes NaN when using multi-gpu ·

bello621: <p>Hello everyone, please I need an expert to help me on this.</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/Lightning-AI/dl-fundamentals/discussions/88">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/Lightning-AI/dl-fundamentals/discussions/88" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/344;"><img src="https://opengraph.githubassets.com/25f31b43ee4c600bcefc6323610294b04b38cb55011f3458559b46616950c460/Lightning-AI/dl-fundamentals/discussions/88" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/Lightning-AI/dl-fundamentals/discussions/88" target="_blank" rel="noopener nofollow ugc">Model Loss functions becomes NaN when using multi-gpu ·...</a></h3>

  <p>Hello guys, I've been trying to debug this code for a week now, but to no aveil, I am running into loss values of NaN when I try to train my model on multi-gpu, but works fine on 1 gpu, I don't kno...</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
Questions about Adaptive Average Pooling 2D

kaixin: <p>Hi,</p>
<p>I am trying to understand how Adaptive Average Pooling 2D works but I could not find a detailed explanation on google.</p>
<p>I did a small test on a 5x5 tensor.</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
import torch.nn.functional as F

x = torch.rand(1, 1, 5, 5)
# x:  tensor([[[[0.7984, 0.6614, 0.4994, 0.9424, 0.8370],
#               [0.9245, 0.0681, 0.8324, 0.9549, 0.8145],
#               [0.6221, 0.1108, 0.5670, 0.2415, 0.4548],
#               [0.1318, 0.3454, 0.5996, 0.4357, 0.1078],
#               [0.3747, 0.5511, 0.5398, 0.2429, 0.6817]]]])
ret = F.adaptive_avg_pool2d(x, (3, 3))
# ret:  tensor([[[[0.6131, 0.6598, 0.8872],
#                 [0.3671, 0.4617, 0.5015],
#                 [0.3507, 0.4524, 0.3670]]]])
</code></pre>
<p>It seems the average is computed in the cells as follows<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/a/a5c6e1f3d1f4644cb9d00306a790101fc07f0596.png" alt="avg_pool" data-base62-sha1="nEwSYqdX9pgsUjWktWg4DrkOi58" width="203" height="203"></p>
<p>I search the source code of PyTorch and it leads me to <a href="https://github.com/pytorch/pytorch/blob/master/aten/src/THNN/generic/SpatialAdaptiveAveragePooling.c" rel="noopener nofollow ugc">SpatialAdaptiveAveragePooling.c</a> and <a href="https://github.com/pytorch/pytorch/blob/master/aten/src/THCUNN/SpatialAdaptiveAveragePooling.cu" rel="noopener nofollow ugc">its cuda counterpart</a>. There are two ways to do the partition, defined as</p>
<pre data-code-wrap="cpp"><code class="lang-cpp">#define START_IND(a,b,c) (int)floor((float)(a * c) / b)
#define END_IND(a,b,c) (int)ceil((float)((a + 1) * c) / b)
// #define START_IND(a,b,c) a * c / b
// #define END_IND(a,b,c)  (a + 1) * c / b + ((a + 1) * c % b &gt; 0)?1:0
</code></pre>
<pre data-code-wrap="cpp"><code class="lang-cpp">int istartH = START_IND(oh, osizeH, isizeH);
int iendH   = END_IND(oh, osizeH, isizeH);
int kH = iendH - istartH;
</code></pre>
<p>In my example, the height and width are same so we only consider height. We have the followings:</p>
<pre><code class="lang-auto">oh           0     1     2
istartH      0     1     3
iendH        1     3     5
kH           1     2     2
</code></pre>
<p>But it seems the actual result computes these statistic following the definition in the comment.</p>
<pre data-code-wrap="cpp"><code class="lang-cpp">// #define START_IND(a,b,c) a * c / b
// #define END_IND(a,b,c)  (a + 1) * c / b + ((a + 1) * c % b &gt; 0)?1:0
</code></pre>
<pre><code class="lang-auto">oh           0     1     2
istartH      0     1     3
iendH        2     4     5
kH           2     3     2
</code></pre>
<p>Is there something in the source I missed?</p>
<p>Many thanks!</p>

pytorch_fun: <p>Did you ever find an answer?</p>
Plugging in new data to a trained neural network - EMNIST Dataset

Moeman6: <p>I am trying to create a program where an application analyzes your hand-drawn letters, but I’m having trouble putting the new data into the already trained model. It keeps coming back with large and strange outputs that don’t match what the training data’s outputs looked like. I am using the “byclass” Emnist set</p>
<p>Here is my Neural Network structure:</p>
<pre><code>class NeuralNet(nn.Module):
     def __init__(self):
         super().__init__()
         self.flatten = nn.Flatten()
         self.linear_relu_stack = nn.Sequential(
             nn.Linear(28*28, 512),
             nn.ReLU(),
             nn.Linear(512,512),
             nn.ReLU(),
             nn.Linear(512, 62),
             )

     def forward(self, x):
         x = self.flatten(x)
         logits = self.linear_relu_stack(x)
         return logits
</code></pre>
<p>And here is where I am trying to call on it in the application:</p>
<pre><code>    image = Image.open("Recent_Analysis.jpg").convert('L')
    imageArray = np.array(image)
    imageArray = imageArray.astype(np.float32)
    imageArray /= 255
    image_vals = torch.tensor(imageArray).unsqueeze(axis=0)
    print(image_vals.size())

    #Padding the batch in order to correct the batch size
    imageBatch = torch.zeros(128, 1, 28, 28)
    imageBatch[0] = image_vals

    with torch.no_grad():
        output = self.model(imageBatch)
        output = list(output[0].detach().numpy())
        print(output.index(max(output)))
</code></pre>
<p>I am confused on what I did wrong or if I fundamentally am unsure of how the output system works. I am very new to pytorch, so any help would be vastly appreciated. If you have more questions let me know.</p>

ptrblck: <p>Try to reuse a known image, which was already used during training, and compare the outputs between these runs. Based on your code it seems you are using a mostly empty batch containing only a single sample. If you haven’t moved the model to <code>model.eval()</code> the zeros would affect the batchnorm layers and the activation normalization of the input so you might want to use the single sample instead.</p>
RuntimeError: Could not run 'quantized::conv2d_relu.new' with arguments from the 'CPU' backend

nishchint98: <p>I am trying to run quantization on a model that I have to try and make the performance much faster. The model I am using to test this out is the pretrained wideres101(I have noted below how you can call it). The code is running on CPU. I put in a couple breakpoints, outputting the model and model size before and after quantization. Before quantization, the model is 510MB and after quantization it is down to 128MB. It seems like the quantization is being done as it should. <strong>The problem arises when the <em>quantized model</em> is called again later in the code when running the tester</strong>.</p>
<p><strong>How I call the model:</strong><br>
model = torchvision.models.wide_resnet101_2(pretrained=True)</p>
<p><strong>The line 113 is where the code is erroring out at:</strong><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/5/f58f9656b1798dd49682546e7aefac3b917de92b.png" data-download-href="https://discuss.pytorch.org/uploads/default/f58f9656b1798dd49682546e7aefac3b917de92b" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/5/f58f9656b1798dd49682546e7aefac3b917de92b_2_690x260.png" alt="image" data-base62-sha1="z2kC6jDq7VYTuijsCxD6BO56E5R" width="690" height="260" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/5/f58f9656b1798dd49682546e7aefac3b917de92b_2_690x260.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/5/f58f9656b1798dd49682546e7aefac3b917de92b_2_1035x390.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/5/f58f9656b1798dd49682546e7aefac3b917de92b_2_1380x520.png 2x" data-dominant-color="303031"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1819×686 83.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><strong>This is the error I get:</strong><br>
File “C:\Users\NishchintUpadhyaya\anaconda3\envs\SoilMoist\lib\site-packages\torch\nn\intrinsic\quantized\modules\conv_relu.py”, line 69, in forward<br>
return torch.ops.quantized.conv2d_relu(<br>
RuntimeError: Could not run ‘quantized::conv2d_relu.new’ with arguments from the ‘CPU’ backend. ‘quantized::conv2d_relu.new’ is only available for these backends: [QuantizedCPU, BackendSelect, Named, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, Tracer, Autocast, Batched, VmapMode].</p>
<p>Any suggestions?</p>

supriyar: <p><a class="mention" href="/u/nishchint98">@nishchint98</a>, this error usually means that the input to conv2d_relu is not a quantized tensor, but a floating point tensor. Please make sure in the code that the input being passed is a quantized tensor.</p>
No change/negligible change in loss while using Transformers

praveenbenedict1: <p>I’ve been trying to work on a text classification problem using LSTM and transformers. When I use the LSTM model, it seems to train fine. But When I train with the transformer model, there the loss and accuracy are constant and there seems to be no training happening.<br>
Where am I doing wrong?<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/d/bdc672b81da436c88942510e9d1756ae6dc5e41a.png" alt="image" data-base62-sha1="r4PpJtg3zXRMVR4c5zVoKrjWdPc" width="610" height="123"></p>
<p>Here is the link to the notebook:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://colab.research.google.com/drive/1iXCXN_CEznG3kDI7o-FuocLjBA8dq3dQ?usp=sharing">
  <header class="source">
      <img src="https://ssl.gstatic.com/colaboratory-static/common/ac0e6bae38aa2b5a52a4f6e10b43c43d/img/favicon.ico" class="site-icon" width="128" height="128">

      <a href="https://colab.research.google.com/drive/1iXCXN_CEznG3kDI7o-FuocLjBA8dq3dQ?usp=sharing" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>

  <article class="onebox-body">
    <img width="260" height="260" src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar">

<h3><a href="https://colab.research.google.com/drive/1iXCXN_CEznG3kDI7o-FuocLjBA8dq3dQ?usp=sharing" target="_blank" rel="noopener nofollow ugc">Google Colaboratory</a></h3>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
How can I train in C++ using a torch::jit::script::Modulel

ch2cl2: <p>Hi. I would like to use the python model for fine tuning in C++(Libtorch)<br>
The model is for example.</p>
<p>------------------- PYTHON</p>
<pre><code class="lang-auto">model = torchvision.models.resnet50(pretrained=True)
for param in model.parameters():
    param.requires_grad = False

model.fc = torch.nn.Linear(model.fc.in_features, 2)

traced_script_module = torch.jit.trace(model, img_batch)
traced_script_module.save(save_path)
</code></pre>
<hr>
<p>Load this model in C++<br>
---------------------  C++</p>
<pre><code class="lang-auto">torch::jit::script::Module module;
module = torch::jit::load("./model/model_resnet50_classification2_TL.pt");
module.to(device(torch::kCPU));
</code></pre>
<p>/*  ****************************************<br>
【   <strong>HOW DO I SET IT IN OPTIMIZER?</strong>  】</p>
<p>[NG1]   torch::optim::Adam optimizer(module.parameters(), torch::optim::AdamOptions(1e-3));<br>
[NG2]           torch::jit::parameter_list params = module.parameters();<br>
std::transform(params.begin(), params.end(), std::back_inserter(paramsVector),<br>
<span class="chcklst-box fa fa-square-o fa-fw"></span>(const torch::jit::IValue&amp; ivalue) { return ivalue.toTensor(); });<br>
torch::optim::Adam optimizer(paramsVector, torch::optim::AdamOptions(1e-3));<br>
[NG3]        std::vector<a>torch::Tensor</a> paramsVector;<br>
int i = 0;<br>
for (const auto&amp; params : module.parameters()) {<br>
std::cout &lt;&lt; "  " &lt;&lt; params.requires_grad() &lt;&lt; std::endl;<br>
paramsVector.push_back(params[i].variable_data());<br>
i++;<br>
}<br>
torch::optim::Adam optimizer(paramsVector, torch::optim::AdamOptions(1e-3));<br>
**************************************** */</p>
<pre><code class="lang-auto">//  ..... MAKE INPUT TENSOR (IMAGE 3*224*224)   &amp; Labels  ......
.........  EPOCH &amp; BATCH LOOP
module.train();
optimizer.zero_grad();
output = module.forward({ input }).toTensor();
loss = torch::nll_loss(torch::log_softmax(output, 1), label);
loss.backward();
optimizer.step();
</code></pre>
<hr>
Batch size and steps in multiple gpus

sl5035: <p>Lets say I have 2 million datasets. If I set batch-size to 1 and use 4 gpus on my machine and train the model using accelerate (huggingface), does it mean that each epoch will have 500,000 steps (2 million / 4) or will each epoch have 2 million steps?</p>
<p>+)<br>
I have tried to train my code with the above scenario using the code below.</p>
<pre><code class="lang-auto">def main():       
    processor = Pix2StructProcessor.from_pretrained('./deplot_models/deplot_base_model/')
    model = Pix2StructForConditionalGeneration.from_pretrained('./deplot_models/deplot_base_model/').to('cuda')
    
    with open('data/full_vocab.txt', 'r+') as f:
        full_v = [v.strip('\n') for v in f.readlines()]
        
    new_t = full_v[50345:]

    processor.tokenizer.add_tokens(new_t)
    model.resize_token_embeddings(len(processor.tokenizer))
    
    def collator(batch):
        new_batch = {"flattened_patches":[], "attention_mask":[]}
        texts = [item["text"] for item in batch]

        text_inputs = processor(text=texts, padding='max_length', return_tensors="pt", add_special_tokens=True, max_length=512)

        new_batch["labels"] = text_inputs.input_ids
        new_batch["labels"][new_batch["labels"] == processor.tokenizer.pad_token_id] = -100
        for item in batch:
            new_batch["flattened_patches"].append(item["flattened_patches"])
            new_batch["attention_mask"].append(item["attention_mask"])

        new_batch["flattened_patches"] = torch.stack(new_batch["flattened_patches"])
        new_batch["attention_mask"] = torch.stack(new_batch["attention_mask"])

        return new_batch
    
    train_dataset = DeplotDataset('./test/images', './test/targets/', processor)
    train_dataloader = DataLoader(train_dataset, shuffle=True, batch_size=1, collate_fn=collator)
    
    optimizer = Adafactor(model.parameters(), scale_parameter=False, relative_step=False, lr=0.01, weight_decay=1e-05)
    # optimizer = torch.optim.AdamW(model.parameters(), lr=1e-5)
    scheduler = get_cosine_schedule_with_warmup(optimizer, num_warmup_steps=100000, num_training_steps=2_000_000*EPOCHS)
    
    accelerator = Accelerator()
    device = accelerator.device
    
    model, optimizer, training_dataloader, scheduler = accelerator.prepare(model, optimizer, train_dataloader, scheduler)
    
    model.to(device)
    # model = torch.nn.DataParallel(model, device_ids=[0, 1, 2, 3])
    # model.cuda()

    model.train()

    for epoch in range(EPOCHS):
        print("Epoch:", epoch)
        for idx, batch in enumerate(train_dataloader):
#             if idx % 50 == 0:
#                 print(torch.cuda.max_memory_allocated())
            outputs = model(**batch)

            loss = outputs.loss

            if idx % 2 == 0:
                print(f"Idx: {idx}, Loss: {loss.item()}")

            accelerator.backward(loss)
            optimizer.step()
            scheduler.step()
            optimizer.zero_grad()
</code></pre>
<p>For some reason, I have noticed that the steps (idx) go above 500000. Is this supposed to happen?</p>
Parallel a style transfer code on multiple GPUs

lex2019: <p>The style transfer demo code at <a href="https://pytorch.org/tutorials/advanced/neural_style_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Neural Transfer Using PyTorch — PyTorch Tutorials 2.0.1+cu117 documentation</a> works fine for me, but I’m wondering what if I have a high-resolution image that cannot fit in a single GPU. I don’t think the dataparallel works for this situation since it works for models. Is there any way to store the image tensor in multiple GPUs and still have the gradient back together?</p>
Compiled twice on Windows10 to failure

DarkAlchy: <p>I am stumped as I am staring at the stub.c.  I can open the stub.c.  I can even make /Tc anything and it will still give the 1083 error.  For all I know something didn’t compile right but not sure how to check.</p>
<p>Here is the last command that is trying to sew it all together.</p>
<p>C:\Windows\System32\pytorch&gt;“C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\bin\HostX86\x64\cl.exe” /c /nologo /O2 /W3 /GL /DNDEBUG /MD -If:\condaEnv\include -If:\condaEnv\Include “-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\include” “-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\ATLMFC\include” “-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include” “-ID:\Windows Kits\10\include\10.0.20348.0\ucrt” “-ID:\Windows Kits\10\include\10.0.20348.0\um” “-ID:\Windows Kits\10\include\10.0.20348.0\shared” “-ID:\Windows Kits\10\include\10.0.20348.0\winrt” “-ID:\Windows Kits\10\include\10.0.20348.0\cppwinrt” “-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um” /Tctorch/csrc/stub.c /Fobuild\temp.win-amd64-cpython-311\Release\torch/csrc/stub.obj /MD /FS /EHsc<br>
stub.c<br>
c1: fatal error C1083: Cannot open source file: ‘torch/csrc/stub.c’: No such file or directory</p>
<p>I used this page to guide me through this process.</p>
<p><a href="https://datagraphi.com/blog/post/2021/9/13/building-pytorch-from-source-on-windows-to-work-with-an-old-gpu" class="onebox" target="_blank" rel="noopener nofollow ugc">https://datagraphi.com/blog/post/2021/9/13/building-pytorch-from-source-on-windows-to-work-with-an-old-gpu</a></p>
Build a 2d mask from sequence of lengths

whichone: <p>hello!<br>
I’d better explain my request with an example.<br>
let there be a sequence of integers <code>s = [2, 1, 3]</code>. I want to create a mask from this sequence as follows <code>mask = [ [1, 1, 0, 0, 0, 0], [0, 0, 1, 0, 0, 0], [0, 0, 0, 1, 1, 1] ]</code>. each row of 2-d mask contains <code>s[i]</code> ones, and these ones going successive. thus, len of each row of <code>mask</code> is <code>sum(s)</code>.<br>
Moreover, I want to effectively complete a task with batched input (a batch of such sequences). ofc, rows of masks in a batch would not be equal, so I want to pad it with trailing zeros.</p>
<p>I have quite a weird vectorized code for the task, but I want it to be simpler. If anyone interested, I’ll provide a code in comment below.</p>

whichone: <pre><code class="lang-auto">def sequence_mask(s):
    len = s.sum().item()
    x = torch.arange(len, dtype=s.dtype, device=s.device)
    return x.unsqueeze(0) &lt; s.unsqueeze(1)

def generate_mask_2d(s):
    B, L = s.shape
    # (B, L)
    s = torch.cumsum(s, dim=1)
    # (B * L)
    s = s.view(B * L)

    # (B * L, len)
    mask = sequence_mask(s)
    # (B, L, len)
    mask = mask.view(B, L, len)
    mask = torch.diff(mask, dim=1, prepend=torch.zeros(B, 1, len, dtype=align.dtype, device=align.device))
    return mask
</code></pre>
How can I convert mutiple arrays with different length to 2d-tensor

kaede: <p>Hi. I’m new to pytorch.</p>
<p>Give three different sized long typed 1d-tensor, for example<br>
[2,3,4]<br>
[1]<br>
[1,2,3,4]</p>
<p>how can I combine them efficiently to a 2d-tensor as follow:<br>
[<br>
[2,3,4,0],<br>
[1,0,0,0],<br>
[1,2,3,4]<br>
]<br>
?</p>

Eta_C: <aside class="quote" data-post="5" data-topic="38699">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/nested-list-of-variable-length-to-a-tensor/38699/5">Nested list of variable length to a tensor</a> 
  </div>
  <blockquote>
    This code might work: 
target = [[[1,2,3], [2,4,5,6]], [[1,2,3], [2,4,5,6], [2,4,6,7,8]]]
max_cols = max([len(row) for batch in target for row in batch])
max_rows = max([len(batch) for batch in target])
padded = [batch + [[0] * (max_cols)] * (max_rows - len(batch)) for batch in target]
padded = torch.tensor([row + [0] * (max_length - len(row)) for batch in padded for row in batch])
padded = padded.view(-1, max_rows, max_cols)
  </blockquote>
</aside>


whichone: <p><a href="https://pytorch.org/docs/stable/generated/torch.nn.utils.rnn.pad_sequence.html" class="inline-onebox" rel="noopener nofollow ugc">torch.nn.utils.rnn.pad_sequence — PyTorch 2.0 documentation</a> is exactly what you are seeking for.</p>
Creating redundant variables in forward pass does not incur memory costs?

Edenzzzz: <p>Here I intentionally added another dummy variable, but memory shown using torch.cuda.memory_allocated is the same before and after adding.<br>
I’ve also observed that using del to delete used variables doesn’t reduce memory usage.</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.functional as F
# from pytorch_wavelets import Learnable2D
import math
import os
import sys
import pdb
import pywt
from pytorch_wavelets import DWTForward, DWTInverse

class volumeNet(nn.Module):
	def __init__(self, nlevel, wave, inchannel, outchannel, learnable_wave, transform, mode):
		super(volumeNet, self).__init__()
		self.nlevel = nlevel
		self.wave = wave
		self.mode = mode
		self.learnable_wave = learnable_wave
		
		self.transform = transform                                          
		self.inchannel = inchannel
		self.outchannel = outchannel
		print("model inchannel:", inchannel)
		print("model outchannel:", outchannel)

		self.approx_conv1 = nn.Conv3d(inchannel, 64, 3, 1, padding='same')
		self.approx_conv2 = nn.Conv3d(64, 128, 3, 1, padding='same')
		self.approx_conv3 = nn.Conv3d(128, 128, 3, 1, padding='same')
		self.approx_conv4 = nn.Conv3d(128, 64, 3, 1, padding='same')
		self.approx_conv5 = nn.Conv3d(64, outchannel, 3, 1, padding='same')

	def forward(self, x, verbose=False, autocast=False):
		device = "cuda" if x.is_cuda else "cpu"
		with torch.autocast(device, enabled=autocast):
			return self._forward(x)

	def _forward(self, x):
		la = x
		la = self.approx_conv1(la)
		# la = F.relu(la)
		la = torch.sin(la)
		la = self.approx_conv2(la)
		# la = F.relu(la)
		la = torch.sin(la)
		la = self.approx_conv3(la)
		# la = F.relu(la)
		la = torch.sin(la)
		la = self.approx_conv4(la)
		# la = F.relu(la)
		dummy = torch.sin(la)
		dummy = self.approx_conv5(dummy)
		signal = dummy
		breakpoint()
		return signal
</code></pre>

ptrblck: <p>Which redundant variables are you referring to?<br>
According to the <code>forward</code> method you are re-assigning the intermediates to <code>la</code> and later <code>dummy</code>.<br>
Are you expecting to see a difference if you would replace <code>dummy</code> with <code>la</code>? If so, then note that Autograd will keep variables alive if these are needed for the gradient calculation in the <code>backward</code> call.</p>

Edenzzzz: <p>Thanks for replying. Yes I’m expecting to see an increase with the addition of dummy, but that seemed not the case.</p>
Is there a way to build pytorch on ppc64le and s390x without cpuinfo

jmontleon: <p>I hit an issue with building cpuinfo on ppc64le and s390x as a dependency of pytorch.</p>
<p>I filed an issue against the cpuinfo repo: <a href="https://github.com/pytorch/cpuinfo/issues/171" class="inline-onebox" rel="noopener nofollow ugc">Compile fails with clang 16 · Issue #171 · pytorch/cpuinfo · GitHub</a></p>
<p>The maintainer of cpuinfo replied that cpuinfo doesn’t work on these platforms, which I was aware of, however I was able to get it to build after a little more investigation.</p>
<p>My outstanding question that wasn’t answered is whether there is a way to build pytorch without cpuinfo on these platforms, or if it remains a dependency for some reason.</p>
Utilizing cuFFT library on the backend source code

Dizzy_Hint: <p>Hi,</p>
<p>I am trying to editing pytorch backend to test an improved conv algorithm. All my work was on ConvolutionMM2d.cu. cuFFT was used for all FFTs in my case. However, I just find out creating individual plan for each batch extremely hurt the performance. Is there any possibility that I could create a plan for all batches, and only need to transfer the created plan to each batch(or, to the ConvMM2d.cu file)?</p>
<p>Best,</p>

Dizzy_Hint: <p>Solved.</p>
<p>Inspired by cufft plan LRU cache in pytorch(source code in CuFFTPlanCache.h, SpectralOps.cpp), I wrote a static harsh map to store the address of plans, and it worked perfectly.</p>
Difference between tensor x[:,:,index] and x[:,:,index,:]

tommywolfheart: <p>Hello. Given a tensor <code>x</code> with shape (1,400,6,1), what is the difference between x[:,:,index] and x[:,:,index,:]? They both seem to give me an output with the desired shape (1, 400, 1).</p>

ptrblck: <p>There is no difference since you can skip the trailing <code>:</code> while indexing.</p>

tommywolfheart: <p>All right. Thank you.</p>
Distributed data parallel for model where the batch size is changing

john_doe_sp: <p>I am trying to implement FNO-MiONet architecture from this paper (<a href="https://arxiv.org/abs/2303.04778" class="inline-onebox" rel="noopener nofollow ugc">[2303.04778] Fourier-MIONet: Fourier-enhanced multiple-input neural operators for multiphase modeling of geological carbon sequestration</a>). In this architecture, the batch size changes during the forward pass. Below is the code for the network till branch-truck operation (please refer to Table 2 in the paper).</p>
<pre><code class="lang-auto">def forward(self, xfield, xscalars, xtime):
        batchsize_x = xfield.shape[0]
        size_x, size_y = xfield.shape[1], xfield.shape[2]
        size_t = xtime.shape[-1]
        
        # ==================================== MIOnet ====================================
        x_b1 = self.fc_b1(xfield) #[bs, nz, nx, width]
        x_b2 = self.fc_b2(xscalars) #[bs, width]
        x_branch = x_b1 + x_b2[:,None,None,:] #[bs, nz, nx, width]
        
        x_trunk = self.fc_t1(xtime) #[nt, width]
        x_branch = x_branch.unsqueeze(1)
        x_branch = x_branch.repeat(1, size_t, 1, 1, 1) #[bs, nt, nz, nx, width]

        # Reshape trunk output to match the dimensions of the branch output
        x_trunk = x_trunk.unsqueeze(0).unsqueeze(-2).unsqueeze(-2) #[1, nt, 1, 1, width]
        
        # multiply branch and trunk output
        x = x_branch * x_trunk
        
        x = x.reshape((batchsize_x*size_t, size_x, size_y, -1))
        # ==================================== Fourier layers====================================
        
        x = x.reshape((batchsize_x, size_t, size_x, size_y, -1))
        return x
</code></pre>
<p>The model works with a single GPU. However, I cannot use this model with the DataParallel option in PyTorch. If I use 4 GPUs with a batch size of four, the model’s output with the DataParallel option is just one sample and not 4.</p>
<pre><code class="lang-auto">for x, s, y in train_loader:
    x, s, y = x.to(device), s.to(device), y.to(device)
    t = train_t.to(device)
    optimizer.zero_grad()
    pred = dp_model(x, s, t) 
</code></pre>
<p>When I tried to do some testing for my code with 4 GPUs and 4 batchsize for the train_loader, I got only 1 sample for the pred variable.</p>

H-Huang: <p>Just to confirm, are you using <code>DataParallel</code> or <code>DistributedDataParallel</code>. Changing the batch size for <code>DistributedDataParallel</code> should be okay. I’m not understanding why it is getting split into 1 sample.</p>

john_doe_sp: <p>I am actually using DataParallel. I am going to try DistributedDataParallel option also. I am not sure if DataParallel works with changing the batch size or not.</p>
Transformer attention padding masks

Daniel_Maier: <p>Hi,<br>
I am pretty confused about how to create padding masks for attention between a query and key. Assume we have vectors for query and key that indicate valid data with 1 and padding with 0 (I know this is reverse to the usual convention). Then we want to create a matrix that is only 1 where both query AND key have valid data like so:<br>
|1_1_0_1<br>
1|1  1  0  1<br>
0|0  0  0  0<br>
1|1  1  0  1</p>
<p>If both of these vectors are batched then this (now 3d) tensor can be created with<br>
torch.einsum(“bm,bn-&gt;bmn”, [query_padding, key_padding])</p>
<p>The problem is that I have never seen such an implementation anywhere. I only saw implementations where one of the vectors is just repeated like so:<br>
|1_1_0_1<br>
1|1  1  0  1<br>
0|1  1  0  1<br>
1|1  1  0  1<br>
which in this case creates wrong entries in the 3rd row. Am I missing something here?</p>
TorchTensorRT: multihead attention can not run in fp16 mode

RhalBao: <p>When I use torch-tenosrrt speed up the inference, the multihead attention module can not run in fp16 mode. Here is a code:</p>
<pre><code>class MyModel(torch.nn.Module):
    def __init__(self, dim, n_heads, d_head, dropout=0., context_dim=None, gated_ff=True, checkpoint=True):
              super().__init__()
              self.attn = nn.MultiheadAttention(n_heads * d_head, n_heads) 

    def forward(self, x):
        x = x.transpose(0, 1)
        x = self.attn(x, x, x, need_weights=False)[0].transpose(0, 1) + x
        return x
</code></pre>
<p>First, I use torch.jit.trace trace the model, and then use torch_tensorrt.compile compile the model. Even though I set enabled_precisions= {torch_tensorrt.dtype.half}, the multihead attention runs in fp32 mode. And If I use with autocast(“cuda”, torch.float16) run the torchtrt model, I will get an error.</p>
Have I shot myself in the foot by training on a cluster but doing inference on a CPU?

pgyurov: <p>I have trained a segmentation model on a large cluster (don’t have the exact specs at the moment but think 512GB RAM, multiple Ampere/Volta GPUs, etc).</p>
<p>When I run inference on the cluster I get excellent results. When I run inference on the same data on my local GPU or CPU machine(s) I get significantly worse results.</p>
<p>In researching this I came across the topic of floating point precision: FP32, TF32, etc…</p>
<p>I cannot rely on the cluster to do inference. <strong>What is my best course of action?</strong><br>
I assume, from my limited understanding that I need to <strong>retrain with TF32 OFF and probably train for longer to achieve comparable results?</strong></p>

ptrblck: <p>Try to narrow down where the difference comes from as the same issue was already discussed multiple times in this forum. Common reasons were:</p>
<ul>
<li>different data processing,</li>
<li>model checkpointing failed (e.g. <code>strict=False</code> was used this not loading any parameters),</li>
<li>batchnorm layers collapsed in <code>eval</code> etc.</li>
</ul>
<p>Numerical could of course also be involved but I see other potential reasons as more likely.</p>

pgyurov: <blockquote>
<p>different data processing</p>
</blockquote>
<p>The same data and same script were used on the cluster and locally so I don’t think this is it</p>
<blockquote>
<p>model checkpointing failed (e.g. <code>strict=False</code> was used this not loading any parameters),</p>
</blockquote>
<p><code>strict</code> was set to <code>True</code></p>
<blockquote>
<p>batchnorm layers collapsed in <code>eval</code> etc.</p>
</blockquote>
<p>This is where I am not sure; how can I check this?</p>
How do the filters differ?

Mah_Neh: <p>If we add a conv layer with 64 filters how are they actually optimized differently and do not end up being all the same value?</p>

ptrblck: <p>Since all filters are initialized randomly (with different random weights) the training let’s them converge to different final values. If you would initialize all with the same static value(s), the training might collapse and the final values could be the same.</p>
Training multiple models on multiple cores in parallel

brian100: <p>Hi,<br>
Im using Optuna for hyperparamter search. Im training my models on the CPU. Im using the Optuna function study.optimize(wrapper, n_trials=trails, n_jobs=10).<br>
But I’m not seeing a performance increase over setting a lower value for n_jobs. Also, my core utilization is around 20% for every core.<br>
I also tried to set n_jobs to one and run the program in parallel from the command line. This does cause all my cores to have 100% utilization but still seems to do not much for speedup.</p>
<p>I’m preloading my train data to ram so data IO is not the bottleneck.<br>
Some performance timings:</p>
<p>1 program call - 50 trails (total = 50) -  n_jobs=5  - 2:10min<br>
1 program call - 50 trials (total = 50) - n_jobs=10   - 2:58min<br>
10 program calls - 5 trials (total = 50) - n_jobs=1 - 2:59min</p>
<p>But I need to run 200+ trials for ~100 models. That’s why I would like some parallelization.<br>
Is there a best/recommended way to train n models on n cores in parallel?</p>
<p>Thanks!</p>
Extra inputs in exported ONNX model

enigma: <p>Hi!</p>
<p>I am having an issue when exporting of PyTorch GNN model to ONNX. Here is my export code:</p>
<pre><code class="lang-auto">torch.onnx.export(
    model=model,
    args=(x_dict, edge_index_dict, edge_attr_dict, {}),
    f=save_path,
    verbose=False,
    input_names=["x_dict", "edge_index_dict", "edge_attr_dict"],
    output_names=["out"],
)
</code></pre>
<p><code>x_dict, edge_index_dict, edge_attr_dict</code> are of type <code>Dict[str, torch.Tensor]</code> (hetero_data is formed <a href="https://github.com/gsvgit/VSharp/blob/a569f0c0800c5d35b0533e9955de206fc371ed8b/VSharp.ML.AIAgent/ml/data_loader_compact.py#L30" rel="noopener nofollow ugc">like this</a>)</p>
<p>In addition to 3 inputs in my <a href="https://gist.github.com/emnigma/50f91731a4e68e64603263a656be1ee6" rel="noopener nofollow ugc">model</a>’s <a href="https://gist.github.com/emnigma/50f91731a4e68e64603263a656be1ee6#file-model-py-L25" rel="noopener nofollow ugc">forward</a> , torch.onnx.export generates 4 additional inputs and when I try to use exported model with onnxruntime I get ValueError:</p>
<p><code>ValueError: Required inputs (['edge_index', 'edge_index.5', 'edge_index.3', 'onnx::Reshape_9']) are missing from input feed (['x_dict', 'edge_index_dict', 'edge_attr_dict']).</code></p>
<p>Setting input dict as dynamic_inputs did not help</p>
<p>here is the jit trace with strict=False .code output:</p>
<pre><code class="lang-auto">def forward(self,
    argument_1: Dict[str, Tensor],
    argument_2: Dict[str, Tensor],
    argument_3: Dict[str, Tensor]) -&gt; Dict[str, Tensor]:
  state_encoder = self.state_encoder
  x = argument_1["game_vertex"]
  x0 = argument_1["state_vertex"]
  edge_index = argument_2["game_vertex to game_vertex"]
  edge_index0 = argument_2["game_vertex in state_vertex"]
  edge_index1 = argument_2["game_vertex history state_vertex"]
  edge_index2 = argument_2["state_vertex parent_of state_vertex"]
  edge_weight = argument_3["game_vertex history state_vertex"]
  _0 = (state_encoder).forward(x, edge_index, x0, edge_index2, edge_index1, edge_weight, edge_index0, )
  _1 = {"state_vertex": _0, "game_vertex": x}
  return _1
</code></pre>
<p>How can I get rid of those inputs?</p>
Pytorch's lstm states

Jesus: <p>Hello everyone. I hope that you all are ok. I am a beginner to deep learning and, i have some questions.<br>
1- Does a  lstm reset his hidden state for each sequence in a batch ?<br>
2- What is sliding windows technique and when should i use it with lstm ?</p>

vdw: <aside class="quote no-group" data-username="Jesus" data-post="1" data-topic="186220">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jesus/48/62979_2.png" class="avatar"> Jesus:</div>
<blockquote>
<p>1- Does a lstm reset his hidden state for each sequence in a batch ?</p>
</blockquote>
</aside>
<p>By default, yes. This means that the LSTM layer will initialize the hidden state if you don’t pass any as input. For example, say you define in your model <code>self.lstm = LSTM(...)</code>, and in your <code>forward()</code> method you call:</p>
<pre><code class="lang-auto">out, (h, c) = self.lstm(inputs)
</code></pre>
<p>then the initialization is done automatically. However, you can also pass your own initial hidden state like:</p>
<pre><code class="lang-auto">out, (h, c) = self.lstm(inputs, (h, c))
</code></pre>
<p>In which case it’s up to you where those come from. Those can come from the last batch or from your own initialization.</p>
<aside class="quote no-group" data-username="Jesus" data-post="1" data-topic="186220">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jesus/48/62979_2.png" class="avatar"> Jesus:</div>
<blockquote>
<p>2- What is sliding windows technique and when should i use it with lstm ?</p>
</blockquote>
</aside>
<p>I don’t think, there’s a single clear answer and most likely will depend on you concrete task. The common alternatives are obviously overlapping and non-overlapping windows. Overlapping seems to be the more intuitive choice to me, but I have nothing to back this up.</p>

Jesus: <p>Thank you very much sir for answering to my questions <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=12" title=":blush:" class="emoji" alt=":blush:" loading="lazy" width="20" height="20">. Do you know if pytorch lstm can use the last hidden state generated as initial hidden state for the next sequence in the batch ?<br>
I mean a way to do it automatically.</p>
PyTorch built-in layer for nanoGPT

dhruvbird: <p>I’m looking at the nanoGPT code from <a href="https://github.com/karpathy/nanoGPT/blob/master/model.py#L99" rel="noopener nofollow ugc">https://github.com/karpathy/nanoGPT/blob/master/model.py#L99</a> which has a single Causal Self Attention Block.</p>
<pre><code class="lang-auto">class Block(nn.Module):

    def __init__(self, config):
        super().__init__()
        self.ln_1 = LayerNorm(config.n_embd, bias=config.bias)
        self.attn = CausalSelfAttention(config)
        self.ln_2 = LayerNorm(config.n_embd, bias=config.bias)
        self.mlp = MLP(config)

    def forward(self, x):
        x = x + self.attn(self.ln_1(x))
        x = x + self.mlp(self.ln_2(x))
        return x
</code></pre>
<p>I’m trying to represent the Transformer model in nanoGPT (decoder only) using one of PyTorch’s built-in layers. I thought of using <a href="https://pytorch.org/docs/stable/generated/torch.nn.TransformerDecoder.html" class="inline-onebox" rel="noopener nofollow ugc">TransformerDecoder — PyTorch 2.1 documentation</a> at first, but looking at the code suggests that it’s built for use as a decoder that accepts an encoder’s output. i.e. the <code>memory</code> argument is required.</p>
<blockquote>
<ul>
<li><strong>memory</strong> (<a href="https://pytorch.org/docs/stable/tensors.html#torch.Tensor" rel="noopener nofollow ugc"><em>Tensor</em></a>) – the sequence from the last layer of the encoder (required).</li>
</ul>
</blockquote>
<p>I’m hoping that there’s some simple 1-2 line solution for this w/o me having to re-implement large parts of the code.</p>
<p>My initial attempt is to use <code>TransformerEncoder</code> by passing in <code>is_causal=True</code>. However, when I pass in <code>is_causal=False</code>, I get the same result. I am a bit confused.</p>
<pre><code class="lang-auto">torch.manual_seed(23)
x = torch.randn(10, 30, 16)

ys = {}

for is_causal in (False, True):
    torch.manual_seed(21)
    enc_layer = nn.TransformerEncoderLayer(d_model=16, nhead=4, batch_first=True,)
    enc = nn.TransformerEncoder(enc_layer, num_layers=2,)

    ys[is_causal] = enc(x, is_causal=is_causal)
#

print(torch.allclose(ys[False], ys[True]))  # prints 'True'.
</code></pre>
<p>I also found a tutorial on next word prediction using <code>TransformerEncoder</code> at <a href="https://pytorch.org/tutorials/beginner/transformer_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Language Modeling with nn.Transformer and torchtext — PyTorch Tutorials 2.1.1+cu121 documentation</a> - however, it seems like the attention mask argument is not passed in to the model’s <code>forward()</code> method.</p>
【libtorch c++ 】 how to make libtorch model distribute train and infer ，please show me one tutorial or example

_Veggie: <p>HI， for libtorch I found distribute package ,but I don’t know how to declare distribute param to make the libtorch model train and infer on distribute machines .need our team help, thanks,pleaase show me one example distribute train model code. thanks</p>
<h3><a name="versions-1" class="anchor" href="#versions-1"></a>Versions</h3>
<p>libtorch 2.0</p>
Flatten all dimension excep batch (?)

jhoanmartinez: <p>I am following the CIFAR-10 tutorial (<a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Training a Classifier — PyTorch Tutorials 1.11.0+cu102 documentation</a>), pretty nice, I understand pretty well and so on, but there is a line in the forward method that says</p>
<p>x = torch.flatten(x, 1) <span class="hashtag">#flaten</span> all dimension excep batch</p>
<p>The comment got me confused, I always have to flatten or not, then, why, it supposed that all must go flatten</p>
<p>Thank you so much fellas, I am new in this journey.</p>

mister_ekole: <p>The flatten layer simply converts the shape of your input data from an n-dimension matrices into a vector format that is suitable for all other densely connected layers of your neural net to interprete.</p>
Adding C++ Frontend to Existing Visual Studio Project (w/o Cmake)

Brendan_Lydon: <p>Hi all,</p>
<p>I am currently having issues finding the namespaces that match what the docs say based on how I have added the libtorch files to my project. Here is how I currently have imported libtorch into my project that does NOT use a Cmake build:</p>
<p>Project Properties-&gt;C/C+±&gt;General-&gt;Additional Include Directories: C:\LibTorch\libtorch\include\torch\csrc\api\include</p>
<p>Project Properties-&gt;Linker-&gt;General-&gt;Additional Library Directories:<br>
C:\LibTorch\libtorch\lib</p>
<p>Project Properties-&gt;Linker-&gt;Input-&gt;Additional Dependencies:<br>
c10.lib;torch.lib;</p>
<p>Project Properties-&gt;Post-Build Event-&gt;Command Line:<br>
xcopy /y “C:\LibTorch\libtorch\lib\c10.dll” “$(TargetDir)” &amp;&amp; xcopy /y “C:\LibTorch\libtorch\lib\torch.dll” “$(TargetDir)”</p>
<p>when I write “<span class="hashtag-raw">#include</span> torch/torch.h” I get the torch::data and torch::cuda namespaces. I have also tried, in additional include directories, C:\LibTorch\libtorch\include, <span class="hashtag-raw">#include</span> torch/library.h and got something similar to some of the examples but its off. I can’t even do torch::Tensor ten or torch::nn::Module in either situation I get errors. Any help/guidance would be appreciated. I am trying to add this library into a large existing project that does not use a CMake build, so right now I will not be taking that route.</p>
<p>Thank you for your help,<br>
Brendan</p>
PytorchStreamReader failed locating file constants.pkl

kabil_dev: <p>We are doing the wearable application for fall detection and we are on the movement to implement python ML into our application.</p>
<blockquote>
<p>val module = Module.load(assetFilePath(this, “test.pt”))</p>
</blockquote>
<p>From the above line we are receiving error like below…<br>
Caused by: com.facebook.jni.CppException: PytorchStreamReader failed locating file constants.pkl: file not found Exception raised from valid at /home/agunapal/pytorch/caffe2/serialize/inline_container.cc:183 (most recent call first):<br>
(no backtrace available)</p>
<pre><code>But when we downloaded someother .pt file from internet, it loads correctly. Our ML model was showing Exception. Please help us to overcome this.
</code></pre>

dhruvbird: <p>You can rename the <code>.pt</code> file to <code>.zip</code> and then unzip it to see if the <code>constants.pkl</code> file exists in there.</p>