akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How do I build this file to be included? (torch_cuda_cu.dll)

username: Hb121234
cooked: <p>hi, I’m currently trying to custom build pytorch. But cuda doesn’t seem to be provided statically. How can I include cuda and cudnn statically?</p>
<p>I have already turned on the following options</p>
<p>USE_CUDA=ON<br>
USE_STATIC_CUDNN=ON<br>
USE_CUDNN=ON<br>
CAFFE2_STATIC_LINK_CUDA</p>
<p>my option</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/f/efe7a3037f2a462ab626da8c0de1201b5136eb8a.png" data-download-href="https://discuss.pytorch.org/uploads/default/efe7a3037f2a462ab626da8c0de1201b5136eb8a" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/f/efe7a3037f2a462ab626da8c0de1201b5136eb8a.png" alt="image" data-base62-sha1="yeiou7wVsgVgBTnaMzaz0l8cwcy" width="447" height="500" data-dominant-color="181818"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1222×1366 60.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I want the following files to be included in my whl file (only for TORCH_CUDA_ARCH_LIST=8.6)</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/4/745fdd0120d8eb844c3ccbc14cee206e86caf345.png" data-download-href="https://discuss.pytorch.org/uploads/default/745fdd0120d8eb844c3ccbc14cee206e86caf345" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/4/745fdd0120d8eb844c3ccbc14cee206e86caf345.png" alt="image" data-base62-sha1="gBuLUq0PGxG0oVMCpZShic5zCVD" width="278" height="500" data-dominant-color="F6F6F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">464×832 33.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>but my<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/2/621e38a412477f83f8f15c7360eb61781efa7c22.png" data-download-href="https://discuss.pytorch.org/uploads/default/621e38a412477f83f8f15c7360eb61781efa7c22" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/2/621e38a412477f83f8f15c7360eb61781efa7c22.png" alt="image" data-base62-sha1="dZZxwZzvAqW2g6xIPsX14fsDfMu" width="213" height="500" data-dominant-color="F8F8F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">348×814 7.06 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Str is not a Module subclass

username: shruthu
cooked: <p>Here is the error below:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/f/f5f1b656777dcc3efd88174c11f3e366e000abbe.png" data-download-href="https://discuss.pytorch.org/uploads/default/f5f1b656777dcc3efd88174c11f3e366e000abbe" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/2X/f/f5f1b656777dcc3efd88174c11f3e366e000abbe.png" alt="image" data-base62-sha1="z5IQtuwUrkEBkJ5DFWFOYnZdacC" width="690" height="431" data-dominant-color="F6F8F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">809×506 24.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I am not able to understand the error. I am a beginner in this and it would be really helpful if someone could assist me with this.</p>

username: MariosOreo
cooked: <p>It seems that the error occurs in your module, could you confirm the type of your module?</p>
<p>By this:</p>
<pre><code class="lang-auto">for i, module in enumerate(modules):
    print(torch.typename(module))
</code></pre>

username: lypxzdxiaodi
cooked: <p>It seems that your module/sequential contains ‘dict’ object and that can not be automatically interpreted into pytorch layers, so you need to check your model structure.</p>
How to fuse variable size feature with a fixed size feature

username: kkchan
cooked: <p>Hi guys <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"></p>
<p>Recently, Im working on a project, I got an encoder-decoder like generator, and the inputs I feed into the generator are vary in size (which is [batch_size, 1, x]). The output size of the bottleneck is [batch_size, 512, y], where y is depend on the input of the generator. I want to fuse this bottleneck output with a feature with size [batch_size, 1, 512], and the fused feature is still with size [batch_size, 512, y]. Is there a way to do this?</p>
<p>Thanks.</p>
Tensorboard SummaryWriter across multiple files

username: sneruz
cooked: <p>Hey,</p>
<p>I’m trying to use Tensorboard in a multi-file project. I have a <code>main.py</code> file that runs the training loop, but I would like to log some values from other packages/modules as well. What is the best practice here? Do I create a <code>SummaryWriter</code> in each module? If so, wouldn’t this cause problems with <code>from foo import *</code> style imports?</p>
<p>Since I already use a customer logger from the <code>logging</code> library, I was considering wrapping <code>SummaryWriter</code> within this.</p>
<p>Thanks!</p>
GPU is not available

username: VicFisher
cooked: <p>I am establishing a model with pytorch and trying to use GPUs to test my model. However, there is an error occuring with the following warning:<br>
"<br>
/home/usrname/softwares/anaconda3/envs/torch12/lib/python3.7/site-packages/torch/cuda/<strong>init</strong>.py:83: UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at …/c10/cuda/CUDAFunctions.cpp:109.)<br>
return torch._C._cuda_getDeviceCount() &gt; 0<br>
"<br>
python: 3.7.13<br>
pytorch: 1.12.0+cu102 (pip)<br>
cuda: 10.2<br>
GPU: NVIDIA GeForce RTX 2080Ti<br>
GPU Driver Version: 440.100<br>
OS: Centos 7<br>
The GPU is working normally which is able to display graphical interface of the server. The output of <code>nvidia-smi</code>:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/0/50cc0be5547c4f7ea08736d35dd0dc10f0ee6d58.png" alt="image" data-base62-sha1="bwLmDK1Mg0jSqm48haHdeQ6hzLi" width="644" height="300"></p>
<p>It is available last month with nothing changed except that we moved the servers to another room. Did someone meet similar problems? Could you please give some advice to solve the problem? Thanks a lot !!!</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="VicFisher" data-post="1" data-topic="165924">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vicfisher/48/54679_2.png" class="avatar"> VicFisher:</div>
<blockquote>
<p>It is available last month with nothing changed except that we moved the servers to another room. Did someone meet similar problems? Could you please give some advice to solve the problem?</p>
</blockquote>
</aside>
<p>Check if any other CUDA application is still executable and if not, reinstall the drivers as it seems your workstation is running into a setup issue.</p>

username: VicFisher
cooked: <p>Thank you for your suggestion. I am sure that no other CUDA application is working and will try to reinstall the driver.</p>
Questions about dispatch to Aten library and control flow

username: sunggg
cooked: <p>Hi, hope you are doing well.</p>
<p>I’ve just started looking into Torch FX and have some high-level questions.<br>
It will be great if you can share your insights or provide any reference to take a look at.</p>
<ol>
<li>
<p>How does TorchFX dispatch each tensor operator to Aten library? For example, I want to know what aten op that TorchFX lowers <code>torch.mul</code> into in the following example:</p>
<pre><code class="lang-auto">%mul : [#users=1] = call_function[target=torch.mul](args = (%x, 0.1), kwargs = {})
</code></pre>
<p>I’m looking for the equivalent-level of info (regarding aten ops) we get from each node in TorchScript after <code>jit.trace</code>.</p>
<pre><code class="lang-auto">Float(10, strides=[1], requires_grad=0, device=cuda:0) = aten::mul(%x, %4)
</code></pre>
</li>
<li>
<p>Does TorchFx have plan to support <a href="https://pytorch.org/docs/stable/fx.html#dynamic-control-flow" rel="noopener nofollow ugc">dynamic control flow</a> in the future?</p>
</li>
</ol>
<p>Thank you in advance!</p>
GPU memory increases in conditional computation

username: mathnow
cooked: <p>Say you have a nn.ModuleList with many layers in it. In each iteration, you select some of the layers at random.</p>
<p>I noticed that the reserved GPU VRAM as reported by nvidia-smi keeps increasing whenever a previously not used layer is selected.</p>
<p>I would rather expect that the memory requirements would remain constant since activations and gradients only need to be computed for the currently active layers.</p>
<p>Is there any way to prevent PyTorch from allocating more and more memory?</p>
<p>Code for reproduction:</p>
<pre><code class="lang-auto">n_layers = 1000

net = nn.ModuleList([nn.Linear(100,100) for i in range(n_layers)]).cuda()
optimizer = torch.optim.Adam(net.parameters(), lr=1e-3)

criterion = nn.MSELoss()

iterations = 1000

net.train()
for _ in range(iterations):
    data = torch.zeros((64, 100)).cuda() # some dummy data

    optimizer.zero_grad()

    idx = torch.randint(0, n_layers, (2,)) # 2 layers will be selected at random

    result = 0
    for i in idx:
        result += net[i](data) # dummy operation

    loss = criterion(result, torch.ones_like(data)) # dummy loss
    loss.backward()
    optimizer.step()
</code></pre>

username: ptrblck
cooked: <p>The increasing memory usage is caused by:</p>
<ul>
<li>the lazily initialized state of the <code>Adam</code> optimizer and</li>
<li>the gradients which are also lazily initialized and zeroed out in each iteration (the zeros would still be allocated in memory)</li>
</ul>
<p>You could delete the <code>.grad</code> attributes by calling <code>optimizer.zero_grad(set_to_none=True)</code> and swap to an optimizer without internal states (e.g. <code>SGD</code>) and would see that the memory usage is constant.<br>
If you want to keep using <code>Adam</code> then note that the memory usage “saturates” once all running estimates were calculated for each layer.</p>
Cant use pin_memory in custom dataset

username: tahiira
cooked: <p>Hi, the custom dataset is a dictionary and I use a custom collate_fn for Dataloader. when the pin_memory =False, the code was run correct, but when I set pin_memory=True, I faced with this error:</p>
<blockquote>
<p>attributeerror: ‘dict’ object has no attribute ‘to’</p>
</blockquote>
<p>this error was occurred when data was transferred to device</p>
<blockquote>
<p>for i, data in enumerate(loader, 1):<br>
if self.move_data_to_gpu:<br>
data = data.to(self.device)</p>
</blockquote>
<p>How can I resolve this error?</p>
Torch::jit::script::module vs torch::nn::Module performance

username: dshin
cooked: <p>This <a href="https://pytorch.org/tutorials/advanced/cpp_export.html" rel="noopener nofollow ugc">tutorial</a> describes how you can train a <code>torch.nn.Module</code> in python and save to disk using <code>torch.jit.trace()</code>. In c++, then, you can load this module and use it for inference.</p>
<p>Compared to re-implementing the same python neural network architecture in c++ as a <code>torch::nn::Module</code>, this is quite convenient. However, I cannot help but wonder whether this convenience comes at a cost. For instance, at the minimum, it seems to me that reconstructing the neural network architecture at c++ runtime must entail gluing together components via vtable lookups, something an equivalent hand-crafted <code>torch::nn::Module</code> compiled by a modern compiler could potentially avoid. Another possibility is if the compiled version has more streamlined memory access patterns than its jit counterpart.</p>
<p>It may very well be that whatever costs exist are dwarfed by other factors in real world applications. Still, I am curious whether any noteworthy tradeoffs exist, and if so, what their nature is.</p>
Feed list of tensors to torch.utils.data.TensorDataset()

username: Blupon
cooked: <p>Hi ! I would like to use <code>torch.utils.data.TensorDataset()</code> with a variable number of input tensors stored in a list and face an <code>AttributeError: 'list' object has no attribute 'size'</code> error. I understand the error but do not know what’s the best way to circumvent it.</p>
<p><strong>What we usually do:</strong> <code>torch.utils.data.TensorDataset(x_train, y_train)</code></p>
<p><strong>What I want to do:</strong> <code>torch.utils.data.TensorDataset([x_train1, y_train1, x_train2,...])</code> the input list of tensors being of variable length (the latter is built with successive .append() calls).</p>
<p>The point of using a list of input tensors is that each tensor can carry an input of different size (but with a shared number of inputs in each input tensor, i.e. a common first dimension size, thus coherent with the _<em>getitem</em>() method of TensorDataset()). The number of input list lengths fed to TensorDataset() varies from experiment to experiment, it can not be hardcoded.</p>
<p>The most direct alternative seems to call TensorDataset() once for each tensor of my list of inputs, and manually go through all the associated dataloaders (stored in a list ?) but the first one with a for loop within the first dataloader batch iteration loop. Maybe the proper way of handling this is a <code>collate_fn</code> as suggested <a href="https://discuss.pytorch.org/t/how-to-create-a-dataloader-with-variable-size-input/8278/3">here</a> for images with variable sizes. I would like to avoid this if possible, since TensorDataset() is already able to handle an arbitrary number of input tensors. Apologies if I missed the solution here or on SO &amp; many thanks.</p>

username: ptrblck
cooked: <p>You might want to write a custom <code>Dataset</code> (deriving from <code>TensorDataset</code> if needed) as described e.g. <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html">here</a>.</p>
Reproducibility issue w.r.t. Functional Transformations with randomness

username: KarlSzp
cooked: <p>I’m working with DDP model with multi-gpus on one single node.</p>
<p>After seeding all reproducibility-related modules(random, numpy, torch), when two training process finished completely, they gived out exactly all the same output.</p>
<p>However, if one training process failed and exited by accident, try restarting from checkpoint gived out different output.</p>
<p>Based on it, I finally found what caused this difference is <code>Functional Transformations with randomness</code>.</p>
<pre><code class="lang-auto">class Augment(object):
    def __call__(self, img):
        rand_int = random.randint(1, 10) # Just a example
        # ... other operations
        return img
</code></pre>
<p>E.g.: consider a training task fails at epoch 3, restarting from checkpoint at epoch 2:</p>
<p>EPOCH — DATA — TRANSFORMATION<br>
1st              —  data_1         — Aug(data_1)  #  Aug called once<br>
2nd              —  data_2         — Aug(data_2)  #  Aug called twice<br>
3rd              —  data_3         — Aug(data_3)  #  Aug should be called the third times, but code failed here</p>
<p>— Restarting —<br>
3rd — data_3_restart    — Aug(data_3_restart)  #  Aug called once</p>
<p>I’ve checked that <code>data_3</code> and <code>data_3_restart</code> are all the same, but <code>Aug(data_3)</code> and <code>Aug(data_3_restart)</code> are different.</p>
<p>Sorry for my fuzzy explaination, any suggestions?<br>
Thanks in advance.</p>
How to create image using same image multiple times?

username: Ragnor
cooked: <pre><code class="lang-auto">import numpy as np
import matplotlib.pyplot as plt
import cv2 as cv

w = 10
h = 10

figure = cv.imread("img.png")

fig = plt.figure(figsize=(8, 8))
columns = 4
rows = 5
for i in range(1, columns*rows +1):
    img = np.random.randint(10, size=(h,w))
    fig.add_subplot(rows, columns, i)
    plt.imshow(figure)
plt.show(
</code></pre>
<p>I used this code but with this code I create the several images.<br>
I want to create the image using same image using multiple time.<br>
ex. take 1 image and use 25 time like 5*5 to create one image.</p>

username: ptrblck
cooked: <p>I’m not sure what the exact issue is, as a single figure is created with 20 subplots:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/8/c/8ce8a985f82b930236828119d8c6eafdb6dc5902.png" alt="image" data-base62-sha1="k6xjZ1SPMLpxYP0oQ9NAPCW2kpk" width="466" height="470"></p>
<p>In any case, this question doesn’t seem to be PyTorch-related, so you might get faster and better help on e.g. StackOverflow.</p>

username: Ragnor
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank yo for your reply. actually I send this  question on StackOverflow as well.</p>

username: Ragnor
cooked: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/0/2/02cbb47681e9170f902c862d44ff10bbfa73b73c.png" alt="Building" data-base62-sha1="oJoe26hLswEiVshjmoVXV2QR7m" width="423" height="288"><br>
<a class="mention" href="/u/ptrblck">@ptrblck</a><br>
I want to create my output image something like this.<br>
I use one Image to create another image like this.<br>
This is also one image but in 9 different patch.</p>
Precision and Recall missunderstanding

username: Lane
cooked: <p>Hi, can somebody explain me what is these: T,R,K,A,M? I’m confused<br>
In <code>cocoeval.py</code> :</p>
<pre><code>```
    T           = len(p.iouThrs)
    R           = len(p.recThrs)
    K           = len(p.catIds) if p.useCats else 1
    A           = len(p.areaRng)
    M           = len(p.maxDets)
    precision   = -np.ones((T,R,K,A,M)) # -1 for the precision of absent categories
    recall      = -np.ones((T,K,A,M))
    scores      = -np.ones((T,R,K,A,M))
</code></pre>
<p>Why is <code>recall </code>use just T,K,A,M without R??</p>
<p>In code below, what is <code>npig</code>, because in recall calculation there is tp/npig → but recall is calculated by formula: Tp/(Tp+Fn), so where are Fn?</p>
<pre><code class="lang-auto">npig = np.count_nonzero(gtIg==0 )
if npig == 0:
    continue
tps = np.logical_and(               dtm,  np.logical_not(dtIg) )
fps = np.logical_and(np.logical_not(dtm), np.logical_not(dtIg) )

tp_sum = np.cumsum(tps, axis=1).astype(dtype=float)
fp_sum = np.cumsum(fps, axis=1).astype(dtype=float)
for t, (tp, fp) in enumerate(zip(tp_sum, fp_sum)):
    tp = np.array(tp)
    fp = np.array(fp)
    nd = len(tp)
    rc = tp / npig
    pr = tp / (fp+tp+np.spacing(1))
    q  = np.zeros((R,))
    ss = np.zeros((R,))

    if nd:
        recall[t,k,a,m] = rc[-1]
    else:
        recall[t,k,a,m] = 0

    # numpy is slow without cython optimization for accessing elements
    # use python array gets significant speed improvement
    pr = pr.tolist(); q = q.tolist()

    for i in range(nd-1, 0, -1):
        if pr[i] &gt; pr[i-1]:
            pr[i-1] = pr[i]

    inds = np.searchsorted(rc, p.recThrs, side='left')
    try:
        for ri, pi in enumerate(inds):
            q[ri] = pr[pi]
            ss[ri] = dtScoresSorted[pi]
    except:
        pass
    precision[t,:,k,a,m] = np.array(q)
    scores[t,:,k,a,m] = np.array(ss)
</code></pre>
<p>Or just can someone help me to realize how can I return and print <code>precision </code>and <code>recall </code>values?</p>
How to estimate minimum GPU requirements for my application

username: rokopi
cooked: <p>Hi,<br>
How do you estimate a minimum GPU requirement for your application ?<br>
This is something I’ve never been able to clarify.<br>
My application will use a model for inference, it’s a pytorch model for OCR (basically I’m using easyocr for this) and I’ll do a single image inference. I’m doing some test on Colab using the following functions after a single prediction:</p>
<p>torch.cuda.max_memory_allocated()<br>
torch.cuda.max_memory_cached()</p>
<p>I always get around 2GB for both (1.9 for the first and 2.5 for the second one), the GPU is a Nvidia T4.<br>
Is this the right way to do it ? Can I safely assume a GPU with 4GB memory is my minimum requirement? Will this still be valid on another GPU model ?</p>

username: marksaroufim
cooked: <p>The total memory you’ll need will be a function of: serialized weights of model, activations in case grad = true, size of batch. So 4GB in your case sounds fine, but often with these things benchmark and see, run <code>nvidia-smi</code> in a watch loop and see if you run into an OOM after long running times.</p>

username: rokopi
cooked: <p>thanks <a class="mention" href="/u/marksaroufim">@marksaroufim</a>, Using <code>nvidia-smi</code> in a loop in colab gives me a maximum value of 4910MiB for memory used during the execution, so I think 4GB GPU is not enough. Anyway… why after the exectution memory used sill report 4910MiB? Is that the maximum usage reported or it is due to the pytorch caching policy?</p>
How to find intersection as a bbox in torchvision

username: sagorsarker
cooked: <p>Hello,<br>
I am following this <a href="https://pytorch.org/vision/main/_modules/torchvision/ops/boxes.html#box_iou:~:text=%23%20implementation%20from%20https,inter%2C%20union" rel="noopener nofollow ugc">https://pytorch.org/vision/main/_modules/torchvision/ops/boxes.html#box_iou:~:text=%23%20implementation%20from%20https,inter%2C%20union</a> link to find the intersection bbox between two input box tensors.</p>
<pre data-code-wrap="py"><code class="lang-nohighlight">def _box_inter_union(boxes1: Tensor, boxes2: Tensor) -&gt; Tuple[Tensor, Tensor]:
    area1 = box_area(boxes1)
    area2 = box_area(boxes2)

    lt = torch.max(boxes1[:, None, :2], boxes2[:, :2])  # [N,M,2]
    rb = torch.min(boxes1[:, None, 2:], boxes2[:, 2:])  # [N,M,2]

    wh = _upcast(rb - lt).clamp(min=0)  # [N,M,2]
    inter = wh[:, :, 0] * wh[:, :, 1]  # [N,M]

    union = area1[:, None] + area2 - inter

    return inter, union
</code></pre>
<p>But I am stuck in the <code>wh</code> part. It will be a great help if someone help me to find the intersection/overlap box as output.</p>
CUDA out of memory error during feature extraction

username: Jack_Shi
cooked: <p>I’m trying to do some simple feature extraction using a pretrained ResNet50 on the CIFAR 100-20 dataset. It should be pretty straightforward, but after a certain number of batches the CUDA out of memory errors would appear. It seems very strange to me as something must have been accumulating across the batches and overwhelmed the GPU, but I could not locate the problem. Here’s a simple code snippet</p>
<pre><code class="lang-auto">import numpy as np

import torch
import torch.nn as nn
import torch.optim as optim
from torch.optim.lr_scheduler import ExponentialLR
import torchvision.datasets as datasets
from torchvision import transforms
from torch.utils.data import TensorDataset, DataLoader, Dataset
from torchvision.models import resnet50


device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model = resnet50(pretrained=True)
model.eval()
model = model.to(device)

transform = transforms.Compose([transforms.ToTensor(),
                                transforms.Resize((224,224)),
                                transforms.Normalize((0.485, 0.456, 0.406),
                                                     (0.229, 0.224, 0.225))
                                ])


trainset = datasets.CIFAR100(root='./', train=True,
                             download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=8,
                                          shuffle=True)

testset = datasets.CIFAR100(root='./', train=False,
                            download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=8,
                                         shuffle=False)

train_f = []
train_l = []
for i,(im,label) in enumerate(testloader):
    print(i)
    im = im.to(device)
    train_f.append(model(im).cpu())
    train_l.append(label)
    im = im.to('cpu')
    torch.cuda.empty_cache()
train_f = torch.cat(train_f)
train_l = torch.cat(train_l)
np.save("features",train_f.detach().numpy())
np.save("labels",train_l.detach().numpy())

</code></pre>
<p>Any suggestions would be appreciated!</p>
<hr>
<p>UPDATE: Even when I run the code above only on CPU, my laptop would kill the program after a certain number of batches.</p>

username: ptrblck
cooked: <p>You are storing the entire computation graph in each iteration in:</p>
<pre><code class="lang-python">train_f.append(model(im).cpu())
</code></pre>
<p>If you don’t want to backpropagate through the model outputs, <code>.detach()</code> them before appending to the <code>list</code>.</p>

username: Jack_Shi
cooked: <p>It worked after I detach <code>model(im)</code>, thanks so much!</p>
<p>Maybe a stupid question: so in normal batch optimization, how does PyTorch erase each batch’s computation graph after backprop? Is it in <code>loss.backward()</code> or <code>optimization.zero_grad()</code>?</p>

username: ptrblck
cooked: <p>In the <code>backward()</code> call PyTorch will release all intermediate tensors which are not referenced by other objects unless you explicitly use <code>retain_graph=True</code>.</p>
Can jacfwd track the variable after calling another function

username: miraboreasu
cooked: <p>For example</p>
<pre><code class="lang-auto">r    = foo(unknown)
J    = jacfwd(foo)(unknown)

def foo (unknown): 
r=torch.zeros(1, requires_grad=False, dtype=torch.float64)
a1= unknown[::2]
a2= unknown[1::2]
b1=nestfunction(a1)
r=b1+a2
return r 


def nestfunction (a1): 
b1=a1**2
return b1
</code></pre>
<p>Can jacfwd track a1 since a1 is passed to another function?<br>
Thank you guys.</p>

username: soulitzer
cooked: <p>Autograd only operates at the aten operator level - it doesn’t know about what happens at the python level, so everything should work as if the function were inlined.</p>
Model training on mobile devices

username: Rinat_Veliakhmedov
cooked: <p>Hi, I’m interested in implementing federated learning in an Android app and I’m hoping I can use pytorch for this.<br>
The main problem I have is that it does not seem to be possible to build pytorch library for Android that can do training and not just inference.<br>
Another problem is that the app is very constrained in terms of binary size it can allocate for this, so I would like to know if it’s possible to build a bare minimum version of libtorch for Android that is &lt;10MB in size.</p>
<p>Is this something that’s possible or should I try something else?</p>
<p>Thank you.</p>

username: cccclai
cooked: <p>To build the bare minimum version of libtorch for Android, lite interpreter probably worths a try. Here is the link to the recipe <a href="https://pytorch.org/tutorials/prototype/lite_interpreter.html" class="inline-onebox" rel="noopener nofollow ugc">(Prototype) Introduce lite interpreter workflow in Android and iOS — PyTorch Tutorials 1.8.0 documentation</a>. Turn on BUILD_LITE_INTERPRETER can help reducing the size. lite interpreter + custom build will reduce size even more, since it build libtorch only with the operators used by models.</p>
<p>As a note, lite interpreter is in prototype stage and still under development.</p>
<p>In terms of training, <span class="mention">@iseeyuan</span> do you know the answer?</p>

username: iseeyuan
cooked: <p>As <a class="mention" href="/u/cccclai">@cccclai</a> mentioned we are building light-weighted runtime on both Android and iOS. It’s not super difficult to add training capability. Actually we have tests running in test/cpp/jit/test_lite_trainer.cpp, for functionality only. We’ll let you know when we put the necessary components into Android build.</p>

username: leigao97
cooked: <p>Hi <a class="mention" href="/u/iseeyuan">@iseeyuan</a>, I am also interested in model training on Android devices with libtorch. I would like to ask what is the progress of supporting such function? Many thanks in advance.</p>

username: Kapsy
cooked: <p>Hi <a class="mention" href="/u/iseeyuan">@iseeyuan</a> I’m interested too, is there any update on this? Also interested to know for iOS.</p>
Does Pytorch support on-device training?

username: ANG_LI
cooked: <p>I am wondering is there any way we can perform on-device (e.g., Android) training using Pytorch or libtorch.</p>

username: Shuo_Zhang
cooked: <p>Also wondering.</p>
<p>Found training on-device impossible with pytorch</p>

username: IvanKobzarev
cooked: <p>At the moment we do not support training in pytorch mobile. We are looking on the options to support it in future. Stay tuned.</p>

username: Kapsy
cooked: <p>Is there any update to this? Also interested.</p>
<p>If not, is there a way to build libtorch (not lite) for mobile? Particularly interested in iOS.</p>
Cuda Out of Memory: Implementing a Paper

username: Aashish_Bhandari
cooked: <p>I have an M1 Mac, and I cannot use Cuda properly. I have used Kaggle for this purpose. I am trying to implement this paper: <a href="https://github.com/cuhksz-nlp/R2GenCMN" rel="noopener nofollow ugc">https://github.com/cuhksz-nlp/R2GenCMN</a></p>
<p>I am constantly getting the error:</p>
<p>Traceback (most recent call last):<br>
File “main.py”, line 135, in <br>
main()<br>
File “main.py”, line 131, in main<br>
trainer.train()<br>
File “/kaggle/working/1R2GenCMN/modules/trainer.py”, line 58, in train<br>
result = self._train_epoch(epoch)<br>
File “/kaggle/working/1R2GenCMN/modules/trainer.py”, line 184, in _train_epoch<br>
loss.backward()<br>
File “/opt/conda/lib/python3.7/site-packages/torch/_tensor.py”, line 363, in backward<br>
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)<br>
File “/opt/conda/lib/python3.7/site-packages/torch/autograd/<strong>init</strong>.py”, line 175, in backward<br>
allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass<br>
RuntimeError: CUDA out of memory. Tried to allocate 6.12 GiB (GPU 0; 14.76 GiB total capacity; 4.51 GiB already allocated; 5.53 GiB free; 8.17 GiB reserved in total by PyTorch) If reserved memory is &gt;&gt; allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF</p>
<p>How to solve this in Kaggle?</p>
Backpropagation is leaking when using einsum with onehot encoded vector

username: hiru
cooked: <p>I have a stack of feature maps as given below. (my batch size is 32 and I have 64 feature maps with each with size 64x64)</p>
<pre><code class="lang-auto">out0 = self.selection[0](x)#output=[32,64,64,64]
out1 = self.selection[1](x)#output=[32,64,64,64]
out2 = self.selection[2](x)#output=[32,64,64,64]

out=torch.stack([out0,out1,out2]).permute(1, 0, 2, 3, 4)#output=[32,3,64,64,64]
</code></pre>
<p>and I am selecting feature maps from the second dimension (3) using a one-hot encoded vector (y, 32x3) by using torch.einsum as given below.</p>
<pre><code class="lang-auto">out=torch.einsum("x y, x y c d e -&gt; x c d e",y,out) 
</code></pre>
<p>Even though the model trains, when I tried to check the backpropagation updates for <code>out0,out1,out2</code>, by providing data only for <code>out0</code> (basically a one hot encoder tensor consists of only <code>1,0,0</code>) however the <code>optimizer.step()</code> is upgrading the weights in layers corresponds to <code>out1,out2</code>.(which shouldn’t be the case they are basically multiplying by zero every time.)</p>
<p>Anyone ever had the same problem?</p>

username: KFrank
cooked: <p>Hi Hiru!</p>
<aside class="quote no-group" data-username="hiru" data-post="1" data-topic="164486" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/h/d2c977/48.png" class="avatar"> Hiru:</div>
<blockquote>
<p>the <code>optimizer.step()</code> is upgrading the weights in layers corresponds to <code>out1,out2</code>.(which shouldn’t be the case they are basically multiplying by zero every time.)</p>
</blockquote>
</aside>
<p>Please note that optimizers do update tensors with zero gradients if they<br>
are using weight decay or momentum (that is already non-zero).</p>
<p>Try your experiment using plain-vanilla <code>SGD</code> without weight decay or<br>
momentum.  If you still see this result, please post a simplified, complete,<br>
runnable script that illustrates your issue, together with its output.</p>
<p>Best.</p>
<p>K. Frank</p>

username: hiru
cooked: <p>It worked. I totally forgot about the weight decay.<br>
Thanks!</p>
Runtime error in Pytorch

username: helms
cooked: <p>RuntimeError: invalid argument 1: only batches of spatial targets supported (3D tensors) but got targets of dimension: 1 at /opt/conda/conda-bld/pytorch_1524584710464/work/aten/src/THCUNN/generic/SpatialClassNLLCriterion.cu:14</p>

username: ptrblck
cooked: <p>Based on the error message the model output shape and the currently used target create a shape mismatch in <code>nn.CrossEntropyLoss</code> or <code>nn.NLLLoss</code>.<br>
Check the <a href="https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss">docs</a> to see which shapes are expected and which tensor creates the mismatch.</p>
MultiheadAttention.forward() got an unexpected keyword argument 'average_attn_weights'

username: PaulOppelt
cooked: <p>Hey,<br>
I got an unexpected keyword Error in <code>MultiheadAttention.forward</code> with the following code:</p>
<pre><code class="lang-auto">att = torch.nn.modules.activation.MultiheadAttention(embed_dim = 512, num_heads = 8)
x = torch.randn(1,5,512)
att(x,x,x,average_attn_weights=False)
</code></pre>
<p>which returned:<br>
forward() got an unexpected keyword argument ‘average_attn_weights’. In the documentation however ‘average_attn_weights’ should be a keyword argument. Does anyone know why this is?</p>
<p>Best, Paul</p>

username: ptrblck
cooked: <p>The <code>average_attn_weights</code> argument was added in <a href="https://pytorch.org/docs/1.11/generated/torch.nn.MultiheadAttention.html?highlight=multiheadattention#torch.nn.MultiheadAttention"><code>1.11</code></a>, so you might need to update your PyTorch installation.</p>
How to convert this code to pytorch format

username: Steamer_Chou
cooked: <pre><code class="lang-auto">def build_conv_lstm(filters, kernel_len, lstm_hidden_size):
    beta = 1e-3
    model = Sequential()
    
    model.add(Conv1D(filters=50, kernel_size=3, input_shape=(151, 4), kernel_regularizer=l2(beta), padding='same'))
    
    model.add(Activation('relu'))    
    model.add(MaxPooling1D())
    model.add(Dropout(0.5))
    
    model.add(LSTM(units=50, return_sequences=True, kernel_regularizer=l2(beta), recurrent_dropout=0.1))
    model.add(Dropout(0.5))
    
    model.add(Flatten())    
   
    model.add(Dense(150, kernel_regularizer=l2(beta), activation='relu'))
	
    model.add(Dropout(0.5))
    model.add(Dense(1, kernel_regularizer=l2(beta), activation='sigmoid'))

    optim = optimizers.Adam(lr=0.0003)
    model.compile(optimizer=optim, loss='binary_crossentropy')

    return model
</code></pre>

username: ptrblck
cooked: <p><a href="https://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html">This tutorial</a> might be a good starter as it explains how a custom neural network is written in PyTorch.<br>
Since your model is quite simple, you could also directly use <code>nn.Sequential</code> and add the desired layers to this container.<br>
Some layer names would need to be changed (e.g. <code>Activation('relu')</code> → <code>nn.ReLU</code>, <code>Dense</code> → <code>nn.Linear</code>).</p>
The memory location of tensors with the same content are different?

username: leyangjin
cooked: <p>Hi,</p>
<p>I run the following code in cpu and find that the memory location of the first x[0], the second x[0] and y[0] are different.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/7/67afaefcf6877a1563fbf7729943c592b527647b.png" alt="image" data-base62-sha1="eNfzWrEV32lmQk9T5RYeXLLCfLZ" width="442" height="252"></p>
<p>I think it is quite strange. If the x and y were python lists, then id() of the first x[0], the second x[0] and y[0] should be the same.</p>
<p>I am looking forward to your help.</p>

username: ptrblck
cooked: <p>The created objects are tensors not lists. Based on your code they are not referencing each other so the memory location is expected to be different.</p>

username: leyangjin
cooked: <p>Thank you for your early reply.</p>
<p>I am new to PyTorch and I’d like to ask one more question. Why does the two y in the followig code have different locations in memory.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/9/69af05c640c15749b9e47a58c6b47c943b72edb0.png" alt="＞ import torch" data-base62-sha1="f4V7luBPrdL5dKIg5OFzdz4GBby" width="526" height="320"></p>
<p>The top and bottom y are all torch.Tensor([1]).</p>

username: ptrblck
cooked: <p>The first <code>y</code> tensor references <code>x</code> and shows thus the same storage, while the second <code>y</code> tensor holds ow memory and doesn’t reference any other object.</p>
TypeError: 'dict' object is not callable

username: Huseyin_Cetin
cooked: <p>Hello,<br>
I am working on object detection(YOLO) inside the ROS2 node. I trained my model from colab and load it in py. code. However, when I give an image inside the model, I got this error :</p>
<p>TypeError: ‘dict’ object is not callable</p>
<p>What is the solution in my case?</p>
<pre><code class="lang-auto">
import torch
import torch.nn as nn
import os
import subprocess
import sys

import sys
sys.path.insert(0, '/home/mev/ObjectDetection/yolov7')

print(os.getcwd())
os.chdir(r"/home/mev/ObjectDetection/yolov7")
print(os.getcwd())
PATH = os.getcwd()


if torch.cuda.is_available():
    map_location=lambda storage, loc: storage.cuda()
else:
    map_location='cpu'
    
model = torch.load('./yolov7.pt', map_location=map_location)

print(model)

img_fdr="./image.png"

processed_image = model(img_fdr)
</code></pre>

username: ptrblck
cooked: <p>I guess <code>torch.load</code> will load a <code>dict</code> containing e.g. different <code>state_dict</code>s etc. while you are assuming it’s the <code>model</code> object.<br>
Check the return value of <code>torch.load('./yolov7.pt', map_location=map_location)</code> and initialize a model manually if needed.</p>

username: Huseyin_Cetin
cooked: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a>. I think, there is no problem related to loading model, cause I already print the model on the console. Additionally, I use Netron to see the model structure. Both of them looks fine. I think the problem is importing image frames to model. What should be the type to give image frames to model ?</p>

username: ptrblck
cooked: <p>Usually it would be a tensor while you are passing a file path as a string, so you might want to check if that’s the expected input type.</p>
Making a model predict on the basis of a particular value

username: Saswat
cooked: <p>I have an input vector of shape say 1 x 400. It’s fed into a network that outputs a 1 x 100 vector. I want to design a model that only considers every 4th value of this tensor and gives me the max values.</p>
<p>For example, a demo tensor could be <code>[0.1, 0.2, 0.4, 3, 0.7, 0.2, 0.6, 8, 0.3, 0.1, 0.5, 12, 0.7, 0.1, 0.9, 8]</code>. For this 1x16 input, the corresponding output would be a 1x4 one. I want the model to look only at values <code>3,8,12,12</code> and then predict an output of the following form <code>0,0,1,1</code>. (note that there could be more than 1 1s in the output)</p>
<p>I have tried using MLPs to do this using CELoss but to no avail. Can this be solved using some particular architecture/loss only or am I doing something wrong?</p>
I need to convert following tensorflow model into pytorch model. Any help?

username: sam5
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/8/3893c99220675c11a90229b9c991818c71a9c8ea.png" data-download-href="https://discuss.pytorch.org/uploads/default/3893c99220675c11a90229b9c991818c71a9c8ea" title="tensorflowToPytorch"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/8/3893c99220675c11a90229b9c991818c71a9c8ea_2_690x279.png" alt="tensorflowToPytorch" data-base62-sha1="84vn1vjvcTJZkrRp64Nz9rKnP6y" width="690" height="279" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/8/3893c99220675c11a90229b9c991818c71a9c8ea_2_690x279.png, https://discuss.pytorch.org/uploads/default/original/3X/3/8/3893c99220675c11a90229b9c991818c71a9c8ea.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/8/3893c99220675c11a90229b9c991818c71a9c8ea.png 2x" data-dominant-color="EFF0F0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">tensorflowToPytorch</span><span class="informations">1014×411 31.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Subclassing or Extending torch::Tensor in C++

username: prashanthr05
cooked: <p>Hi,</p>
<p>Are there any good documentation/examples on how to correctly subclass a <code>torch::Tensor</code> in C++ so that the derived class remains compatible with the usual <code>autograd</code> and optimization workflows?</p>
<p>Thanks in advance to anyone who could add some pointers on how to implement a custom tensor type by inheriting from the <code>torch::Tensor</code> properly.</p>
<p>Related question in <a href="https://discuss.pytorch.org/t/how-to-extend-torch-via-its-c-frontend/125917" class="inline-onebox">How to Extend Torch via it's C++ Frontend</a></p>
Stack multiple custom modules using nn::Sequential

username: alireza_dizaji
cooked: <p>I implemented multiple modules inherited from <code>torch::nn::Module</code> to stack together within a <code>nn::Sequential</code>, though I got an error. Just to make it simple, in the following I made a simple example of this.</p>
<pre><code class="lang-auto">class SampleImpl: public nn::Module {
  private:
    nn::Conv2d conv1;
  public:
    SampleImpl(int c1, int c2, int k): nn::Module(),
    conv1(nn::Conv2dOptions(c1, c2, k))
    {
      register_module("conv1", conv1);
    }

    at::Tensor forward(at::Tensor&amp; x) {
      return this-&gt;conv1(x);
    }
};
TORCH_MODULE(Sample);

int main(int argc, const char* argv[]) {
  nn::Sequential seq = nn::Sequential();
  seq-&gt;push_back(Sample(3, 12, 3));
  return 0;
}
</code></pre>
<p>And the following error appears:</p>
<pre><code class="lang-auto">error: static assertion failed: Modules stored inside AnyModule must not take references. Use pointers instead.
  336 |       torch::detail::check_not_lvalue_references&lt;ArgumentTypes...&gt;()
</code></pre>
<p>I have wrapped <code>std::make_shared</code> around it by using <code>TORCH_MODULE</code> macro but I don’t understand why I get this problem.</p>

username: alireza_dizaji
cooked: <aside class="quote no-group" data-username="alireza_dizaji" data-post="1" data-topic="164825">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alireza_dizaji/48/38663_2.png" class="avatar"> alireza_dizaji:</div>
<blockquote>
<p><code>error: static assertion failed: Modules stored inside AnyModule must not take references. Use pointers instead</code></p>
</blockquote>
</aside>
<p>I found the problem, which is in the forward function arguments and its reference part should be removed.</p>
Alternate concatenation of tensors

username: Saswat
cooked: <p>I have 2 tensors of shape [2, 1, 9] and [2, 1, 3]. I’d like to concatenate across the 3rd dimension alternatively (once every 4).</p>
<p>For example:</p>
<pre><code class="lang-auto">a = [[[1,2,3,4,5,6,7,8,9]],[[11,12,13,14,15,16,17,18,19]]]
b = [[[10, 20, 30]], [[1, 2, 3]]]
result = [[[1,2,3,10,4,5,6,20,7,8,9,30]],[[11,12,13,1,14,15,16,2,17,18,19,3]]]
</code></pre>
<p>How can I do this in pytorch?</p>

username: Saswat
cooked: <p>Solved by this</p>
<pre><code class="lang-auto">torch.concat([a.reshape((2, 1, 3, 3)), b.reshape(2, 1, 3, 1)], axis=-1).reshape((2, 1, -1))
</code></pre>
Is there a reason not to use dropout with GRU models?

username: laro
cooked: <ul>
<li>I have seen many examples of networks (<strong>without</strong> RNN/LSTM/GRU) which <strong>use</strong> dropout layers in order to reduce overfitting.</li>
<li>In contrast, I have hardly seen examples of networks <strong>with</strong> RNN/LSRM/GRU using dropouts ?<br>
(i.e  nn.GRU( dropout=, …)</li>
</ul>
<ol>
<li>
<p>Is there a reason not to use dropout when using RNN layers (GRU/LSTM) ?</p>
</li>
<li>
<p>What is the advantage or disadvantage of RNN networks with or without dropout ?</p>
</li>
</ol>