akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
AttributeError: 'Tensor' object has no attribute 'load_state_dict'

1152768760: <p>Hi. I am trying to load a model with:</p>
<pre><code class="lang-auto"># Save the model checkpoint
torch.save(model.state_dict(), 'model.ckpt')

# 创建一个与原始模型具有相同结构的模型实例
model = model(inputs)

# 加载模型的状态字典
model.load_state_dict(torch.load('model.ckpt'))

</code></pre>
<p>But it says:</p>
<pre><code class="lang-auto">AttributeError: 'Tensor' object has no attribute 'load_state_dict'
</code></pre>

ptrblck: <p><code>model(inputs)</code> will return a tensor in common use cases.<br>
Since you are re-assigning the <code>model</code> variable to it:</p>
<pre><code class="lang-python">model = model(inputs)
</code></pre>
<p><code>model</code> will now be a tensor instead of an <code>nn.Module</code> instance and will thus fail in the next line of code.<br>
Assign the output to another variable and it should work, e.g.:</p>
<pre><code class="lang-python">output = model(input)
</code></pre>
Implementation of vanilla policy gradient (reinforce) method

Ategantos: <p>I am having trouble training an agent to play Doom through the vizdoom library using as a state the frame of the game and a convolutional neural network. I am giving you my implementation of the training loop and I wanted to ask if everything is right and the gradients are computed correctly.</p>
<blockquote>
<p>def train_policy_gradient_net(env, net, optimizer, num_episodes, gamma, max_trajectory_len, standardize):<br>
rewards_per_episode = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
steps_per_episode = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
loss_per_epoch = <span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<pre><code>for i in range(num_episodes):
    state = env.reset()
    done = False
    total_reward = 0
    steps = 0

    count = 0
    log_probabilities = []
    rewards = []
    while(count &lt; max_trajectory_len and not done):
        # use the network's stochastic policy to choose action and convert shape from [1, num_actions] to [num_actions]
        log_prob = net(torch.tensor(np.expand_dims(np.expand_dims(state, axis=0), axis=0)).to(device)).squeeze(0)
        # get probabilities from log probabilities
        prob = torch.exp(log_prob)
        
        # for numerical precision reasons the probabilities are normalised with their sum to sum exactly at 1 (because before they might not sum up exactly to 1)
        prob = Tensor.cpu(prob).detach().numpy()
        action = np.random.choice(range(env.game.get_available_buttons_size()), p=prob / np.sum(prob))

        # keep all the log probabilities of the actions that were selected
        log_probabilities.append(log_prob[action])

        next_state, reward, done = env.step(action)
        
        total_reward += reward
        steps += 1

        rewards.append(reward)

        state = next_state
        count += 1
    
    rewards_per_episode.append(total_reward)
    steps_per_episode.append(steps)
    
    # train network on the trajectory generated from the episode
    # compute G for each action
    g_vals = np.zeros(len(rewards))
    g_vals[-1] = rewards[-1]
    for j in range(len(rewards) - 2, -1, -1):
        g_vals[j] = rewards[j] + gamma * g_vals[j + 1]

    if(standardize and len(g_vals) &gt; 1):
        g_vals = (g_vals - np.mean(g_vals)) / np.std(g_vals)

    # we want to maximize the gain multiplied by the probabilities so we use a - to make the task to minimize the -gain so we can use pytorch's gradient descent
    log_probabilities_tensor = torch.stack(log_probabilities)
    g_vals = torch.tensor(g_vals).to(device)
    loss = -torch.sum((log_probabilities_tensor * g_vals))
    loss_per_epoch.append(loss.item())

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

return rewards_per_episode, steps_per_episode, loss_per_epoch
</code></pre>
</blockquote>
Models and datasets didn't complete the downloading process

Abdelkareem_Elkhateb: <p>when ever i try to donwload any datasets from torchvision.datasets the donwload reach 100% but didn’t success! this may continue for more than 2 hours in vain !<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/d/8dd2a178bdb5a9466529ee0577fed79bda9dca63.png" data-download-href="https://discuss.pytorch.org/uploads/default/8dd2a178bdb5a9466529ee0577fed79bda9dca63" title="Screenshot from 2023-05-14 03-23-58"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/d/8dd2a178bdb5a9466529ee0577fed79bda9dca63_2_690x212.png" alt="Screenshot from 2023-05-14 03-23-58" data-base62-sha1="keCB3ibV519QVJxDLxv6MsD1c9t" width="690" height="212" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/d/8dd2a178bdb5a9466529ee0577fed79bda9dca63_2_690x212.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/d/8dd2a178bdb5a9466529ee0577fed79bda9dca63_2_1035x318.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/d/8dd2a178bdb5a9466529ee0577fed79bda9dca63_2_1380x424.png 2x" data-dominant-color="262729"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2023-05-14 03-23-58</span><span class="informations">1918×590 41.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Are you able to download the file shown in your console manually? If so you could also move it to the expected data folder to avoid re-downloading it.</p>
How to install cuDNN for PyTorch 2.1.0 with CUDA 12.1

Kai2810: <p>Hi, new to machine learning and trying to run with my 4090.</p>
<p>I followed the instructions here on the pytorch website, installed for CUDA 12.1</p>
<p>But I read on Nvidia’s docs that I should install cuDNN as well, so downloaded v8.9.1 for CUDA 12.x</p>
<p>But realized is just a bunch of files with no installer. Searched and found the following youtube video where it showed that I simply have to copy the files from that cuDNN download into the respective “bin”, “include”, “lib” folders.</p><div class="youtube-onebox lazy-video-container" data-video-id="_U5ssERAN74" data-video-title="How to run Tensorflow/Pytorch off your GPU + Installing CUDA and CUDNN in 2023" data-video-start-time="0" data-provider-name="youtube">
  <a href="https://www.youtube.com/watch?v=_U5ssERAN74" target="_blank" rel="noopener nofollow ugc">
    <img class="youtube-thumbnail" src="https://img.youtube.com/vi/_U5ssERAN74/maxresdefault.jpg" title="How to run Tensorflow/Pytorch off your GPU + Installing CUDA and CUDNN in 2023" width="690" height="388">
  </a>
</div>

<p>Question1: Am I installing cuDNN correctly?</p>
<p>Question2: Is Pytorch using cuDNN?</p>
<p>Question3: When running the following in my visual studio code…</p>
<pre><code class="lang-auto">import torch
print(torch.backends.cudnn.version())
</code></pre>
<p>It printed out “8801”</p>
<p>Should I be concerned, since the cuDNN I downloaded is v8.9.1?</p>
<p>Please advise!</p>

ptrblck: <p>The PyTorch binaries ship with their own CUDA, cuDNN, NCCL, etc. dependencies. You can directly install the experimental nightly binaries with CUDA 12.1 and cuDNN 8.9 or you could build PyTorch from source using your locally installed CUDA toolkit and cuDNN in the versions you want.</p>
What is the best way to move data to the GPU?

Nobutaka_Kim: <p>I’ve read through a couple of gpu implementations for Kaggle Titanic and as a noob to Pytorch (basically left it until now since joining this forum a few years ago), noticed that data is moved to the gpu device in batches during training.<br>
Are there any other more efficient ways?<br>
I know in CUDA C/C++ programming there are many ways to move data synchronously or asynchronously to the GPU but with Pytorch I am lost.  Is the above the only or best way?  That just seems like a lot of data movement.</p>

ptrblck: <p>You could move the data asynchronously to the device manually by using pinned memory or via <code>pin_memory=True</code> in the <code>DataLoader</code> but would of course need some overlapping work to see any benefits.</p>

Nobutaka_Kim: <p>Oh wow, I just learned that in my Cuda course “pinned memory in the host”.</p>
<p>If you have a moment, can you point out the parts of the Pytorch repo where I can see all the Cuda code?</p>
<p>I want to see if prefetchAsync, streams, calculating the number of blocks based on the number of SM’s are being used.</p>

Nobutaka_Kim: <p>Found it: <a href="https://github.com/pytorch/pytorch/tree/674e52b0b913d7b7f733ce1e73a42cb383860d55/c10/cuda" class="inline-onebox" rel="noopener nofollow ugc">pytorch/c10/cuda at 674e52b0b913d7b7f733ce1e73a42cb383860d55 · pytorch/pytorch · GitHub</a></p>
Validation loss question

Ali_Youssef: <p>Hello!</p>
<p>I had a question regarding validation loss.</p>
<p>I’m doing  semi-supervised binary semantic segmentation with 20% laballed data, my predcited mask is improving every epoch, and the metrics at each epoch is quite good, for exmaple:</p>
<p><strong>Epoch: 6,Running Train loss: 0.018475, Running Validation loss: 0.153047, Validation Accuracy: 94.0433, Dice Score: 93.5111, BinaryJacIndx Score: 89.1448</strong></p>
<p>My problem is for the longest time I though my model is overfitting, even though augmented the training images (Reszied random crop, random rotation, random horizontal flip, Color jitter and Gaussian Blur), I also made sure to balance my training data.</p>
<p>I’m using a batch size of 32, the training data is roughly 5120 images so the length of the trainning loader is 160, my valdiation data is about 1100 images and the length of the validation loader is 31.</p>
<p>What I’m doing is I’m dividing the running training loss by the length of the training loader and running validation loss by the length of the validation loss.</p>
<p>Should I multiply the length of the loaders by the batch size ( running loss/ length of loader* batchsize), or is what I’m already doing is correct and the model is indeed overfitting?</p>
<p>Thank you!</p>
Resnet18 inference returning extremely low probabilities

kpullela10: <p>I’m following the PyTorch guide for using a pre-trained Resnet18 for simple inference. I’m trying to use a png image of a car, but when my model seems to predict very odd classes, with extremely low probabilities. Since nearly all of my code is simply from the guide, I’m a bit confused as to why I’m getting these predictions - the only thing I’ve changed is related to loading the initial image, so I assume I have made some errors there. Here’s what the top five classes and their probabilities I’m getting:</p>
<pre><code class="lang-auto">bucket 0.0083833709359169
hook 0.007907437160611153
plunger 0.006670754868537188
ashcan 0.006278147920966148
water jug 0.00571846729144454
</code></pre>
<p>I’ve tried using other images as well, yet for some reason bucket remains the top class (the probability isn’t the exact same, but is still quite small). Here’s the code I’m using:</p>
<pre><code class="lang-auto">import torch
import torchvision.models as models
from PIL import Image
import torchvision.transforms as transforms

model = models.resnet18(pretrained=True)

image = Image.open('car.png').convert('RGB')

preprocess = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

input_tensor = preprocess(image)
input_batch = input_tensor.unsqueeze(0) 

with torch.no_grad():
    output = model(input_batch)

probabilities = torch.nn.functional.softmax(output[0], dim=0)

with open("imagenet_classes.txt", "r") as f:
    categories = [s.strip() for s in f.readlines()]

# Show top categories per image
top5_prob, top5_catid = torch.topk(probabilities, 5)
for i in range(top5_prob.size(0)):
    print(categories[top5_catid[i]], top5_prob[i].item())
</code></pre>
Applying a transformation to a ConcatDataset

ado_sar: <p>I have my data (very large <code>np.array</code>’s) saved on disk as batches with <code>np.memmap</code>.  I am reading these batches as following:</p>
<pre><code class="lang-auto">x1 = np.memmap('path_to_file1', mode='c')
x2 = np.memmap('path_to_file2', mode='c')
...
</code></pre>
<p>and combine them with <code>ConcatDataset</code>. I would like to apply some <strong>preprocessing to this combined dataset</strong> but I don’t know how I should continue.</p>
<p>The reason I am troubled is because I don’t want to create copies of the arrays. For example, lets say that I want to apply some standardization. Prior to converting to a <code>ConcatDataset</code> I calculate the mean (weighted mean) and std (weighted std), then transforming the <code>x</code>’s and finally converting to a <code>ConcatDataset</code>.</p>
<p>That is, I am doing the following:</p>
<pre><code class="lang-auto">x1 = np.memmap('path_to_file1', mode='c')
x2 = np.memmap('path_to_file2', mode='c')
...
mean, std = get_weighted_mean_and_std([x1, x2, ...])

for x in [x1, x2, ...]:
    x -=  mean
    x *= 1/std

dataset = ConcatDataset([x1, x2, ...])
</code></pre>
<p>The problem is during the assignmets <code>-=</code> and <code>*=</code> which return copies instead of view’s.</p>
Parralellization in the case of using batch size of 1 with each being a list

Khumbaba: <p>Hello,</p>
<p>Sorry if the title is unclear, formulating a  short one for my question is a bit tricky.<br>
I’m working on an already existing project and I’m questioning whether or not the data loading part is running in parallel or there’s inactive workers.<br>
Let’s assume the data is loaded the following way:</p>
<pre><code class="lang-auto">class Dataset(torch.utils.data.Dataset):
	def __init__(...):
		self.minibatches = [
			... #Load list of minibatch indices with batch size 16
		]
	def __getitem__(self, index):
		return self.minibatches[index]
		
def custom_collate_fn(minibatch):
    data = []
    for i in range(len(minibatch)):
        data.append( Load(minibatch[i]) )
        ... #code to pad and convert to tensor ...
    return data
  
train_dataset = Dataset(...)
training_loader = torch.utils.data.Dataloader(dataset=train_dataset, batch_size = 1, num_workers = 8, collate_fn = custom_collate_fn)
</code></pre>
<p>Data loading can’t possibly run in parallel because of that for loop, right? Even though it seems fast with a batch size of 16, we’re losing potential speedup for a big dataset in a whole epoch, right?</p>

Khumbaba: <p>Turns out, I had a wrong understanding of how pytorch handles the workers. They work in parallel with each worker handling a separate batch. They don’t fetch samples in parallel to fill a single batch.</p>
Jacobian of function that takes a dict as input

espresso: <p>I have written some code which allows the user to create vector-valued functions of the form <code>f(x, y, theta)</code>, where <code>theta</code> is a dict containing parameters. The idea is that <code>x</code> and <code>y</code> will always be a part of the function but <code>theta</code> could be any number of different parameters depending on the function of interest. Note that I am not using these functions as part of a neural network module or anything like that. I am purely using the autograd functionality to differentiate functions.</p>
<p>For simplicity in this example, here is a scalar input-output function:</p>
<pre><code class="lang-auto">def f(x, y, theta):
    a, b = theta["a"], theta["b"]
    return x**4 + y**3 + a*b**2
</code></pre>
<p>Then I’d like to take the Jacobian as in</p>
<pre><code class="lang-auto">import torch
from torch.autograd.functional import jacobian

x = torch.tensor(5.)
y = torch.tensor(3.)
theta = {"a": 2, "b": 6}
jacobian(f, (x, y, theta))
</code></pre>
<p>This of course does not work since the <code>jacobian</code> expects a tuple of tensors as input. The error says that the function got a dict instead of a tensor or tuple of tensors. I tried to cast the dict to a tensor but this is not possible. I tried looking into  <code>Tensordict</code> but this does not appear to be accepted by the <code>jacobian</code> function. I also tried passing kwargs into the function which get unpacked in the function body. This does not work because <code>jacobian</code> is expecting <code>f()</code> to take two positional arguments (just <code>x</code> and <code>y</code>).</p>
<p>In Jax it is possible to differentiate w.r.t. a dict input and I was originally able to obtain the correct answer in that framework. Is there any way I could do this in PyTorch?</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/espresso">@espresso</a>,</p>
<p>Have a look at the <code>torch.func</code> package, which is a pytorch version of JAX! It can handle the parameters as input via the use of <code>torch.func.functional_call</code></p>
<p>Docs here: <a href="https://pytorch.org/docs/stable/func.html" class="inline-onebox" rel="noopener nofollow ugc">torch.func — PyTorch 2.0 documentation</a></p>
Getting 6fps instead of 30fps

Vikram_M: <p>I followed the <a href="https://pytorch.org/tutorials/intermediate/realtime_rpi.html" rel="noopener nofollow ugc">official tutorial</a> on my raseberry pi, but I got 6.5fps instead of 30fps?<br>
Any fix?<br>
(One small change which I made is: Used video which is saved in disk, instead of video)</p>

ptrblck: <p>Are you seeing the reported performance without the small change you’ve made?</p>

Vikram_M: <p>No, 6fps is for the video from the disk, and not the camera</p>
Recommend a kaggle dataset for university classes

alsrlspah0922: <p>Hi im university student in korea</p>
<p>I want some information about datasets which in Kaggle</p>
<p>Because, the final examination of our class is graded by Kaggle Leaderboard<br>
Professor told to students, if you propose proper datasets then i give you 1point and if that dataset is adopted then i give you 2point.</p>
<p>I made my poor middle examination so i realllly need a proper datasets for kaggle competition</p>
<p>thanks for recommendation <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

ptrblck: <p>Your question doesn’t seem to be PyTorch specific and it seems you are looking for some support to improve your grades. I don’t think this discussion board is the right place for this request.</p>
First time access torch.tensor too slow (strange thing)

Aakira: <p>Update : My question was too long, so I was finding a way to explain it shortly:D, ty for accept my question</p>
<p>Hi, I was training my model but I noticed something strange, I rewrited my code for easy look, look at these code below :</p>
<pre><code class="lang-auto">matched = torch.LongTensor(8732).to("cuda")
for epoch in range(config.start_epoch, config.end_epoch):
    for imgs, targets in dataloader:
        
        imgs = imgs.to(config.device)
        targets = [target.to(config.device) for target in targets]

        with torch.set_grad_enabled(False):
            loc, conf = net(imgs)  

            for i in range(5):
                t = time.time()
                matched[0] = -1
                print(time.time() - t)

        print("finished 1 iteration!")
</code></pre>
<p>I created <code>matched</code> tensor and pass <code>image</code> into my neural network, after that, I measure how much time it take to assign just 1 element in matched :</p>
<pre><code class="lang-auto">finished 1 iteration!
0.20601940155029297
0.0
0.0
0.0
0.006075859069824219
finished 1 iteration!
0.21062469482421875
0.0
0.0
0.0
0.0
finished 1 iteration!
</code></pre>
<p>You can see that the first time I assign <code>matched[0]</code> take so much time, but when I don’t pass images into neural network, it take fewer time :</p>
<pre><code class="lang-auto">matched = torch.LongTensor(8732).to("cuda")
for epoch in range(config.start_epoch, config.end_epoch):
    for imgs, targets in dataloader:
        
        imgs = imgs.to(config.device)
        targets = [target.to(config.device) for target in targets]

        with torch.set_grad_enabled(False):
            #loc, conf = net(imgs)  not pass images in to network

            for i in range(5):
                t = time.time()
                matched[0] = -1
                print(time.time() - t)

        print("finished 1 iteration!")
   
</code></pre>
<p>the result is :</p>
<pre><code class="lang-auto">0.0
0.0
0.0010001659393310547
0.0
0.0
finished 1 iteration!
0.0
0.0
0.0
0.0
0.0
finished 1 iteration!
</code></pre>
<p>I don’t understand why, please let me know the reason</p>
<p>// config.device = “cuda” in my code</p>

ptrblck: <p>CUDA operations are executed asynchronously so you would need to synchronize the code via <code>torch.cuda.synchronize()</code> before starting and stopping the timers. Otherwise you would measure the dispatching, kernel launches, and implicit synchronizations in your code.</p>

Aakira: <p>It has been very helpful to me, thank you</p>
How to infernece pytorch model on list of images?

sal_zak: <p>I have a list of numpy arrays each one represent an image and I want to predict all of them at same time.<br>
Here’s my code:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/9/c/9c753d1b1d976bab8944a3aecd02eccaa2b51908.png" alt="Screenshot from 2023-05-11 14-41-01" data-base62-sha1="mk5FqjqQV5El9Y4eWdDDG4gOhAs" width="313" height="313"><br>
But I got this error:<br>
imgs = torch.stack(imgs, 0)<br>
TypeError: expected Tensor as element 0 in argument 0, but got numpy.ndarray</p>

marksaroufim: <p>PyTorch doesnt work directly on numpy arrays, you need to convert them to torch tensors first, so as a first line you can add something like <code>imgs = [torch.from_numpy(img) for img in imgs]</code></p>

sal_zak: <aside class="quote no-group" data-username="marksaroufim" data-post="2" data-topic="179691">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/marksaroufim/48/36747_2.png" class="avatar"> marksaroufim:</div>
<blockquote>
<p><code>imgs = [torch.from_numpy(img) for img in imgs]</code></p>
</blockquote>
</aside>
<p>ok but the imgs is still a python list and can’t be fed to tranform or to the model itself</p>

knoriy: <pre><code class="lang-python">def predict(imgs, device):
    model.eval()
    with torch.no_grad():
        imgs = [torch.from_numpy(img) for img in imgs]
        imgs = torch.stack(imgs).to(device)
        out = model(imgs).detach().cpu().numpy()
    return out
</code></pre>
Dealing with teacher forcing and window length in seq2seq models

fuzhanr: <p>Hello pytorch community,</p>
<p>I have a question about handling “teacher forcing” in seq2seq while also applying a sliding window to improve my prediction.<br>
My seq2seq consists of an encoder and a decoder and for future time steps, I am predicting in an autoregressive fashion with a sliding window of 3 (so with a sliding window from t-3 to t I am predicting the step time t).  Here, I am also randomly applying (with decreasing probability over time) teacher forcing.<br>
When teacher forcing is applied, which timesteps should be replaced for the next prediction steps (t+1)? Just  “t” or also “t-1” and “t-2”?</p>
<p>Example: Window_lenght is 3 with steps 20, 21, 22 as input I for the step 22 there will be teacher forcing. For prediction step 23 should teacher forcing replace just the step 22 or also 20, 21 and 22?</p>
<p>Any hint or help would be greatly appreciated!</p>
Doubt about using seaborn.kdeplot()

Benja: <p>Hi!. I am usig seaborn.kdeplot() to obtain two density functions of different data. Here I post my code:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/c/4c0cc2b5d541cbf85d79c39864da94914270f2a5.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/4c0cc2b5d541cbf85d79c39864da94914270f2a5" title="WhatsApp Image 2023-05-13 at 12.23.07"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4c0cc2b5d541cbf85d79c39864da94914270f2a5_2_690x209.jpeg" alt="WhatsApp Image 2023-05-13 at 12.23.07" data-base62-sha1="aQLCSTW5VnZIt2f0B4Jpqd715s1" width="690" height="209" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4c0cc2b5d541cbf85d79c39864da94914270f2a5_2_690x209.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4c0cc2b5d541cbf85d79c39864da94914270f2a5_2_1035x313.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4c0cc2b5d541cbf85d79c39864da94914270f2a5_2_1380x418.jpeg 2x" data-dominant-color="EBEDED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">WhatsApp Image 2023-05-13 at 12.23.07</span><span class="informations">1600×485 152 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>My problem is that when I obtain the graphic, labels are changed. I suppose it is because x0,y0 corresponds to activ_reg_lineal_2hidden, and x1,y1 to activ_reg_lineal_numpy, but I do not understand why. If someone can explain me the reason would be grateful. Thanks in advance!!</p>
Fast instance segmentation program

Sylvain_Ard: <p>Hello,<br>
I am looking for a very fast instance segmentation program with a good recognition rate, which one do you recommend?<br>
Thank you<br>
Best regards</p>

Manuel_Alejandro_Dia: <p>Try the SOLO and SOLOv2 family, they may be of help!</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/WXinlong/SOLO">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/WXinlong/SOLO" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/13c899f35cd1414360110d6beab799edd9889998063b57ec0a2aa1ead739beff/WXinlong/SOLO" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/WXinlong/SOLO" target="_blank" rel="noopener nofollow ugc">GitHub - WXinlong/SOLO: SOLO and SOLOv2 for instance segmentation, ECCV 2020...</a></h3>

  <p>SOLO and SOLOv2 for instance segmentation, ECCV 2020 &amp; NeurIPS 2020. - GitHub - WXinlong/SOLO: SOLO and SOLOv2 for instance segmentation, ECCV 2020 &amp; NeurIPS 2020.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


Sylvain_Ard: <p>thank you very much but I fail to install it, please read my precedent message : <a href="https://discuss.pytorch.org/t/fail-to-install-solov2/179720" class="inline-onebox">Fail to install solov2</a><br>
best regards</p>
What is the license of libtorch?

dirard: <p>Hi! This question is not related to programming. I can’t get any clear answer from anywhere else so I’m trying my luck here.</p>
<p>I want to use libtorch without GPL.<br>
Pytorch license looks like BSD in LICENSE file.<br>
But in pytorch/c10/util/Array.h, Array.h looks like under the GPL3 license.</p>
<p>Is libtorch under the GPL3 license too?</p>

ptrblck: <p>CC <a class="mention" href="/u/jspisak">@jspisak</a> for licensing questions.</p>

wang_liang: <p>Hi dirard, did you find the answer?</p>
What is the C10:Error?

xwjBupt: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/2/72f839e242f5091e27b2f8b5482f561462887376.png" data-download-href="https://discuss.pytorch.org/uploads/default/72f839e242f5091e27b2f8b5482f561462887376" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/2/72f839e242f5091e27b2f8b5482f561462887376_2_690x314.png" alt="image" data-base62-sha1="gp4fITTc2o3KtmXKMCjZfnZMOkC" width="690" height="314" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/2/72f839e242f5091e27b2f8b5482f561462887376_2_690x314.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/2/72f839e242f5091e27b2f8b5482f561462887376_2_1035x471.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/2/72f839e242f5091e27b2f8b5482f561462887376_2_1380x628.png 2x" data-dominant-color="15232D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">2028×924 145 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>yesterday it was fine, and  i do not change my code, today is not ok, what sholud i do?</p>
<p>here is the python enviroment info:</p>
<p>torch-2.0.0<br>
torchvision-0.15.0</p>
Noob: what are these lines doing in calculating epoch loss and accuracy?

Nobutaka_Kim: <pre><code class="lang-auto">    epoch_loss = (epoch_loss / len(data_loader)).item()
    epoch_acc = (epoch_acc / len(data_loader.dataset)) * 100
</code></pre>
<p>from <a href="https://github.com/windsuzu/Titanic-Machine-Learning-and-Deep-Learning/blob/main/deep-learning.ipynb" class="inline-onebox" rel="noopener nofollow ugc">Titanic-Machine-Learning-and-Deep-Learning/deep-learning.ipynb at main · windsuzu/Titanic-Machine-Learning-and-Deep-Learning · GitHub</a></p>
<p>data_loader is a class that conveniently loads data in batches so how is that taking <code>.item()</code> from the math done inside the <code>(())</code>?</p>

srishti-git1110: <p>Hey,<br>
<code>.item()</code> returns the value of a (single valued) tensor as a python object.</p>
<p>In the first line, it’s just taking the average of the losses from all the batches in a single epoch to get the <code>epoch_loss</code>.</p>
<p>Similarly, <code>epoch_acc</code> is the average accuracy in the second line.</p>

Nobutaka_Kim: <p>Thanks <a class="mention" href="/u/srishti-git1110">@srishti-git1110</a> !</p>
FSDP + compile says "RuntimeError: assigned grad has data of a different size"

cliven: <p>Running the follow code with <code>torchrun --nnodes 1 --nproc-per-node 2 small.py</code> causes the compiler to fail with “RuntimeError: assigned grad has data of a different size”. Any thing obvious I am doing wrong here?</p>
<p>Code:</p>
<pre><code class="lang-python">import torch
from torch import nn
from torch.nn import functional as F
import os

from torch.distributed.fsdp import FullyShardedDataParallel as FSDP

class SwiGLU(nn.Module):
    def forward(self, x):
        x, gate = x.chunk(2, dim=-1)
        return F.silu(gate) * x

class SwiGLUMLP(nn.Module):
    def __init__(self, model_dim, device=None, dtype=None):
        super().__init__()
        hidden_dim = int(round(model_dim * 21 / 8))
        self.input = nn.Linear(model_dim, hidden_dim * 2, bias=False, device=device, dtype=dtype)
        self.output = nn.Linear(hidden_dim, model_dim, bias=False, device=device, dtype=dtype)
        self.act = SwiGLU()
        
    def forward(self, x):
        x = self.input(x)
        x = self.act(x)
        return self.output(x)
    
if __name__ == "__main__":
    from torch import _dynamo
    _dynamo.config.verbose=True
    local_rank = int(os.environ['LOCAL_RANK'])

    group = torch.distributed.init_process_group('nccl')
    torch.cuda.set_device(local_rank)

    model = SwiGLUMLP(32).to(f'cuda:{local_rank}')
    model = FSDP(model, use_orig_params=True, device_id=local_rank)
    compiled_model = torch.compile(model)
    input = torch.randn(1,32, device=f'cuda:{local_rank}')

    with torch.autocast('cuda'):
        loss = model(input).sum()
    loss.backward()

    with torch.autocast('cuda'):
        loss = compiled_model(input).sum()
    loss.backward()
</code></pre>

cliven: <p>Huh, error is because of running the model before running the compiled one - must be confusing something in the compiler?</p>
Batched torch.histc

Bianjiang_Yang: <p>Will torch.histc support batched inputs in the future? Or any tricky way to achieve this?</p>
<p>Assume I have Bxc input, B is the batch_size</p>
<pre><code class="lang-auto">histc_input = torch.rand(B, c)
histc_output = torch.histc(histc_input, bins=100, min=0, max=1)
</code></pre>
<p>I want a histc_output with the shape Bx100.<br>
Do you think it’s possible to implement?</p>

Bianjiang_Yang: <p>I found one possible solution using <code>one_hot</code> <a href="https://stackoverflow.com/questions/69429586/how-to-get-a-histogram-of-pytorch-tensors-in-batches" rel="noopener nofollow ugc">here</a></p>
<pre><code class="lang-auto">def batch_histogram(data_tensor, num_classes=-1):
    """
    Computes histograms of integral values, even if in batches (as opposed to torch.histc and torch.histogram).
    Arguments:
        data_tensor: a D1 x ... x D_n torch.LongTensor
        num_classes (optional): the number of classes present in data.
                                If not provided, tensor.max() + 1 is used (an error is thrown if tensor is empty).
    Returns:
        A D1 x ... x D_{n-1} x num_classes 'result' torch.LongTensor,
        containing histograms of the last dimension D_n of tensor,
        that is, result[d_1,...,d_{n-1}, c] = number of times c appears in tensor[d_1,...,d_{n-1}].
    """
    return torch.nn.functional.one_hot(data_tensor, num_classes).sum(dim=-2)
</code></pre>
<p>But <code>data_tensor</code> must be converted to <code>Torch.LongTensor</code> or <code>Torch.cuda.LongTensor</code> depending on the device, if input images with shape (b,3,h,w) are in the value range (0,1), and if we want a (b,3,num_classes) histogram output for each channel, each batch,</p>
<p>Then what we need to do is</p>
<pre><code class="lang-auto">num_class = 10 # take 10 as an example
b, c, h, w = image.shape
output = batch_histogram(data_tensor=(image*num_class).type(torch.LongTensor).reshape(b,c,h*w), num_class=num_class)
</code></pre>
<p>The <code>one_hot</code> function occupies lots of memory as expected.</p>
My trained model predicts only label=1

kuaranir: <p>Please help. I have trained my <code>resnet50</code> model for binary classification of pneumonia.<br>
But every my test image model <code>predicts as only 1 label</code>.<br>
Here is my GoogleColab notebook and my code for prediction is in the end of the ipnb.<br>
<a href="https://drive.google.com/file/d/19NbG66zGk1PMbf5M0Zb04rYPZv3yX4zW/view?usp=share_link" rel="noopener nofollow ugc">Colab notebook with my code</a></p>
Prevent duplicated copy of tensor to CPU

thyeros: <p>Hello. This is about extending save_on_cpu: <a href="https://pytorch.org/docs/stable/_modules/torch/autograd/graph.html#save_on_cpu" class="inline-onebox" rel="noopener nofollow ugc">torch.autograd.graph — PyTorch 2.0 documentation</a></p>
<p>In my case, I found the same tensor got saved to CPU twice due to some branches in the forward path, and I like to avoid it. Hence, I did the following, For an incoming tensor X,</p>
<pre><code class="lang-auto">global_tensor_on_cpu = {}

key_x = key_func(X)
if key_x not in global_tensor_cpu:
   global_tensor_on_cpu[key_x] = x.cpu()
 
return key_x
</code></pre>
<p>But, I found once I start applying this, the model accuracy drops for some reason. I wonder if there can be any side-effect with doing this. Also, as a follow-up question, is there any good “key_func” to get the unique id of a tensor? I’m using a mix of the shape, data_ptr, and dististribution info, but it seems a bit expensive to compute.</p>
Sparse Gradients and Computational Efficiency

zabboud: <p>I have a use case which involves updating specific parts of a tensor while keeping others fixed. I have successfully achieved this using the <code>register_hooks</code> method, which allows me to manipulate the gradients of the tensor (by zeroing the gradients of the elements that I don’t want to update). However, this approach does not fully exploit the sparsity of the gradients, resulting in high computational costs.</p>
<p>I am seeking advice on how to improve the computational efficiency of my approach, considering the sparse nature of the gradients. I would like to reduce the computational cost by performing operations and backpropagation only on the non-zero gradient elements of the tensor.</p>
<p>I would appreciate any guidance or recommendations on how to efficiently update only the desired elements of the tensor.</p>
Can't install torch on macOS

orome: <p>On macOS (Ventura 13.2.1, MBA M2)</p>
<pre><code class="lang-auto">pip install torch torchvision 
</code></pre>
<p>gives</p>
<pre><code class="lang-plaintext">ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
ERROR: No matching distribution found for torch
</code></pre>
<p>Just starting out with Torch here but hit an immediate dead end just following the first few lines of the documentation with a fully uptodate environment.</p>
<hr>
<pre><code class="lang-bash">$ which -a python
.../venvs/models/bin/python
/opt/homebrew/opt/python@3.11/libexec/bin/python
$ python -VV
Python 3.11.2 (main, Feb 16 2023, 02:55:59) [Clang 14.0.0 (clang-1400.0.29.202)]
</code></pre>

ptrblck: <p>I’m unsure if Python 3.11 is already supported on Mac, so could you downgrade Python to e.g. 3.10 and rerun the install command?</p>

orome: <p>PyTorch 2.0 works fine!</p>
Using multiple models in single microservice

Ivan_Sizykh: <p>I am building service and microservice for all ml models for it.</p>
<ol>
<li>Are there downsides and which is most reasonable way to do it?</li>
<li>If I had to split models in two microservices running on single GPU (even more to say, if one of them is docker container and another systemctl daemon) would it eventually backfire and how?</li>
</ol>
Torch.cuda.is_available() is True while I am using the GPU

ChangGao: <p>Hi! I have a doubt about how the <code>torch.cuda.is_available()</code> works.<br>
While training my network, I usually use the code:</p>
<pre><code class="lang-auto">device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
network.to(device)
data.to(device)
...
</code></pre>
<p>But I found that <code>torch.cuda.is_available()</code> is still <code>True</code> when the network is being trained.<br>
I am not sure why this happens. Does this mean that the code isn’t running on a GPU?</p>

albanD: <p>Hi,<br>
<code>torch.cuda.is_available()</code> is just telling you whether or not cuda is available. If it is available, that flag will remain True throughout your program.</p>

ChangGao: <p>Thanks a lot! I thought it was a way to find out whether I can use the GPU.<br>
Is there any way I can find out if my program is really using the GPU?</p>

albanD: <p>You can use <code>nvidia-smi</code> in your command line to check that your program is using some ressourses on your gpu.</p>

ChangGao: <p>Thanks. I know this command could work.<br>
By the way, do you think that I can train two networks with the same GPU at the same time?<br>
I just found that no error is returned while doing this, but I thought that the GPU couldn’t handle two tasks at the same time.</p>

albanD: <p>You can do it, that’s no problem.<br>
Now wether it’s going to be faster than running the two one after the other is very hard to guess. Depends on how much each task use the GPU. You will need to check.</p>

ChangGao: <p>Okay! Thanks a lot! That’s very helpful.</p>

wd_c: <p>*My return value is true but I can’t call the gpu，why？</p>
Implementation of PCGrad in Pytorch Lightning

haldunbalim: <p>Hello,</p>
<p>I would like to implement the paper <a href="https://arxiv.org/abs/2001.06782" class="inline-onebox" rel="noopener nofollow ugc">[2001.06782] Gradient Surgery for Multi-Task Learning</a>.<br>
In summary, I am working on a multitask learning setting so there are more than one loss function. I need to calculate separate gradients for each of these losses and manipulate them depending on their cosine similarity.</p>
<p>In standart pytorch I can do it using torch.autograd.grad; however, after calculating these gradients and manipulating them I would like to pass them to optimizer (no need to recalculate). How can I do it in torch lightning? Can I call manual_backward for each loss in a single train_step, or can I pass the summed up calculated gradients to optimizer?</p>
<p>Thanks for the help<br>
Haldun</p>

panpan: <p>I encountered similar things recently, have you solved the problem?</p>

haldunbalim: <p>Unfortunately, no.</p>
<p>I was using simple MLP’s after convolutional backbone and I calculated their derivatives in runtime with torch.autograd which is redundant but allowed me to implement PCGrad without changing the structure of my code a lot.</p>
<p>An alternative solution can be implemented using hooks to check derivatives but may involve more tedious design.</p>

panpan: <p>I see. Thanks a lot.</p>

MingChaoXu: <p>Does Pcgrad work? i also try to use this algorithm, but it doesn’t work in my experiments</p>
Torch.cuda.is_available() never returns anything

Ivan_Sizykh: <p>After I restarted docker container with torch based application torch.cuda.is_available() freezes there, not letting to start application over. Not only it never returns anything inside container now, but also on outside as well in different environments.</p>
<p>CUDA Version: 12.0. OS Ubuntu 20.04. Container image: pytorch/pytorch:1.9.0-cuda11.1-cudnn8-devel</p>
<p>The last error in logs before it started happening:<br>
Traceback (most recent call last):<br>
File “/code/app.py”, line 2, in <br>
from few_pixels_as_function import few_pixels_attack<br>
File “/code/few_pixels_as_function.py”, line 24, in <br>
seed = torch.tensor(seed).to(device)<br>
RuntimeError: CUDA error: all CUDA-capable devices are busy or unavailable<br>
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.<br>
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.</p>
<p>dmesg --level=err returns nothing. warn level as well.</p>
<p>Any help much appreciated</p>
<p>Update: I checked what now torch.cuda.memory_reserved() and torch.cuda.memory_allocated return. It’s zero in both cases, even thou nvidia-smi shows me it’s not true. torch.cuda.init() also stucks indefinetely. Is it something with drivers?</p>

ptrblck: <p>The errors sound like driver issues and I would recommend reinstalling them and updating PyTorch to the latest version if possible.</p>
Resume training from a specific batch in the dataloader

Adam_Oud: <p>Hello,<br>
Is there a way to resume training from a specific batch in the data loader.<br>
What I mean is instead of looping the dataloader from the first batch, to start the loop of the dataloader from a given batch “k”. Something like this would work if the dataloader was a simple list:</p>
<pre><code class="lang-auto">from torch.utils.data import DataLoader
loader = DataLoader([...])
for batch in loader[k:]:
    # training loop
</code></pre>
<p>My use case for this is that I want to resume training, with the same RNG seed for reproducibility, but with one hyperparameter changed.</p>
<p>Thanks for your help!</p>

amin-nejad: <p>Hi Adam, did you find a way to do this?</p>

Adam_Oud: <p>Not really…<br>
I tried the solution to fast forward an iterator on the dataloader to a specific batch, but this takes quite some time on large datasets since you have to load all batches in memory.</p>
<pre><code class="lang-auto">iterator = iter(loader)
for i in range(len(loader)):
    data = next(iterator)
    while i &lt; k: # k would be the batch index where to resume training
        continue
    # training loop
</code></pre>

amin-nejad: <p>No worries, thank you</p>

shenshanf: <p>your solution might be wrong if you use <code>shuffle==True</code> in dataloader, you should save the generator state in dataloader</p>
Best practice for porting Pytorch model to C++

netty: <p>I have an audio-related usecase in mind that requires a model in C++ for use on desktop (Mac/Windows) and mobile.</p>
<p>I have both the Python code and the pretrained model weights.</p>
<p>I have started porting the code to use Torchscript, but through reading some posts on here, it seems like Torchscript is in maintenance mode? And no more features are going to be developed?</p>
<p>What’s the recommended way going forward to port a pytorch model to C++?</p>
<p>Don’t want to learn something that is outdated or going away soon.</p>

netty: <p>Maybe that was too broad.</p>
<ol>
<li>What does “maintenance mode” mean at Pytorch? (for torchscript support)</li>
<li>Is Pytorch deliberately trying to stay away from the C++ conversion functionality because it’s not their core competency? or is there another reason.</li>
</ol>

spiegelball: <p>Whilst not being an answer to your question, this post is directly related: <a href="https://discuss.pytorch.org/t/pytorch-2-and-the-c-interface/168034/6" class="inline-onebox">Pytorch 2 and the c++ interface - #6 by ezyang</a></p>

marksaroufim: <p>Torchscript maintenance mode means there’s very little dev attention going to it, as in no new features, if critical bugs are found then they will probably be fixed. many torch.compile backends also leverage torchscript so it’s not going away anytime soon but won’t evolve all that much either</p>
<p>As of today you’re right there isn’t a good story for a conversion in environments where python is not available, eventually that’s supposed to be <code>torch._dynamo.export()</code> but that’s still in very early days</p>
Is there a .cuda() equivalent for MPS?

Pranav_Bharadwaj: <p>Is there a .cuda() equivalent for MPS? For example, X_test.cuda() in MPS is what? Is it automatic if the device is “mps”?</p>

marksaroufim: <p>I don’t see one so yes you would need to add <code>to()</code> calls or make sure your tensors are instantiated on an MPS device</p>
<p>Alternatively something I’ve been using quite a bit is this global flag <a href="https://pytorch.org/docs/stable/generated/torch.set_default_device.html" class="inline-onebox" rel="noopener nofollow ugc">torch.set_default_device — PyTorch 2.0 documentation</a></p>
Unable to Import TorchDispatch

Vinayaka_Hegde: <p>Hi, I’m unable to figure out how to import torchdispatch for PyTorch v1.12.0.<br>
The same import statement works without error for Pytorch2.0, but I needed to use it for Pytorch v1.12.0</p>
<p>I’m getting the below errror :</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/c/6c67a0bfd927ac811158344e1703ed99c24f0a11.png" data-download-href="https://discuss.pytorch.org/uploads/default/6c67a0bfd927ac811158344e1703ed99c24f0a11" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/c/6c67a0bfd927ac811158344e1703ed99c24f0a11.png" alt="image" data-base62-sha1="fsZAuEndvHMYYLuQqvSgvXncE01" width="690" height="45" data-dominant-color="131313"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1832×120 7.44 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>The import statement looks like below :<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/2/b229498f50db9abeee0f42e68a947e57863cf5a7.png" alt="image" data-base62-sha1="pq5rK3jWB1Q5kAb997hr6TTmO8f" width="559" height="236"></p>

marksaroufim: <p>I do see TorchDispatchMode show up here <a href="https://github.com/pytorch/pytorch/blob/release/1.12/torch/utils/_python_dispatch.py#L113" class="inline-onebox" rel="noopener nofollow ugc">pytorch/_python_dispatch.py at release/1.12 · pytorch/pytorch · GitHub</a></p>