akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Cuda 12 pytorch 11.8

Vikas_Agrawal: <p>Sir, i purchased new machine nvidia Geoforce rtx 4090 graphics card it takes as cuda 12 by default but in pytorch the latest version is 11.8 how can I run it .it’s not compatible please reply</p>

eqy: <p>The RTX 4090 is sm89 which should be usable with CUDA 11.8. Note that if you install the 11.8 pip wheels e.g., via <code>pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118</code> PyTorch will use the CUDA that is bundled with it (11.8 in this case) even if your system has CUDA 12. Please let us know if you have already tried this and it did not work.</p>

NiKiuS: <p>If you’re working with Pytorch+cu118 in Stable Difussion, using some WebUI like Comfy or A1111, it’s better to maintain with CUDA toolkit 11.8 even when 12.2 update is available. The UIs will work fine with both versions, but the image segmentation (SAM), with some ultralytics (YOLO), will give you some unexpected results with cu122, and cu118 will give you better results. So, it’s better to wait Pytorch cu122 compatible version to be sure to update CUDA toolkit.</p>
<p>In any other case (not using StableDifussion), it’s better cu122.</p>
RuntimeError: Quantized cudnn conv2d is currenty limited to groups = 1; received groups =16

ANKIT_KUMAR1: <p>Error during Quantazation awer  training<br>
quantization code like<br>
backend = “qnnpack”<br>
model.qconfig = torch.quantization.get_default_qat_qconfig(backend)<br>
quantized_model = torch.quantization.prepare_qat(model, inplace=False).to(device)</p>
<p>getting error during convert the model;</p>
<p>model_qat = torch. quantization.convert(model, inplace=False)</p>
Mat1 and mat2 shapes cannot be multiplied (61200x3 and 61200x42)

LunaR: <p>Hi!</p>
<p>I’m new at PyTorch and getting “mat1 and mat2 shapes cannot be multiplied (61200x3 and 61200x42)” on “x = self.fc(x)”.</p>
<p>My 3000 input tensors had the shape (1, 51, 3) and I want 3000 output tensords each of the shape (1, 42, 3)</p>
<pre><code class="lang-auto">import torch.nn as nn

class Net(nn.Module):

    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(in_channels=1, out_channels=12, kernel_size=3, stride=1, padding=1)
        self.conv2 = nn.Conv2d(in_channels=12, out_channels=12, kernel_size=3, stride=1, padding=1)
        self.conv3 = nn.Conv2d(in_channels=12, out_channels=24, kernel_size=3, stride=1, padding=1)
        self.drop = nn.Dropout2d(p=0.2)
        self.fc = nn.Linear(61200, 42)

    def forward(self, x):
        print(x.shape)
        x = F.relu(self.conv1(x))
        print(x.shape)
        x = F.relu(self.conv2(x))
        print(x.shape)
        x = F.relu(self.drop(self.conv3(x)))
        print(x.shape)
        x = F.dropout(x, training=self.training)
        print(x.shape)
        x = self.fc(x)
        return F.softplus(x, dim=2)
</code></pre>
<p>The layers shapes change like that<br>
torch.Size([50, 1, 51, 3])<br>
torch.Size([50, 12, 51, 3])<br>
torch.Size([50, 12, 51, 3])<br>
torch.Size([50, 24, 51, 3])<br>
torch.Size([50, 24, 51, 3])<br>
before the “x = self.fc(x)”</p>
<p>Will be thankfull for help</p>

ptrblck: <p>It seems you are forgetting to flatten the activation via: <code>x = x.view(x.size(0), -1)</code> or an <code>nn.Flatten</code> layer before passing it to the <code>self.fc</code> layer.<br>
Once this is done you will run into another shape mismatch error since the activation has <code>24 * 51 * 3 = 3672</code> features, so set the <code>in_features</code> argument of <code>self.fc</code> to <code>3672</code>.</p>

LunaR: <p>This works, thank you!</p>
How do the transforms actually work in PyTorch?

Soumya_Kundu: <pre><code class="lang-auto">torchvision.transforms.v2.RandomEqualize(p: float = 0.5)
</code></pre>
<p>If I increase the probability to 1 here, does that mean it equalizes all the images and the original images are never seen in the network?</p>
<p>Is there any way to see how many images are actually being shown to the network. Like a dry run to see the number of images passing with and without augmentation?</p>

JamesDickens: <p>A random number is generated from U(0, 1). The function initialized a probability threshold. If the value is larger thsn the threshold the whole batch is histogram equalized, image per image.</p>

Soumya_Kundu: <p>Thank you for the answer! Clears my doubt.</p>
<p>So essentially then, if the probability is set as 1, it acts as a pre-processing step rather than an augmentation? Am I right in saying this?</p>

JamesDickens: <p>Ya I think you could write something like</p>
<pre><code class="lang-auto">transform = torchvision.transforms.v2.RandomEqualize(p=1.0)
ims = transform(ims) # ims a tensor of shape [N, 3, H, W] or [N, 1, H, W]
</code></pre>
<p>where N is the batch size, 3 or 1 is the number of channels, H is the image height, W is the image width, and this would histogram equalize the batch.</p>
<p>Alternatively if you wanted to randomly histogram equalize within the batch, I think you could do something like</p>
<pre><code class="lang-auto">for i in range(ims.shape[0]):
    if random.rand() &gt; 0.5:
         ims[i] = transform(torch.unsqueeze(ims[i], dim=0))[0]
</code></pre>
Problem with torchvision dataset type information

Mah_Neh: <p>Hi, been enjoying this library a lot, there are two single problems that I couldn’t solve:</p>
<ol>
<li>Warning keeps being printed</li>
<li>Type information error for Dataset.</li>
</ol>
<p>For the first one, I am using as recommended:</p>
<pre><code class="lang-auto">torchvision.disable_beta_transforms_warning()
</code></pre>
<p>But the warning keeps being printed ?</p>
<p>For the second one, this is a code example:</p>
<pre><code class="lang-auto">def train_loop(
    dataloader: DataLoader,
    model: nn.Module,
    loss_fn: torch.nn.CrossEntropyLoss,
    optimizer: torch.optim.Adam,
):
   # inside fn
    dataset_size = len(dataloader.dataset)  # type: ignore
   ...
</code></pre>
<p>Notice that I have to use type ignore or it complains with this error message:</p>
<blockquote>
<p>Argument of type “Dataset[Unknown]” cannot be assigned to parameter “__obj” of type “Sized” in function “len”<br>
“Dataset[Unknown]” is incompatible with protocol “Sized”<br>
“<strong>len</strong>” is not presentPylance<a href="https://github.com/microsoft/pyright/blob/main/docs/configuration.md#reportGeneralTypeIssues" rel="noopener nofollow ugc">reportGeneralTypeIssues</a></p>
</blockquote>
<hr>
<p>Help or pointers are highly appreciated ! Thank you !</p>
Optimization of torch DL model based on mask2former Architecture for NVIDIA Jetson platform

saprakash: <p>Hello Team,<br>
We have a  DL model called panoptic segmentation developed with pytorch framework. We want to optimize this model for NVIDIA JETSON platform to achieve better inference. Model used recent transformer based (mask2former) architecture.<br>
Could you pls suggest any good optimization techniques which can speed-up inference and have maximum FPS? Since it is used transformer based architecture, we are facing difficulties to optimize.</p>
<p>Could you please suggest?</p>
<p>Thanks<br>
Prakash Sahoo</p>
How to make a dataloader with a directory of subfolders relevant to each class

Sasika_Amarasinghe: <p>I have a dataset that contains images of brain tumoursl.  I want to make a CNN to classify these images.What I have seen is a directory of images which is separated in “train” , “test”  folders.</p>
<p>However, in this case, the dataset directory structure is as follows.<br>
dataset_dir<br>
|_____tumor_type_1<br>
|_____tumor_type_2<br>
|_____tumor_type_3<br>
|_____no_tumor</p>
<p>Now, I want to make 3 dataloaders. ( a train_dataloader,a validation_dataloader &amp; a test_dataloader.)<br>
Does anyone know how to do this in PyTorch.</p>

ptrblck: <p>You could use the <code>torchvision.datasets.ImageFolder</code> class for the datasets, create the splits via <code>torch.utils.data.Subset</code>s, and wrap them in <code>DataLoader</code>s.</p>
Set requires_grad=False for earlier layers in a model

Kevin_Dong: <p>I am trying to fine-tune a pre-trained model by freezing different parts of the model. Say the model has an encoder and decoder. If I only want to train the encoder, I can freeze the decoder by settings its Tensor to be requires_grad=False. However, does it block the gradient flow back to the encoder?</p>
<p>In practice, I found the autograd still computes the gradient for the encoder even the grad for decoder is 0. I am just curious what <code>requires_grad</code> means for Tensor. Is it blocking all gradients or just discarding the records of gradient, even once being computed?</p>

ptrblck: <aside class="quote no-group" data-username="Kevin_Dong" data-post="1" data-topic="187063">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kevin_dong/48/63458_2.png" class="avatar"> Kevin_Dong:</div>
<blockquote>
<p>However, does it block the gradient flow back to the encoder?</p>
</blockquote>
</aside>
<p>No, it doesn’t and Autograd is smart enough to backpropagate the gradients to earlier parameters.</p>
<aside class="quote no-group" data-username="Kevin_Dong" data-post="1" data-topic="187063">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kevin_dong/48/63458_2.png" class="avatar"> Kevin_Dong:</div>
<blockquote>
<p>Is it blocking all gradients or just discarding the records of gradient, even once being computed?</p>
</blockquote>
</aside>
<p>Autograd will use this attribute to decide if a gradient computation is needed or not.<br>
E.g. freezing the parameter will reduce the <code>wgrad</code> kernels as seen in <a href="https://discuss.pytorch.org/t/for-freezing-certain-layers-why-do-i-need-a-two-step-process/175289/2">this example</a>.</p>
Proper way to create Hessian as tensor with torch.func.hessian

hlzl: <p>When using <code>torch.func.hessian</code> to compute the Hessian of a model’s parameters with regard to the input loss, you end up with a nested <code>dict</code>.</p>
<p>Working with a nested <code>dict</code> can be cumbersome, especially for mathematical operations. I’m thus converting it to a <code>torch.tensor</code>, in the form of a standard symmetric Hessian matrix, using the script below.</p>
<p>However, this is unintuitive and requires quite a good number of lines of code.</p>
<p>Is there a way to do this with less lines of code and more efficiently?</p>
<pre><code class="lang-auto">import torch

model = torch.nn.Linear(2, 2)
params = dict(model.named_parameters())

x = torch.randn(2)
y = torch.tensor(1)


def f(model, params, input, target):
    predictions = torch.func.functional_call(model, params, (input,))
    return torch.nn.functional.cross_entropy(predictions, target)


hessian_dict = torch.func.hessian(f, argnums=1)(model, params, x, y)

hessian = None
# Loop over layers
for key in hessian_dict:
    param_dim = params[key].numel()

    # Using list comprehension for the inner loop (derivatives of one layer)
    concat_tensor = torch.cat(
        [hessian_dict[key][k].view(1, param_dim, -1) for k in hessian_dict[key]],
        dim=2,
    )

    # Concat all second derivatives of that layer in same dimension
    hessian = (
        torch.cat([hessian, torch.sum(concat_tensor, dim=0)], dim=0)
        if hessian is not None
        else torch.sum(concat_tensor, dim=0)
    )
</code></pre>
Computing Hessian diagonals efficiently with torch.func

hlzl: <p>I’m looking for ways to compute the diagonal of a model’s parameter Hessian with regard to the model’s input loss efficiently, ideally with <code>torch.func</code> and its <code>vmap</code> capabilities.<br>
Due to memory constraints, I want to avoid computing the full Hessian and instead opt for an implementation using a Hessian vector product.</p>
<p>So far, I’ve been able to correctly compute the diagonal values, but the computation is incredibly slow. Any ideas how to make this computationally faster?</p>
<pre><code class="lang-auto">import torch


def f(model, params, input, target):
    predictions = torch.func.functional_call(model, params, (input,))
    return torch.nn.functional.cross_entropy(predictions, target)


def hvp(f, model, params, input, target, v):
    return torch.func.grad(
        lambda params: torch.dot(
            torch.cat(
                [
                    p.flatten()
                    for p in torch.func.grad(f, argnums=1)(
                        model, params, input, target
                    ).values()
                ]
            ),
            v,
        )
    )(params)


model = torch.nn.Linear(2, 2)
params = dict(model.named_parameters())

x = torch.randn(2)
y = torch.tensor(1)

num_params = sum(p.numel() for p in params.values())
hessian_diag = None
for p_i in range(num_params):
    v = torch.zeros(num_params)
    v[p_i] = 1
    hvp_dict = hvp(f, model, params, x, y, v)
    hessian_diag = (
        torch.cat(
            [
                hessian_diag,
                torch.cat([hvp_dict[param].flatten() for param in hvp_dict])[
                    p_i
                ].unsqueeze(0),
            ],
            dim=0,
        )
        if hessian_diag is not None
        else torch.cat([hvp_dict[param].flatten() for param in hvp_dict])[
            p_i
        ].unsqueeze(0)
    )
</code></pre>
<p>I tried to vectorise the computation of the dot product with no luck. That would definitely speed up the computation, however, I’m open to completely different ideas for this as well.</p>
Exploit shared memory between CPU and GPU of Jetson devices

texflip: <p>I have a Jetson Nano Orin which I’m running a model with TensorRT.<br>
I’ve seen that the most time consuming operation is when I transfer a torch Tensor into the cuda device using the function to(“cuda:0”).</p>
<p>By profiling with torch.profiler, I’ve seen that the 97% of the overhead came from the copy of the tensor. However the Jetson Orin has shared memory between the CPU and the GPU, so technically I could avoid to copy the tensor because the GPU has the access to the same memory.</p>
<p>It is possible to transfer the tensor’s device without copying it? If yes, how?</p>
Numworkers AttributeError: 'TTree' object has no attribute 'x'

joachiha: <p>Hi All,</p>
<p>I have an issue when trying to set num_workers above 0 in my DataLoader.</p>
<pre><code class="lang-auto">class MyDataSet(Dataset):
    def __init__(self,TTree):
        self.TTree = TTree

    def __getitem__(self,idx):

        self.TTree.GetEntry(idx) 

        return self.TTree.X 

    def __len__(self):
        return self.TTree.GetEntries()

</code></pre>
<p>Resulting in:</p>
<pre><code class="lang-auto">AttributeError: 'TTree' object has no attribute 'X'
</code></pre>
<p>I am using ROOT framework by cern for my data.</p>
<p>TTree.GetEntry(idx) sets the address of the variable X corresponding to the index. It work’s perfectly fine when I have num_workers=0, but it seems that when I increase this number, that it loses track of the address somewhere.</p>
<p>Is there an easy fix for this?</p>
<p>Cheers</p>
How to apply transforms to satellite Images and its masks

Anishkumar_Iyer: <p>I have satellite a Image segmentation task<br>
With 3 channel Images and a 1 channel Binary mask</p>
<p>I have troubles implementing Transforms</p>
<p>Earlier I was working with these transforms which worked well</p>
<pre><code class="lang-auto"> img_train_transform = transforms.Compose(
        [
            transforms.ToTensor(),
            transforms.Resize(
                (config.INPUT_IMAGE_HEIGHT, config.INPUT_IMAGE_WIDTH),
                antialias=False,
            ),

mask_val_transform = transforms.Compose(
        [
            transforms.ToTensor(),
            transforms.Resize(
                 (config.INPUT_IMAGE_HEIGHT, config.INPUT_IMAGE_WIDTH),
                 interpolation=InterpolationMode.NEAREST,
                antialias=False,
             ),
</code></pre>
<p>But my model accuracy was just around 0.6 dice score.<br>
I figured I need to use some data augmentation,</p>
<p>But whenever I try to add a any new transform like this</p>
<pre><code class="lang-auto">transforms.ToTensor(),
transforms.Resize(
                (config.INPUT_IMAGE_HEIGHT, config.INPUT_IMAGE_WIDTH),
                antialias=False,
            ),
transforms.RandomRotation(180)
</code></pre>
<p>The predictions get distorted and Eventually die out in few epochs</p>
<p>I tried to implement the transforms in another way that changes to PIL first then to Tensor like this</p>
<pre><code>       transforms.ToPILImage(),
       
        transforms.Resize(
            (config.INPUT_IMAGE_HEIGHT, config.INPUT_IMAGE_WIDTH),
            antialias=True,
        ),
        
        transforms.ToTensor(),
</code></pre>
<p>I get predictions like this after 5 epochs, these are very distorted</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/7/27e5d073eb51c0c29f25479ec53126f4115e0841.png" data-download-href="https://discuss.pytorch.org/uploads/default/27e5d073eb51c0c29f25479ec53126f4115e0841" title="pred_25_1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/7/27e5d073eb51c0c29f25479ec53126f4115e0841_2_690x87.png" alt="pred_25_1" data-base62-sha1="5GWZLbRmdg0XSJ4T4Lrr23VSeqZ" width="690" height="87" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/7/27e5d073eb51c0c29f25479ec53126f4115e0841_2_690x87.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/7/27e5d073eb51c0c29f25479ec53126f4115e0841_2_1035x130.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/7/27e5d073eb51c0c29f25479ec53126f4115e0841_2_1380x174.png 2x" data-dominant-color="4B4B4B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">pred_25_1</span><span class="informations">1554×196 143 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I’m stuck for the past 3 days<br>
Please help !</p>
Reference for scale_grad_by_freq option in nn.Embedding

111146: <p>I found the scale_grad_by_freq option in the nn.Embedding module.</p>
<p>However, I couldn’t find the reference for that option.</p>
<p>Does anyone know which survey or paper used this technique?</p>
<p>Thank you.</p>

gene_takavic: <p>Can anyone provide some updates on this? I wanted to know the motivations and the results of using this technique for learning embeddings…</p>
What is `model.device` in FSDP?

KUSHAL_JAIN: <p>I am finetuning a llama model using FSDP on 2 GPUs. I am debugging an error, so I am printing  <code>model.device</code> during runs. Unintuitively, on different iterations, different device ids are printed as <code>model.device</code> (sometimes 1 and sometimes 0). This has got me to wonder what actually is <code>model.device</code> when using FSDP. Because the model is sharded across both devices, what determines this value? Furthermore, if I want to know on which device my input layer is sharded to (since that should be the actual model device), how do I do that? How do I ensure that I am casting my inputs to the correct device id (or how does torch does this if it’s done automatically)? I am running into obscure <code>illegal memory access</code> errors and I need to get to the root of this.<br>
Thanks</p>

agu: <p>Did you call something like <code>torch.cuda.set_device(local_rank)</code>?</p>
<p>As far as I know, <code>FullyShardedDataParallel</code> instances do not have a <code>.device</code> attribute. Do you mean <code>.compute_device</code>? If so, the <code>compute_device</code> should be set at FSDP initialization time and never changed thereafter.</p>
Lightning MultiGPU Training + Merlin-Dataloader

AleTL: <p>Hello!</p>
<p>I want to train a model with multiple GPUs. For that I am using Lightning since the API makes it easier. However, I am using a Merlin-dataloader module as data module for the Lightning trainer. This way, I call the trainer like this: <code>trainer.fit(model=model, datamodule=Merlin_module)</code>.</p>
<p>In my Merlin module (<code>Merlin_module</code>), each GPU should access to a different part of the dataset, which is determined by the name of the files it retrieves. For getting the correct files, the datamodule must access to the device name of the process that is accessing the files. So now, the question is: how can I know, within the datamodule, which is the GPU that is executing it?</p>
<p>My datamodule has a function <code>train_dataloader</code> that generates the dataset and puts it in a <code>Loader</code> from Merlin. Ideally, I would like to do something like:</p>
<pre><code class="lang-auto">def train_dataloader(self):

rank = get_rank() # get rank of device

data = get_dataset(rank) # get correspondent files for the device

return Loader(data)
</code></pre>
<p>Nevertheless, I don’t know how to access the rank within the datamodule. Hence, my questions is: can that be done? If so, how?<br>
Cause maybe the datamodule is just called in the main process and then the problem is even more complex to be solved.</p>
<p>Thanks a lot!!</p>
CarRacing not learning with A2C in torchrl

phoenix1: <p>I am trying to code the Gym env CarRacing-v2 using torchrl, I have mostly followed the <a href="https://github.com/pytorch/rl/blob/bd316e9abe6e8fcab9513f3120b07ee3c954120a/examples/a2c/a2c.py" rel="noopener nofollow ugc">example of torchrl</a> But even after about 400 episodes, it doesn’t give any good results. Am I collecting the collectors data correctly? I just don’t know why it’s not learning properly? Is the network too basic?</p>
<p><a href="https://colab.research.google.com/drive/1R3uPY821ihNOgpkXup0ifztHLeKjf5j5?authuser=1#scrollTo=gmngI6edLGzR" rel="noopener nofollow ugc">Here is the code on Colab</a></p>
<p>I’d like to know if this is the right method to go about creating a proper actor critic method for this environment, also I would really appreciate for any suggestions of  suitable upgrades in the network…</p>
Better accuracy when using fewer GPUs w/ DistributedDataParallel

Roy1: <p>Hi,</p>
<p>I am finding that my model trains a lot better when using 2 GPUs (batch size 512 per GPU) than using 8 GPUs (batch size 128 per GPU). The learning rates are both scaled to be the same, i.e:</p>
<blockquote>
<p>base_lr * batch_size * world_size / 512.</p>
</blockquote>
<p>The observed training losses after every iteration are pretty much the same for both settings too (2 GPUs or 8 GPUs). The testing function also gives the same accuracy if I use 2 GPUs or 8 GPUs with the same set of checkpoint weights.  From my understanding, the validation accuracy should be roughly the same in both of these settings right? I am using DDP for both and so the only difference is just the number of GPUs.</p>
<p>Here are some of the logs.<br>
2 GPUs val logs:</p>
<blockquote>
<p>epoch,acc1,acc5<br>
0,1.99600,6.77200<br>
1,8.25600,22.42600<br>
2,14.43200,33.19000<br>
3,19.72200,41.67400<br>
4,24.84400,49.10000<br>
5,29.33000,54.44000<br>
6,33.52400,59.05400<br>
7,37.01000,62.97200<br>
8,40.27800,66.18400<br>
…</p>
</blockquote>
<p>8 GPUs val logs:</p>
<blockquote>
<p>epoch,acc1,acc5<br>
0,0.97200,4.01400<br>
1,2.47400,8.75200<br>
…</p>
</blockquote>
<p>=========================================</p>
<p>8 GPUs train logs:</p>
<blockquote>
<p>epoch,itr,xe_loss,distill_loss,kl_loss<br>
0,0,6.97100,0.2382,4.47523<br>
0,1,6.99785,0.3105,4.11384<br>
0,2,7.02923,0.30575,4.20045<br>
0,3,7.00414,0.28821,3.98422<br>
0,4,6.99461,0.27749,4.11011<br>
0,5,7.01260,0.26803,4.05124<br>
0,6,6.99693,0.28150,4.34939<br>
0,7,6.97944,0.26662,4.38219<br>
0,8,7.00785,0.28510,4.59107<br>
…</p>
</blockquote>
<p>2 GPUs train logs:</p>
<blockquote>
<p>epoch,itr,xe_loss,distill_loss,kl_loss<br>
0,0,6.97061,0.24541,4.40004<br>
0,1,7.05148,0.30718,4.09486<br>
0,2,7.01019,0.30563,4.32012<br>
0,3,6.98066,0.28823,3.98150<br>
0,4,7.04321,0.27916,4.32263<br>
0,5,7.04827,0.26795,4.03329<br>
0,6,6.93550,0.28153,4.53965<br>
0,7,7.07279,0.27601,4.47247<br>
0,8,7.06095,0.29057,4.46109<br>
0,9,6.99917,0.27686,4.19178<br>
…</p>
</blockquote>

ptrblck: <aside class="quote no-group" data-username="Roy1" data-post="1" data-topic="187011">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/roy1/48/61241_2.png" class="avatar"> Roy1:</div>
<blockquote>
<p>From my understanding, the validation accuracy should be roughly the same in both of these settings right? I am using DDP for both and so the only difference is just the number of GPUs.</p>
</blockquote>
</aside>
<p>Not necessarily, since the batch size is different between your approaches and will influence e.g. batchnorm layers and their running stats.</p>
Issue while PyTorch installation on cuda 12.0

Asha_Bishnoi: <p>Hello PyTorch community,</p>
<p>I’m encountering an issue while trying to run PyTorch on my GPU, and I’m seeking some guidance on how to resolve it.<br>
<strong>Problem Description:</strong></p>
<p>I’m attempting to run a PyTorch script on my GPU, but I’m consistently getting the following error and warning:<br>
Warning : H100 80GB HBM13 cuda compatible with the current PyTorch installation.<br>
RuntimeError: CUDA error: no kernel image is available for execution on the device</p>
<p><strong>System Configuration:</strong></p>
<ul>
<li>OS: Redhat 8.6</li>
<li>GPU Machine: H100 80GB HBM13</li>
<li>CUDA Version: 12.0</li>
<li>PyTorch Version: 2.0.1(python package)<br>
value for print(torch.version.cuda) = 11.7</li>
<li>GPU Driver Version: Nvidia-smi 525.60.13 driver version : 525.60.13</li>
</ul>
<p>I’m running out of ideas on how to resolve this issue. If anyone has encountered a similar problem or has suggestions on what I might try next, I’d greatly appreciate your guidance.</p>
<p>Thank you in advance for your help.</p>

ptrblck: <p>Your Hopper GPU needs CUDA 11.8, so either install the latest stable release with 11.8 or the nightly binaries with CUDA 11.8 or 12.1.</p>
torch.Tensor vs. torch.FloatTensor

haj: <p>What is the relationship between <code>torch.Tensor</code> and <code>torch.FloatTensor</code>? In the Pytorch document, its says that <code>torch.Tensor</code> is an alias for<code> torch.FloatTensor</code>. I don’t quiet understand what this means.</p>
<p>For example, I created a tensor <code>x</code> using <code>torch.tensor</code> function which should return an instance of <code>torch.Tensor</code> class.</p>
<pre><code class="lang-auto">&gt;&gt;&gt; x = torch.tensor([1., 2.])
&gt;&gt;&gt; isinstance(x, torch.Tensor)
True
&gt;&gt;&gt; isinstance(x, torch.FloatTensor)
True
&gt;&gt;&gt; issubclass(torch.FloatTensor, torch.Tensor)
False
</code></pre>
<p>It turns out <code>x</code> is an instance of <code>torch.FloatTensor</code> as well as <code>torch.Tensor</code>. So, I initially thought that <code>torch.FloatTensor</code> was a subclass of <code>torch.Tensor</code>, which it turned out that it <strong>wasn’t</strong>!!</p>
<p>How is this possible that <code>x</code> is an instance of both <code>torch.FloatTensor</code> and <code>torch.Tensor</code> when these two classes are not in inheritence relationship?</p>
<p>Also, what is the relationship between <code>torch.Tensor</code> and <code>torch.FloatTensor</code> and how are they different?</p>
How to choose different msvc build tools when building with cpp_extension?

jzellinger: <p>I have used the cpp extensions to build some open source packages and for the most part everything seems to work fine. However, I have come across some compatibility issue which should be resolved with an older compiler version (I think…).<br>
Anyway I have to versions of msvc build tools installed:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/5/b54f67b32dbba217ee9398f14ea5e07cd794efb4.png" data-download-href="https://discuss.pytorch.org/uploads/default/b54f67b32dbba217ee9398f14ea5e07cd794efb4" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/5/b54f67b32dbba217ee9398f14ea5e07cd794efb4.png" alt="image" data-base62-sha1="pRWxTGQumMOPYptPHDJY0DuTdKk" width="690" height="315" data-dominant-color="2F2F2F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">724×331 7.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>but when running <code>python setup.py install</code> it always uses the latest version. Is there an argument I can pass to <code>CUDAExtension</code> or <code>BuildExtension</code> in order to set the correct version?</p>
<p>For further information, I have already posted a <a href="https://stackoverflow.com/questions/76967322/how-to-choose-msvc-build-tool-version-with-torch-utils-cpp-extension" rel="noopener nofollow ugc">stackoverflow question</a> with some more details about the package I want to build and the <code>setup.py</code> code.</p>
NN architecture and L1Loss (not decreasing), MSE (smoothing)

Karan_Tin: <p>Hi,<br>
I have pairs of images like below (inputs and labels), they are both single channel images with real-valued pixels. I want to train a CNN that maps from one to the other. I have tried UNet and variations of UNet, and it somehow works okay but I get very smoothed outputs (as you can see the target has discrete individual pixels). I have tried using MSE loss and when I switch to L1Loss the loss doesn’t decrease. Also I’ve explored other options such as (SSIM, perceptual loss etc.) but none worked.</p>
<p>Maybe I should explore different architecture.</p>
<p>My questions now are:</p>
<ul>
<li>What kind of architecture would be suitable for this “simple” 1-to-1 pixel-wise mapping problem?</li>
<li>What loss function to use?</li>
</ul>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/1/f17622ed95c9d63f29f759351aff247cb7470699.png" alt="input_epoch_1_iter_2_1" data-base62-sha1="ys4aWzL2k44NoFiSIkrrYYnNVs5" width="512" height="32"></p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/6/36ad82fd7afe6b06983e5c29db7b77c39ec969bf.png" alt="target_epoch_1_iter_2_1" data-base62-sha1="7NHwT6Cs6BKuV4hxnnCV2Ok3wT5" width="512" height="64"><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/f/1/f17622ed95c9d63f29f759351aff247cb7470699.png" alt="input_epoch_1_iter_2_1" data-base62-sha1="ys4aWzL2k44NoFiSIkrrYYnNVs5" width="512" height="32"></p>