akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Loading Custom .pth model (mobilenet) and Inference

username: S_Hamzeh74
cooked: <p>Hi <img src="https://discuss.pytorch.org/images/emoji/apple/raised_hand_with_fingers_splayed.png?v=12" title=":raised_hand_with_fingers_splayed:" class="emoji" alt=":raised_hand_with_fingers_splayed:" loading="lazy" width="20" height="20"><br>
I have Trained SSD-Mobilenet Object Detection model with this [<a href="https://github.com/qfgaohao/pytorch-ssd.git" rel="noopener nofollow ugc">pytorch-ssd</a>] repository on GitHub and get mobilenet_checkpoint.pth model file<br>
now I want to convert it to a .pt file and Inference…<br>
how can I do that?<br>
I try a lot of methods and don’t get an answer…<br>
another thing I want to convert the model to onnx<br>
when I Try to convert the .pth model to onnx with that repository it gives me an error…</p>

username: whatisslove7
cooked: <p>Hi!</p>
<p>I think this example (not for tour model, but similar) can help you</p>
<pre><code class="lang-auto">import torch
import torchvision
from unet import UNet 

model = UNet(3, 2)
model.load_state_dict(torch.load("best_weights.pth"))
model.eval()
example = torch.rand(1, 3, 320, 480)
traced_script_module = torch.jit.trace(model, example)
traced_script_module.save("model.pt")
</code></pre>
<p>Here example is the tensor that has the same dimension as your data</p>
Installing Nightly version of pytorch using suggested pip command keeps getting weird errors

username: Hakuna_Matata
cooked: <p>Hi guys,</p>
<p>Not sure if anyone has encountered errors like this before, but my Windows 11 setup keeps getting the error after using the command suggested on the installation guide page. The reason why I would like to install the preview version is I would like to try on CUDA12. I’ve got CUDA12 installed and also it’s the matching version of cudnn 8.9.2.26_cuda12.</p>
<p>Really appreciate your help. Thanks in advance.</p>
<p>Installation command <code>pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu121</code></p>
<p>Error message:</p>
<blockquote>
<p>Blockquote<br>
ERROR: Exception:<br>
Traceback (most recent call last):<br>
File “C:\Users\Edward\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_internal\cli\base_command.py”, line 169, in exc_logging_wrapper<br>
status = run_func(*args)<br>
^^^^^^^^^^^^^^^<br>
File “C:\Users\Edward\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_internal\cli\req_command.py”, line 248, in wrapper<br>
return func(self, options, args)<br>
^^^^^^^^^^^^^^^^^^^^^^^^^<br>
File “C:\Users\Edward\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_internal\commands\install.py”, line 449, in run<br>
installed = install_given_reqs(<br>
^^^^^^^^^^^^^^^^^^^<br>
File “C:\Users\Edward\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_internal\req_<em>init</em>_.py”, line 72, in install_given_reqs<br>
requirement.install(<br>
File “C:\Users\Edward\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_internal\req\req_install.py”, line 800, in install<br>
install_wheel(<br>
File “C:\Users\Edward\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_internal\operations\install\wheel.py”, line 731, in install_wheel<br>
_install_wheel(<br>
File “C:\Users\Edward\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_internal\operations\install\wheel.py”, line 591, in _install_wheel<br>
file.save()<br>
File “C:\Users\Edward\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_internal\operations\install\wheel.py”, line 390, in save<br>
shutil.copyfileobj(f, dest)<br>
File “C:\Users\Edward\AppData\Local\Programs\Python\Python311\Lib\shutil.py”, line 197, in copyfileobj<br>
buf = fsrc_read(length)<br>
^^^^^^^^^^^^^^^^^<br>
File “C:\Users\Edward\AppData\Local\Programs\Python\Python311\Lib\zipfile.py”, line 955, in read<br>
data = self._read1(n)<br>
^^^^^^^^^^^^^^<br>
File “C:\Users\Edward\AppData\Local\Programs\Python\Python311\Lib\zipfile.py”, line 1045, in _read1<br>
self._update_crc(data)<br>
File “C:\Users\Edward\AppData\Local\Programs\Python\Python311\Lib\zipfile.py”, line 973, in _update_crc<br>
raise BadZipFile(“Bad CRC-32 for file %r” % self.name)<br>
zipfile.BadZipFile: Bad CRC-32 for file ‘torch/lib/cublasLt64_12.dll’</p>
</blockquote>
Idiomatic way of using the same model on subparts of the input?

username: Yeb02
cooked: <p>I am trying to implement the following architecture, where NetB’s are the same network, and as such share the same gradient at the update.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/f/7f8aa5667fec6ed1718e633b3317978f5cc30dde.png" data-download-href="https://discuss.pytorch.org/uploads/default/7f8aa5667fec6ed1718e633b3317978f5cc30dde" title="Blank diagram"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/f/7f8aa5667fec6ed1718e633b3317978f5cc30dde.png" alt="Blank diagram" data-base62-sha1="ichGzqCSDmMXNoZdst1mT6TXD2K" width="690" height="286" data-dominant-color="E1CDC0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Blank diagram</span><span class="informations">1120×465 8.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Each Net B takes as input a subpart of the output of Net A (potentially overlapping). The final output is a concatenation of all NetB’s output.<br>
There is also a subtlety with the training process : Different learning rates are used for each item of a mini-batch, but those learning rates are only known after all items of the batch have been processed. So gradients need to be saved individually until the update. (I am training both net A and net B)</p>
<p>I am looking for the best way to implement this architecture, so that it can be ported to GPU and is efficient with gradients. The closest feature I have found is <a href="https://pytorch.org/docs/stable/generated/torch.nn.DataParallel.html#torch.nn.DataParallel" rel="noopener nofollow ugc">this</a> which is not what I am looking for.<br>
(I am using the c++ front-end)</p>
VAE: Noisy decoder output

username: Joseph_M
cooked: <p>Hello everyone!<br>
I’m trying to implement a simple VAE by following several tutorials (<a href="https://towardsdatascience.com/variational-autoencoder-demystified-with-pytorch-implementation-3a06bee395ed" rel="noopener nofollow ugc">[1]</a>,<a href="https://2" rel="noopener nofollow ugc">https://sannaperzon.medium.com/paper-summary-variational-autoencoders-with-pytorch-implementation-1b4b23b1763a</a>. This is the code that I have written:</p>
<pre><code class="lang-auto">class VAE(torch.nn.Module):
    def __init__(self,input_shape,h_dim,z_dim):
        super(VAE,self).__init__()
        #Encoder, image to hidden space
        self.img_2hid = Modules.Linear(input_shape,h_dim)
        # From the hidden we move to z space and we get mu and std
        self.fc_mu,self.fc_std = Modules.Linear(h_dim,z_dim),Modules.Linear(h_dim,z_dim)
        # The decoder from the z space goes back to the hidden
        self.z_2hidden = Modules.Linear(z_dim,h_dim)
        # Finally from the hidden we go back to the input space
        self.hidden_2img = Modules.Linear(h_dim,input_shape)
        self.relu = Modules.Relu()
        self.rloss = torch.nn.MSELoss('sum')


    def encode(self,x):
        x = self.relu(self.img_2hid(x))
        return self.fc_mu(x),torch.log(self.fc_std(x))

    def decode(self,x):
        x = self.relu(self.z_2hidden(x))
        return self.relu(self.hidden_2img(x))

    def normalize(self,x):
        v_min, v_max = x.min(), x.max()
        new_min, new_max = torch.Tensor([0]).float(), torch.Tensor([255]).float()
        x = torch.add(torch.mul(torch.div(torch.sub(x, v_min), torch.sub(v_max, v_min)), torch.sub(new_max, new_min)),new_min)
        return x

    def forward(self,x):
        original = x.shape
        # encode x to get the mu and variance parameters
        mu, log_sigma = self.encode(x.reshape(-1))
        # sample z from q
        sigma = torch.exp(log_sigma)
        epsilon = torch.randn_like(sigma)
        z = mu + sigma * epsilon
        # decoded
        x_hat = self.normalize(self.decode(z)).reshape(original)
        # reconstruction loss
        recon_loss = self.rloss(x_hat,x)
        # kl
        # The formula here is suppose for a normal distribution for q(z|x)
        kl_div = - 0.5 * torch.sum(1 + torch.log(sigma) * 2 - mu.pow(2) - torch.exp(log_sigma*2)) 
        # elbo
        elbo = recon_loss + kl_div
        return elbo,recon_loss,kl_div,x_hat
</code></pre>
<p>The problem that I’m having is that the output of the decoder is extremely noisy. For reference by training on this image:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/2/9/29c2c578111b76136cafbeaaea74846cc7ce7c64.jpeg" alt="105901" data-base62-sha1="5XqS40lB8Qpz4ztY5ZyHO9S6BnK" width="178" height="218"><br>
I get:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/c/b/cb2675e95cf1521bbfde72a753cd866c042801ef.jpeg" alt="x" data-base62-sha1="sZ9kkBcOrOgkj0XB2pztmSAUlk3" width="177" height="217"><br>
Any idea why? On the MNIST dataset the VAE works well, but as soon as I pass real images it behaves this way</p>
Is it not possible to gradient checkpoint PatchEmbed in VisionTransformer?

username: yxchng1
cooked: <p>I tried to add gradient checkpoint to vision transformer through the following code</p>
<pre><code class="lang-auto">class PatchEmbed(nn.Module):
    """
    Image to Patch Embedding.
    """

    def __init__(
        self,
        kernel_size: Tuple[int, int] = (16, 16),
        stride: Tuple[int, int] = (16, 16),
        padding: Tuple[int, int] = (0, 0),
        in_chans: int = 3,
        embed_dim: int = 768,
        use_checkpoint: bool = False
    ) -&gt; None:
        """
        Args:
            kernel_size (Tuple): kernel size of the projection layer.
            stride (Tuple): stride of the projection layer.
            padding (Tuple): padding size of the projection layer.
            in_chans (int): Number of input image channels.
            embed_dim (int): Patch embedding dimension.
        """
        super().__init__()

        self.use_checkpoint = use_checkpoint
        self.proj = nn.Conv2d(
            in_chans, embed_dim, kernel_size=kernel_size, stride=stride, padding=padding
        )

    def forward(self, x):
        if self.use_checkpoint:
            return torch.utils.checkpoint.checkpoint(self._forward, x)
        else:
            return self._forward(x)

    def _forward(self, x: torch.Tensor) -&gt; torch.Tensor:
        x = self.proj(x)
        # B C H W -&gt; B H W C
        x = x.permute(0, 2, 3, 1)
        return x
</code></pre>
<p>However, it gives the following error when use_checkpoint is True.</p>
<pre><code class="lang-auto">RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`, and by 
making sure all `forward` function outputs participate in calculating loss. 
If you already have done the above, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable).
Parameter indices which did not receive grad for rank 6: 1 2
 In addition, you can set the environment variable TORCH_DISTRIBUTED_DEBUG to either INFO or DETAIL to print out information about which particular parameters did not receive gradient on this rank as part of this error
</code></pre>
<p>I have already checkpointed the rest of the transformer blocks but still need a bit extra memory and thought maybe I could get that by checkpointing the PatchEmbed too. However, it gives error when I try to do so.</p>
Is BCEWithLogitsLoss obtained as weighted BCE loss?

username: M_J
cooked: <p>hi everyone<br>
can we obtain the BCEWithLogitsLoss as weighting<br>
or it just BCE loss function combined with a sigmoid activation function?</p>

username: KFrank
cooked: <p>Hi M!</p>
<aside class="quote no-group" data-username="M_J" data-post="1" data-topic="181237" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/m_j/48/58959_2.png" class="avatar"> M_J:</div>
<blockquote>
<p>can we obtain the BCEWithLogitsLoss as weighting</p>
</blockquote>
</aside>
<p>No.  They take different inputs – logits vs. probabilities – and you can’t<br>
compensate for that difference with weighting.</p>
<aside class="quote no-group">
<blockquote>
<p>or it just BCE loss function combined with a sigmoid activation function?</p>
</blockquote>
</aside>
<p>Mathematically yes.  In its numerical implementation, <code>BCEWithLogitsLoss</code> uses<br>
the (mathematically-equivalent) <code>logsigmoid()</code> instead of <code>log (sigmoid())</code> for<br>
reasons of numerical stability.</p>
<p>Best.</p>
<p>K. Frank</p>
Dataloader datalength not fitting for last batch

username: Neurox
cooked: <p>Hey, I’m new to ML. I have a small dataset and I want to make the most of my RTX 3080. Therefore, I’m trying to use a large batch size of 64. When I used a batch size of 8, it took 97 minutes, but with a batch size of 64, it only took 65 minutes. However, I’m concerned about potential data loss.</p>
<p>The following code prints the number of required images (for complete batch usage) and the actual number of available images:</p>
<pre><code class="lang-auto">print(len(train_loader) * 64, train_data_len)
print(len(validation_loader) * 64, val_data_len)
print(len(test_loader) * 64, test_data_len)
</code></pre>
<p>Output:</p>
<pre><code class="lang-auto">7616 7563
896 841
960 924
</code></pre>
<p>With a batch size of 64, I have 118 iterations to process the entire dataset, which means I need 7616 images. However, I only have 7563 images available for training. What happens to the last batch with just 7616 - 7563 = 53 images?</p>
<p>Will it be ignored or filled with random data?</p>

username: srishti-git1110
cooked: <p>The last batch isn’t dropped unless you set <code>drop_last=True</code> while instantiating the DataLoader.</p>

username: Neurox
cooked: <p>If I may ask one more small question, is there a drawback to using a relatively large batch size to speed up training with such a small dataset, assuming that no data is lost as I initially thought?</p>

username: srishti-git1110
cooked: <p>It shouldn’t be.<br>
Smaller batch sizes <em>might</em> help with regularization and prevent overfitting, but this is just a technique that you could try if you aren’t able to regularize using other techniques.</p>
_module 'torch.nn' has no attribute '_VF'

username: meghdad
cooked: <p>Hello<br>
I want to use the downloaded code which uses pytorch 1.01.<br>
hx, cx = nn._VF.lstm_cell(<br>
embedding<br>
prev_hidden,<br>
params[“lstm.weight_ih”],<br>
params[“lstm.weight_hh”],<br>
params[“lstm.bias_ih”],<br>
params[“lstm.bias_hh”],<br>
)<br>
I installed torch version 1.13.0+cu117 in the new environment, but when I run the above code, I get the following error:</p>
<p>AttributeError: module ‘torch.nn’ has no attribute ‘_VF’</p>
<p>Please advise on how to fix the error or what are the last four input parameters in lstm_cell and how can I use this code in PyTorch 2 if there are 2 input parameters in PyTorch 2 version.<br>
Thank you for your guidance</p>
Help Training RetinaNet

username: Dicko
cooked: <p>Hello folks, I’m looking for some help in training a custom RetinaNet model with a Resnet-101 backbone.<br>
I have my dataset in coco format, I’ve tried a few implementations such as keras and others on GitHub but I’m either not seeing the model learn, and the model training validation losses are not being saved.<br>
If anyone knows where I can look to follow an implementation that works, if you could let me know that will be great and thanks in advance. I want to save the model after each epoch, I want to track the mAP, I want to save the training and validation losses so I can plot them. Thanks folks.</p>
Compiling Pytorch Models to run on Apple ANE

username: Jonathan_Fuchs
cooked: <p>I benchmarked 2 different Resnet50 Models - the Apple CoreML model, available on the Apple website, and a pretrained Torchvision Resnet50 model which I converted using ONNX (Opset9) and CoreMLTools (iOS Version 13).</p>
<p>I tested both models on a brand new iPhone XR.</p>
<p>Inference Times:</p>
<ol>
<li>
<strong>Apple Resnet50</strong> : CPU Inference 100ms, GPU Inference 60ms, ANE Inference 15ms</li>
<li>
<strong>Torchvision Resnet50</strong> : CPU Inference 100ms, GPU Inference 60ms, ANE Inference 60ms.</li>
</ol>
<p>As you can see, the ANE Inference for the Apple version is 4x faster.</p>
<p>Memory Usage:</p>
<ol>
<li>
<strong>Apple Resnet50</strong> : 46 MB CPU, 92 MB GPU, 35 MB ANE</li>
<li>
<strong>Torchvision Resnet50</strong> : 48 MB CPU, 91 MB GPU, 91 MB ANE</li>
</ol>
<p>The Apple version uses 3x less memory than the Torchvision one.</p>
<p>I am concluding the Torchvision Model doesn’t run on the ANE.</p>
<p>I was wondering if anyone knew how the Apple Repo models were compiled in order to run on the ANE? From the model description, the Apple Models were originally written in Keras, but as far as I know the Keras implementation isn’t different from the Pytorch Torchvision one, so I am not sure where the difference comes from.</p>
<p>Thanks!</p>

username: a15315888737
cooked: <p>Hello! First, I tested the results on my ANE-GPT model, considering that the ANE engine will prioritize 4-dimensional tensors, so you need to make appropriate changes to the neural network architecture. <img src="https://discuss.pytorch.org/images/emoji/apple/grinning.png?v=12" title=":grinning:" class="emoji" alt=":grinning:" loading="lazy" width="20" height="20"></p>

username: Elias_Vansteenkiste
cooked: <aside class="quote no-group" data-username="a15315888737" data-post="2" data-topic="80546">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/a15315888737/48/50531_2.png" class="avatar"> a15315888737:</div>
<blockquote>
<p>ANE-GPT</p>
</blockquote>
</aside>
<p>Could you clarify what you exactly mean with prioritization of 4D tensors?  Should we better use batches for testing the speed or just add an extra dimension, which already happens by default<br>
4D → , &lt;channels/feature maps&gt;, , , e.g. (1, 3, 224, 224)</p>
Vq-diffusion for face generation

username: Jdazdet
cooked: <p>Hello everyone,</p>
<p>this is the first time i using this forum, so sorry if i make some mistake.</p>
<p>I’m working on VQ-Diffusion i will appreciate some suggestion. The task is: use diffusion model to generate new faces. The input images have size (3x64x64), so i want to encode them to work on the latent space. My questions are: which are the best size and the number of feature maps i should give in input at the Unet used for the backward pass in the diffusion process? Is it useful to work on the latent space?</p>
<p>Thanks in advance.</p>
ONNX conversion error

username: sardo97
cooked: <p>I am learning to convert models to the onnx. I wrote a simple classification model to classify cats and dogs but the onnx conversion is giving shape unmatch error.</p>
<p><a class="hashtag-cooked" href="/c/vision/5" data-type="category" data-slug="vision" data-id="5"><span class="hashtag-icon-placeholder"></span><span>vision</span></a><br>
class Cnn(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Cnn,self).<strong>init</strong>()</p>
<pre><code>    self.layer1 = nn.Sequential(
        nn.Conv2d(3,16,kernel_size=3, padding=0,stride=2),
        nn.BatchNorm2d(16),
        nn.ReLU(),
        nn.MaxPool2d(2)
    )
    
    self.layer2 = nn.Sequential(
        nn.Conv2d(16,32, kernel_size=3, padding=0, stride=2),
        nn.BatchNorm2d(32),
        nn.ReLU(),
        nn.MaxPool2d(2)
        )
    
    self.layer3 = nn.Sequential(
        nn.Conv2d(32,64, kernel_size=3, padding=0, stride=2),
        nn.BatchNorm2d(64),
        nn.ReLU(),
        nn.MaxPool2d(2)
    )
    
    self.fc1 = nn.Linear(3*3*64,10)
    self.fc2 = nn.Linear(10,2)
    self.relu = nn.ReLU()
    
def forward(self,x):
    out = self.layer1(x)
    out = self.layer2(out)
    out = self.layer3(out)
    out = out.view(out.size(0),-1)
    out = self.relu(self.fc1(out))
    out = self.fc2(out)
    return out
</code></pre>
<p>model = Cnn()<br>
model.load_state_dict(torch.load(‘cat_dog.pth’))</p>
<p>dummy_input = torch.randn(1, 3, 224, 224)  # Example input with shape (batch_size, channels, height, width)</p>
<p>torch_out = model(dummy_input)</p>
<p>input_names = [“input”]<br>
output_names = [“output”]<br>
onnx_path = “cnn_model.onnx”  # Path to save the ONNX model file</p>
<h1>
<a name="export-the-model-to-onnx-1" class="anchor" href="#export-the-model-to-onnx-1"></a>Export the model to ONNX</h1>
<p>torch.onnx.export(<br>
model,<br>
dummy_input,<br>
onnx_path,<br>
input_names=input_names,<br>
output_names=output_names,<br>
opset_version=10,  # Use ONNX opset version 10<br>
do_constant_folding=True,<br>
dynamic_axes={<br>
“input”: {0: “batch_size”},<br>
“output”: {0: “batch_size”}<br>
}<br>
)</p>
<h1>
<a name="error-2" class="anchor" href="#error-2"></a>ERROR</h1>
<p>The error says as follows: RuntimeError: Given groups=1, weight of size [16, 3, 3, 3], expected input[1, 1, 224, 224] to have 3 channels, but got 1 channels instead</p>

username: sardo97
cooked: <p>Solved need to restart my computer</p>
PyTorch multi-processing while syncing the model weight

username: Dazitu616
cooked: <p>I am training an RL model, which consists of two steps:</p>
<ul>
<li>Collecting data by letting the RL agent interact with the simulation environment</li>
<li>Training the RL agent (policy network) on the collected data</li>
</ul>
<p>During the entire training process, we will do the above 2 steps iteratively for multiple rounds. Now the issue is, the data collection part involves some post-processing (using a big pre-trained segmentor to generate object masks on the collected images), which is very slow.</p>
<p><strong>Currently, I’m thinking of having 1 GPU training the RL agent (GPU1), and another 1 GPU dedicated to the data collection + segmentation part (GPU2)</strong>. The only communication between the 2 GPUs is syncing the model weight, because we always want to collect data using an up-to-date model weight (i.e. update the GPU2 weight every 100 training steps on GPU1). We don’t need to send the collected data to GPU1 because we plan to write them to disk, and the dataloader on GPU1 should be able to read them from disk (i.e. reading all the files presented in a folder, regardless of when it’s generated).</p>
<p>At first, I was thinking of using PyTorch DDP, as it handles all the low-level communication stuff and can sync model weight across GPUs. However, later on, I realized the sync is achieved by syncing the gradients during the backward pass. In my case, I only compute loss/gradient on GPU1, while GPU2 is doing different operations, which will lead to a deadlock.</p>
<p>Now I think probably I should look into the multi-processing code to somehow manually sync the model weight. Are there any resources I should look at (I’m looking at <a href="https://pytorch.org/docs/stable/notes/multiprocessing.html" rel="noopener nofollow ugc">torch.multiprocessing</a>). Since the only thing I want to sync here is the model weight, I guess it won’t be very complex.<br>
(Well, worst case I can also save the model weight to disk, and load it on GPU2 whenever there is a new weight. Then there is no communication between GPUs)</p>

username: Dazitu616
cooked: <p>OK we finally decide to use <code>torch.multiprocessing</code> to send sync the model weight. Turns to be quite fast</p>
Mutex in processes spawned by torchrun?

username: ringohoffman
cooked: <p>I want to make a logging buffer in shared memory so I can batch my writes to the filesystem, since it occurs as a network call when I run my setup in kubernetes.</p>
<p>I’m trying to use <code>multiprocessing.Lock</code> and <code>multiprocessing.Semaphore</code>, but when I create it in the local rank 0 process and broadcast it via <code>torch.distributed.broadcast_object_list</code>, I get a <code>multiprocessing</code> <code>RuntimeError</code> that mutexes must be created on the parent process. Creating a mutexes from a <code>multiprocessing.Manager</code> doesn’t work either as it fails some kind of authorization check.</p>
<p>How can I create mutexes for a shared resource using what is available in <code>torch.distributed</code>?</p>
Anti Money Laundering and Fraud Detection using Pytorch

username: Ashish_Verma
cooked: <p>Hi Team, I have praposed a solution to implement anit money laundering and fraud detection to my company and my design is moved to phase-2, I wanted to implemment same in pytorch, can help on this to design complete life cycle of pYtorch application and perform a POC. Any references, idea will be highly appretiated!!</p>

username: Albert_Dexter
cooked: <p>wow and very nice i like</p>
Getting the gradient of the gradient

username: zanellar
cooked: <p>I have a function f(x) which includes an approximation with Taylor expansion, e.g. f(x,a)=g(a)+g’(a)(x-a) where g’(x)=dg(x)/dx is computed with torch.autograd. Now I would like to differentiate it in “a”, i.e. df(x,a)/da. To simplify even more the function let’s say that f(a)=g’(a):</p>
<blockquote>
<p>def g(x):<br>
return x</p>
<p>def f(a):<br>
return  torch.autograd.functional.jacobian(g, a)</p>
</blockquote>
<p>If I visualize the computational graph, it seems that I lose the gradient when computing the Jacobian.<br>
In particular, using these lines I get an empty graph:</p>
<blockquote>
<p>a = torch.ones(1, requires_grad=True)<br>
torchviz.make_dot(f(a), params=dict(a=a))</p>
</blockquote>
<p>Now, my question is: how can I implement this computation?</p>

username: KFrank
cooked: <p>Hi Riccardo!</p>
<aside class="quote no-group" data-username="zanellar" data-post="1" data-topic="182018" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zanellar/48/61249_2.png" class="avatar"> zanellar:</div>
<blockquote>
<p>f(x,a)=g(a)+g’(a)(x-a) where g’(x)=dg(x)/dx is computed with torch.autograd. Now I would like to differentiate it in “a”, i.e. df(x,a)/da.</p>
</blockquote>
</aside>
<p>If I understand your use case, I would use <code>autograd.grad()</code> twice, with<br>
<code>create_graph = True</code> for the first call so that autograd will be able to<br>
differentiate the derivative in the second call.</p>
<p>Here is an example script where <code>g</code> is assumed to be a scalar function:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
print (torch.__version__)

def g (x):
    return  x**3

def f (x, a):
    ga = g (a)
    gpa = torch.autograd.grad (ga, a, create_graph = True)[0]
    return  ga + gpa * (x - a)

x = torch.tensor ([2.1])
a = torch.tensor ([2.0], requires_grad = True)
fxa = f (x, a)
print ('fxa =', fxa)

fpa = torch.autograd.grad (fxa, a)[0]
print ('fpa =', fpa)
</code></pre>
<p>And here is its output:</p>
<pre><code class="lang-plaintext">2.0.0
fxa = tensor([9.2000], grad_fn=&lt;AddBackward0&gt;)
fpa = tensor([1.2000])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

username: zanellar
cooked: <p>Thank you very much!!</p>
Pytorch runtimeError:cublas runtime error : the GPU program failed to execute at /pytorch/aten/src/THC/THCBlas.cu:331

username: bibo
cooked: <p>I run my program in pytorch environment, and I got a message like this:“RuntimeError: cublas runtime error : the GPU program failed to execute at /pytorch/aten/src/THC/THCBlas.cu:331”<br>
How can I find a solution for this problem? Thanks for any help!</p>

username: ptrblck
cooked: <p>Could you post a minimal and executable code snippet reproducing the error as well as the output of <code>python -m torch.utils.collect_env</code>, please?</p>
Will DataLoader pick always the same samples if a seed is used?

username: bornexmachina
cooked: <p>My code follows this structure</p>
<pre><code class="lang-auto">
def train_epoch(epoch):
  # assume it loads 1000 samples
  # using 10 samples in batch - so it will have 100 batches
  train_loader_long = self.DataLoader(...)
  
  # assume it loads 100 samples
  # using 10 samples in batch it will have 10 batches
  train_loader_short = self.DataLoader(...something else..)
  
  # this should be 10
  min_len = min(len(train_loader_short), len(train_loader_long))
  
  for i in range(min_len):
    batch_1 = next(iter(train_loader_long))
    batch_2 = next(iter(train_loader_short))
...
</code></pre>
<p>After 10 batches I will see all sample of loader_short, but only 10% of the loader_long. My question is:</p>
<p>When I call the <code>train_epoch()</code> function for 100 epochs, will it take 100 times SAME samples for the loader_long or will it randomly pick some samples each time. In the latter case, I would (in probability) see all samples if I use sufficiently many epochs.</p>

username: ptrblck
cooked: <p>If you do not set the seed again inside the <code>train_epoch</code> method and use <code>shuffle=True</code> then random samples should be returned.</p>
Use of Field, RawField and Example in torchtext 0.15

username: neenaa
cooked: <p>Im trying to update the code in a github rep which uses Pytorch 1.4.0 and the compatible torchtext version, to latest Pytorch and torchtext version 0.15. Im not sure how to use torchtext.data.Field, torchtext.data.RawField and torchtext.data.Example in the newer version. torchtext.data in 0.15 doesnt have Field,RawField or Example. please help.</p>
Batch size in RNNs

username: Nemfor
cooked: <p>Hi!<br>
Can anyone please explain how RNNs process batches of different sizes?<br>
For example<br>
First batch was 128 samples, hidden vector was 128*DIM size<br>
Second batch was 64 samples. hot to process in with previous hidden size of 128?<br>
How to process addition of different batch sizes inside?</p>

username: ptrblck
cooked: <p>The common approach would be to pad the sequence length to the max length or to create batches with samples using the same sequence length. PyTorch has an experimental <code>NestedTensor</code> support, but I don’t think RNNs support it.</p>

username: Nemfor
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Hi! Probably i was not so clear.<br>
I understand how to process batches of different lenght, I mean how LSTM under the hood process batches of different batch sizes? Because size of hidden vector depends on Batche size.<br>
If we processed tensor with BS=128, hidden vector would be 128 x Num_dimentions, but how to process   if the next tensor has shape 64 x Num_dimentions?<br>
Each gate has addition operator, how we can add to tensors of different batch size?</p>

username: J_Johnson
cooked: <p>All the LSTM layer cares about, from a functional standpoint, is that the batch size of the inputs, the hidden_state and the cell state match. If they don’t, you’ll get an error.</p>

username: Nemfor
cooked: <p>Finnally Understood, that hidden_state resets to zeros for every batch)</p>
Autograd Error on carrying out two times backpropagation

username: Sumesh_Sankar
cooked: <p>I got this error on optimizing a model, how the computational graph can be modified without getting this error?<br>
one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [500, 1]], which is output 0 of AsStridedBackward0, is at version 1001; expected version 1000 instead.</p>
Why the cuda 0 is aways used in my case?

username: JohnHerry
cooked: <p>I am training a simple classifier with BERT pretrained, the example code is very simple</p>
<pre><code class="lang-auto"># ------ Define Model
MyModel(nn.Module):
    def __init__(self, output_size):
         self.bert = BertModel.from_pretrained("...",  output_attention=False, ouput_hidden_states=False)
         self.linear = nn.Linear(bert.config.hidden_size, output_size)

    def forward(self,  tokenized_input):
         return self.linear(self.bert(tokenized_input).last_hidden_state)

model = MyModel().to(device)
#------
for tokenized_inputs,  tokenized_lables in data_loader:
      tokenized_inputs = tokenized_inputs.to(device)
      tokenized_labels = tokenized_labels.to(device)
      pred = model(tokenized_inputs)
      loss = loss_function(pred, tokenized_lables)
</code></pre>
<p>When I run the trainer, I had specified the device by script parameter, say ,If I specify gpu_id=1, then inside the script, device = get_device(“cuda:1”) will be used.</p>
<p>But I found that ,  not only my CUDA:1 card is used, and the CUDA:0 card is also taken by other process.  I had iterate all parameters in MyModel instance, all the weights are in CUDA:1,  and all my data batches are in CUDA:1,   So why there is the another card used?<br>
and I found that no matter which card I had specified with script argument,  there is always a second process taking the CUDA:0;</p>
<p>Can any body explain why and give me suggestion to let my training take only one process and only taken the card I had specified  【except using the CUDA_VISIBLE_DEVICES】?</p>
<pre><code>  ...
</code></pre>
Build from source for older cuda version

username: wasifbaloch
cooked: <p>simple question: I need to use pytorch’s latest stable release. Problem is I have CUDA 11.0 so if I build pytorch from source will it work for this CUDA version. And also is build from source possible without root access?</p>

username: ptrblck
cooked: <p>You should be able to build with CUDA 11.0, but note that it’s not tested anymore (I believe the oldest CUDA release used in CI is 11.4). Note however that you might reintroduce already fixed issues in the CUDA stack.<br>
What’s the reason you want to use such an old CUDA release instead of e.g. just installing the binaries with a newer CUDA?</p>

username: malfet
cooked: <p>CUDA-11.x is forward compatible, i.e. you don’t need to install new driver to be able to use PyTorch binaries build with CUDA-11.7 or so.</p>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a> also, oldest CUDA still tested in CI is 11.4 <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

username: wasifbaloch
cooked: <p>Resaon is I use shared cluster in Lab and I could not convience them to update CUDA or our GPUs suppert this version</p>

username: wasifbaloch
cooked: <p>but when I install newer pytorch with conda it install cpu-only pytorch</p>

username: ptrblck
cooked: <p>You don’t need to install a local CUDA toolkit as it will only be used if you build PyTorch from source or a custom CUDA extension. The PyTorch binaries ship with all CUDA requirements and you only need to properly install an NVIDIA driver as <a class="mention" href="/u/malfet">@malfet</a> mentioned.<br>
So what’s the reason you want to use such an old CUDA version?</p>

username: wasifbaloch
cooked: <p>If I install pytorch normaly it is cpu-only version</p>

username: ptrblck
cooked: <p>Could you post the command you are using as well as the output from your terminal, please?</p>

username: wasifbaloch
cooked: <pre><code class="lang-auto">conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/b/fb29007217f464828956cd624992da142264bd13.png" data-download-href="https://discuss.pytorch.org/uploads/default/fb29007217f464828956cd624992da142264bd13" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/b/fb29007217f464828956cd624992da142264bd13.png" alt="image" data-base62-sha1="zPRGPQBzaaPipWO9f4KnpkCeMWn" width="690" height="119" data-dominant-color="080808"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">732×127 2.79 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Type Error in Python Code

username: Prittylox
cooked: <p>Hi! I keep running into this type error code:</p>
<p>sns.lmplot(x=‘BodyweightKg’,<br>
y=‘BestDeadliftKg’,<br>
data=PL.dropna(),<br>
hue=‘Equipment’,<br>
markers=‘x’,<br>
size=7,<br>
aspect=2)<br>
plt.title(‘Best Deadlift by Equipment Used’,fontsize=20)<br>
plt.xticks(fontsize=15)<br>
plt.yticks(fontsize=15)<br>
plt.ylabel(‘Best3DeadliftKg’,fontsize=15)<br>
plt.xlabel(‘Body3weightKg’,fontsize=15)<br>
plt.show()<br>
print(‘Equipment Used by Lifters:\n’)<br>
print(df[‘Equipment’].dropna().value_counts())</p>
<p>This is the output:</p>
<hr>
<p>TypeError                                 Traceback (most recent call last)<br>
Input In [17], in &lt;cell line: 1&gt;()<br>
----&gt;sns.lmplot(x=‘BodyweightKg’,<br>
y=‘BestDeadliftKg’,<br>
data=PL.dropna(),<br>
hue=‘Equipment’,<br>
markers=‘x’,<br>
size=7,<br>
aspect=2)<br>
plt.title(‘Best Deadlift by Equipment Used’,fontsize=20)<br>
9plt.xticks(fontsize=15)</p>
<p>TypeError: lmplot() got an unexpected keyword argument ‘size’</p>
<p>List item</p>

username: ptrblck
cooked: <p>This error doesn’t seem to be related to PyTorch but <code>seaborn</code>.<br>
Based on the <a href="https://seaborn.pydata.org/generated/seaborn.lmplot.html">docs</a> the <code>size</code> argument is indeed invalid and you would need to remove it.</p>

username: Prittylox
cooked: <p>Thank you so much!!!</p>
Doc for functions at FUNCTIONS.H

username: Jinchi_Zhang
cooked: <p>Hi, torch. Recently I am using APIs at FUNCTIONS.H. <a href="https://pytorch.org/cppdocs/api/file_build_aten_src_ATen_Functions.h.html#definition-build-aten-src-aten-functions-h" class="inline-onebox" rel="noopener nofollow ugc">File Functions.h — PyTorch main documentation</a> in python to optimize my ml code, such as torch._foreach_add() to do some tensor list operation.<br>
I am wondering is there a documentation to illustrate these APIs and Will they be maintained as torch keeps updating, because I want to use them in production and want to make sure their stability. Thanks</p>

username: ptrblck
cooked: <p>The preceding underscore indicates that the method is internal and their API can change, so keep this in mind if you want to use it in your production code.</p>
Intro to PyTorch - Quickstart Runtime Error

username: coder_cat
cooked: <p>I’m working through the Introduction to PyTorch available on your website and I copy and pasted all the code for the quickstart to download the training and test data. With the code from this section I keep getting the following error:</p>
<p>File “/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torchvision/datasets/mnist.py”, line 99, in <strong>init</strong><br>
self.download()<br>
File “/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torchvision/datasets/mnist.py”, line 195, in download<br>
raise RuntimeError(f"Error downloading {filename}")<br>
RuntimeError: Error downloading train-images-idx3-ubyte.gz</p>
<p>I’m not quite sure what the issue is and why they’re not downloading. I tried some trouble shooting but nothing seemed to work. Thanks for your help!</p>

username: ptrblck
cooked: <p>Try to delete temporary files in the specified root folder (if there are any) and try to download it again.</p>
How to create a very basic example of Transformers' per-pixel classification?

username: tldrafael
cooked: <p>Despite having tons of Transformer material on the internet, I can’t find just a barely plain pytorch implementation of a single Transformer layer that receives and outputs an image. I’m struggling to figure out how it could be easily done.</p>
<p>Not regarding performance matters, I’d just like to figure out how to apply a simple operation with Transformers using pytorch code. For example, checking a simple CNN below, it receives a tensor of (B,3,224,224), and it outcomes a tensor of (B,13,224,224). My doubt is how could I do a similar operation with Transformers?</p>
<pre><code class="lang-auto">from torch import nn

class PlainCNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.cnn1 = nn.Conv2d(3, 100, 3, padding='same')
        self.cnn2 = nn.Conv2d(100, 100, 3, padding='same')
        self.cnn3 = nn.Conv2d(100, 13, 3, padding='same')
        self.relu = nn.ReLU()

    def forward(self, x):
        x = self.cnn1(x)
        x = self.relu(x)
        x = self.cnn2(x)
        x = self.relu(x)
        x = self.cnn3(x)
        return x
        

cnn = PlainCNN()
cnn(torch.randn(1,3,224,224)).shape
# torch.Size([1, 13, 224, 224])
</code></pre>
Do pretrained torch.hub models automatically resize images?

username: chrisuz
cooked: <p>Hello, I have what might be a basic question that I can’t seem to find the answer to. Do torch.hub models automatically resize images during inference? Running some tests on model resolutions and was curious about this. Thanks!</p>

username: ptrblck
cooked: <p>It might depend on the used model but I assume (some) CNNs allow you to use variable input shapes as they would use adaptive pooling layers before the first linear layer.</p>
Vae can overfit on one data sample, but outputs blank/noise if trained with more data

username: Pillorygem
cooked: <p>Hello,<br>
I’d really appreciate some insight on this, since I’ve been struggling for a few days now.</p>
<p><strong>Here is my model’s architecture:</strong></p>
<pre><code class="lang-auto">latent_size = 5
class VAE(nn.Module):
    def __init__(self):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Conv1d(1, 16, 3, stride=3),
            nn.ReLU(),
            nn.Conv1d(16, 32, 3, stride=3),
            nn.ReLU(),
            nn.Conv1d(32, 1, 3),
            nn.ReLU(),

        )
        self.mean = nn.Linear(6942, latent_size)
        self.logvar = nn.Linear(6942, latent_size)

        self.decoder = nn.Sequential(
            nn.Linear(latent_size, 6942),
            nn.ReLU(),
            nn.ConvTranspose1d(1, 64, 3, stride=3),
            nn.ReLU(),
            nn.ConvTranspose1d(64, 32, 3, stride=3),
            nn.ReLU(),
            nn.ConvTranspose1d(32, 1, 23),
        )

    def reparameterize(self, mu, logvar):
        std = torch.exp(0.5*logvar)
        eps = torch.randn_like(std)
        return mu + (eps * std)
    
    def forward(self, x):
        encode = self.encoder(x)
        encode = encode.view(1, 6942)
        mu = self.mean(encode)
        logvar = self.logvar(encode)
        z = self.reparameterize(mu, logvar)
        decode = self.decoder(z)
        return decode, mu, logvar
    
   
net = VAE()

</code></pre>
<p><strong>And here is the loss function:</strong></p>
<pre><code class="lang-auto">def VAE_loss(x, x_hat, mu, logvar):
    reconstructed_loss = F.cross_entropy(x_hat, x, reduction='sum')
    kl_divergence = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())
    return reconstructed_loss + kl_divergence

loss = VAE_loss
</code></pre>
<p>Sincere thanks to anyone who can point me in the right direction. <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=12" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:" loading="lazy" width="20" height="20"></p>
Discrepancy between NVIDIA-smi and torch.cuda.memory_allocated()

username: Surya_Narayanan
cooked: <p>NVIDIA-SMI prints 37 GB allocated</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/8/e8110cc3e34f8c8f762cc1d4d83d42afa4456bff.png" data-download-href="https://discuss.pytorch.org/uploads/default/e8110cc3e34f8c8f762cc1d4d83d42afa4456bff" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/8/e8110cc3e34f8c8f762cc1d4d83d42afa4456bff_2_690x433.png" alt="image" data-base62-sha1="x6XiK4MD2VOyJzFpV04eT3gF37F" width="690" height="433" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/8/e8110cc3e34f8c8f762cc1d4d83d42afa4456bff_2_690x433.png, https://discuss.pytorch.org/uploads/default/original/3X/e/8/e8110cc3e34f8c8f762cc1d4d83d42afa4456bff.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/e/8/e8110cc3e34f8c8f762cc1d4d83d42afa4456bff.png 2x" data-dominant-color="191919"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">974×612 99.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>but <code>torch.cuda.memory_allocated()</code> prints 15GB allocated- im curious, why is there a discrepancy?</p>

username: marksaroufim
cooked: <p>From the PyTorch docs <a href="https://pytorch.org/docs/stable/generated/torch.cuda.memory_allocated.html" class="inline-onebox">torch.cuda.memory_allocated — PyTorch 2.1 documentation</a></p>
<blockquote>
<p>This is likely less than the amount shown in nvidia-smi since some unused memory can be held by the caching allocator and some context needs to be created on GPU. See <a href="https://pytorch.org/docs/stable/notes/cuda.html#cuda-memory-management">Memory management</a> for more details about GPU memory management.</p>
</blockquote>
<p>But TL;DR is <code>torch.cuda.memory_allocated()</code> is the amount of memory allocated to tensors whereas there’s also the CUDA context and cuda caching allocator which are being shown in <code>nvidia-smi</code></p>
My code doesn't work after update to Pytorch 2.0

username: QFuser
cooked: <p>After a while I came back to working with Pytoch I updated<br>
I use VS 2022 for it everything worked fine but I got now an error</p>
<pre><code class="lang-auto">from torchmetrics import R2Score

def train(model, optimizer, x1,x2, y):
    
    # Reset gradient
    optimizer.zero_grad()
  
    # Forward
    fx = model.forward(x1,x2)
    #loss = model.N1L(fx, y)
    loss = 0
    r2 = 0
    count = 0
    count2 = 0
    for i,result in enumerate(fx):
        loss += model.r2_dd_loss(result, y[i], 3)
        count += 1
    #loss= model.r3_loss(fx,y)
    #print(count)
    loss /= count
    for i,result in enumerate(fx):
        r2 += model.r2(result,y[i])
        count2 += 1
    # Backward
    r2 /= count2
    loss.backward()
    
    # Update parameters
    optimizer.step()

</code></pre>
<p>basically the model.r2(result,y[i]) which is the R2 function gives the error: R2Score’ object has no attribute ‘_backward_pre_hooks’. I actually don’t need any backwards tracking because its just collecting data so not sure what going on here</p>

username: ptrblck
cooked: <p>Could you add the missing code pieces to create a minimal and executable code snippet reproducing the issue, please?</p>

username: QFuser
cooked: <p>Unfortunately the project is a bit too big to break it easy down . I tested a few things when R2 from torchmetrics is part of the backtracking / losing function everything works great when not doesn’t matter where I put it it always cause errors. I solved it by using" from torcheval.metrics.functional import r2_score" instead. Also the  torchmetrics library seems not have been updated in a while</p>

username: marksaroufim
cooked: <p>FWIW torchmetrics and torcheval are 2 different but similar projects</p>