akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Memory usage drastically increases when torch.cuda.is_available() is present in the code

juleNissen: <p>My code was consuming a very high amount of RAM, and i eventually narrowed the problem down to this:</p>
<pre><code class="lang-auto">import torch
import psutil

torch.cuda.is_available()

torch.set_default_device('cuda')  # This bug only happens on GPU

# Mem: 318.99 MB either way
print(f"Mem: {psutil.Process().memory_info().rss / 1024 / 1024:.2f} MB")  

# Changing the dtype or size doesn't measurably change high memory usage 
t = torch.rand(10_000_000, dtype=torch.float32)  

# Mem: 1461.05 MB, but 411.43 MB when torch.cuda.is_available() is absent
print(f"Mem: {psutil.Process().memory_info().rss / 1024 / 1024:.2f} MB")
</code></pre>
<p>Is this a bug in PyTorch?</p>

ptrblck: <p>This is expected behavior as libraries are loaded on the host. The memory footprint was reduced in CUDA 11.8+ via CUDA’s lazy module loading, which is enabled by default on PyTorch.</p>
Referring to torch upgradation and downgradation of versions

sudip_jena: <p>Can i try installing pytorch version 1.9.0 , if yes then how do i install because now what it says is the versioning starts from 1.11.0 , so the reason i want to install 1.9.0 version is because there’s a bug in my code and that bug has been fixed in 1.9.0 version. Please help me with relevant answers , it would be of great help.</p>

ptrblck: <aside class="quote no-group" data-username="sudip_jena" data-post="1" data-topic="195175">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sudip_jena/48/66859_2.png" class="avatar"> sudip_jena:</div>
<blockquote>
<p>so the reason i want to install 1.9.0 version is because there’s a bug in my code and that bug has been fixed in 1.9.0 version</p>
</blockquote>
</aside>
<p>If the bug was fixed in <code>1.9.0</code> all future versions should also contain it unless it was accidentally re-introduced.<br>
You can take a look at <a href="https://pytorch.org/get-started/previous-versions/#v190">these instructions</a> to install old releases if it’s really needed.</p>
Adjusting weights of autoencoders based on known output per iteration

Salman_Abbasi: <p>Hi</p>
<p>I am trying to mimic a mathematical iterative optimization using Autoencoders. I want to learn this network in same manner, just like this optimization does. I would like to take output of each iteration of this optimization using pytorch, and then adjust the weights of the autoencoder network, so the output for this network iteration is same as the optimization.</p>
<p>Any suggestions?</p>
<p>Thanks<br>
Sal</p>
Multi GPU on single GPU, Is there a way to utilize the remaining GPU RAM to run multi-GPU on a single algorithm

Toan_Nguyen_Duc1: <p>If algorithm A uses 15GB out of 40GB, can I allocate the remaining 25GB to run multi-GPU, extracting 15GB from this 25GB pool?<br>
I am running Latent Diffusion; however, my data only works for a batch size smaller than 8, and I’m only using around 11GB out of 40GB. This makes me feel quite wasteful because I’m using Colab Pro+, and the cost is calculated per hour rather than per GB of RAM per hour.</p>

ptrblck: <p>I’m not familiar enough with Colab and don’t know if and how you can launch multiple processes on the same instance and GPU, but generally it’s possible to run multiple processes on the same GPU.</p>

Toan_Nguyen_Duc1: <p>My intention is not to use the remaining amount of RAM to run multi-processes but rather to utilize this remaining portion to further accelerate the running algorithm.<br>
For example: <code>!python main.py --base ./configs/i2i.yaml -t --gpus 0,</code> (using only 12GB out of 40GB).<br>
My intention is to allocate the remaining 28GB, so I can modify the initial command to:<br>
<code> !python main.py --base ./configs/i2i.yaml -t --gpus 0, 1,</code> (24GB/40GB) I’m not sure if there’s a way to do this because, as I understand, the more GPUs, the faster the training speed.</p>
<p>Actually, I’m not proficient in the field of multi-threading. If there’s any misunderstanding about this issue, please point it out to help me. I appreciate it. <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=12" title=":sweat_smile:" class="emoji" alt=":sweat_smile:" loading="lazy" width="20" height="20"></p>

ptrblck: <p>Increase the batch size or use a larger model.</p>
How to add LibTorch-Lite as a framework in Xcode

alisonqiu: <p>I’m trying to use  LibTorch-Lite in my Development Pod files but doing pod install with cocoapods won’t add it as a framework. I tried to directly add the folder installed from running pod install into frameworks, but it doesn’t work. How can I add LibTorch-Lite as a framework in Xcode (like AVFoundation)?</p>

Cyrill: <p>Hi alisonqiu<br>
Did you eventually find a solution?</p>
concatDataset make training speed slow down exponentially

Weixin_Jin: <p>Hello. I have several numpy <code>.npy</code> dataset files. Each of them have thousands of samples, with shape like <code>(num_sumples, channels, width, height)</code>.</p>
<p>If I use one of the files as training set and load it in <code>Dataset</code>:</p>
<pre><code class="lang-python">class MyDataset(Dataset):
    """npy dataset"""

    def __init__(self, *args):
        self.inputs = np.load(input_file, mmap_mode='r')
        self.labels = np.load(label_file, mmap_mode='r')

    def __len__(self):
        return len(self.labels)

    def __getitem__(self, idx):
        X = torch.from_numpy(np.copy(self.inputs[idx]))
        y = torch.from_numpy(np.copy(self.labels[idx]))
        return X, y

trainset = MyDataset(*args)
trainloader = DataLoader(trainset, batch_size=256)
</code></pre>
<p>It trains very fast. But if I use <code>Dataset</code> to load multiple <code>npy</code> files and concat them to a <code>concatDataset</code>, the speed slow down exponentially. Such as 1 epoch with 10 files will take 100x time than training with only 1 file.</p>
<p>How do I speed up the training process? I guess I can load the dataset into memory one file by one file. Is it possible?</p>
<p>Thank you!</p>

ptrblck: <p>Could you check, if you are running out of host memory and are using the swap?</p>
<aside class="quote no-group" data-username="Weixin_Jin" data-post="1" data-topic="111309">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/weixin_jin/48/32493_2.png" class="avatar"> Weixin_Jin:</div>
<blockquote>
<p>I guess I can load the dataset into memory one file by one file. Is it possible?</p>
</blockquote>
</aside>
<p>Yes, this would be possible, but you would have to consider how and if shuffling should be working.<br>
E.g. if you are lazily opening each file in the <code>__getitem__</code> method, I would expect the performance to be really bad.<br>
On the other hand, you could implement a logic, which opens a single numpy file, shuffles the indices, reads all samples, and loads the next file.</p>
<p>In case you are indeed running out of memory, you could try to use <code>np.memmap</code>, which doesn’t read the complete array, but allows you to index it from your disc.</p>

jsetty: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>, Could you please share an example of what you mean? Do you mean inside the <code>getitem</code>?</p>
<blockquote>
<p>you could implement a logic, which opens a single numpy file, shuffles the indices, reads all samples, and loads the next file</p>
</blockquote>
<p>How to handle the index argument for <code>getitem</code>? If have 10 files, each with 1000 images, should my <code>len()</code> be 10 or 10000?</p>
Pytorch flash attention error: RuntimeError: !grad_accumulator_.expired() INTERNAL ASSERT FAILED

Doffery: <p>Error reported when using flash attention 2 from transfomers with pytorch.</p>
<pre><code class="lang-auto">File "/opt/ml/code/train_fsdp.py", line 614, in main
    loss.backward()
  File "/opt/conda/lib/python3.10/site-packages/torch/_tensor.py", line 487, in backward
    torch.autograd.backward(
  File "/opt/conda/lib/python3.10/site-packages/torch/autograd/__init__.py", line 200, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
  File "/opt/conda/lib/python3.10/site-packages/torch/autograd/function.py", line 274, in apply
    return user_fn(self, *args)
  File "/opt/conda/lib/python3.10/site-packages/flash_attn/flash_attn_interface.py", line 529, in backward
    q, k, v, out, softmax_lse, rng_state = ctx.saved_tensors
RuntimeError: !grad_accumulator_.expired() INTERNAL ASSERT FAILED at "../torch/csrc/autograd/saved_variable.cpp":226, please report a bug to PyTorch. No grad accumulator for a saved leaf
</code></pre>
<p>Using</p>
<pre><code class="lang-auto">pip==23.3.1
torch==2.0.1
transformers==4.34.1
tokenizers&gt;=0.13.3
sentencepiece&gt;=0.1.99
rouge_score==0.1.2
accelerate&gt;=0.20.3
optimum&gt;=1.11.1
tqdm&gt;=4.54.1
pyarrow&gt;=10.0.1
datasets&gt;=1.18.2
xformer&gt;=1.0.1
packaging&gt;=23.1
ninja&gt;=1.11.1
flash-attn&gt;=2.3.3
</code></pre>
<p>Tried Pytorch 2.1.0, more errors came out.</p>
nn.CrossEntropyLoss not giving expected value

Vidit_Agarwal: <p>In my case I am passing inputs (logits) and targets both have same shape (N, C).<br>
In the example below the cross entropy loss should have been <code>0</code> but it isn’t.</p>
<pre data-code-wrap="py"><code class="lang-py">torch.nn.CrossEntropyLoss()(torch.tensor([[1,0,0,0,0.0]]), torch.tensor([[1,0,0,0,0.0]]))
&gt;&gt;&gt; tensor(0.9048)
</code></pre>
<p>Any possible reason?</p>

ptrblck: <p>This is expected since logits are unbound to <code>[-Inf, +Inf]</code>. Pass a large positive value and the loss should converge to zero:</p>
<pre data-code-wrap="python"><code class="lang-python">torch.nn.CrossEntropyLoss()(torch.tensor([[100.,0,0,0,0.0]]), torch.tensor([[1,0,0,0,0.0]]))
</code></pre>

Vidit_Agarwal: <p>So how should I modify my model or training pipeline so it works correctly and I get a 0 loss as expected?</p>

Vidit_Agarwal: <p>Thank you, it would be great if you could explain why this happens as per the formula in the docs the value should be 0 even for the values I used in the example.</p>

ptrblck: <p>No, based on the formula in the docs the output is expected since logits are used for <code>x</code>:</p>
<pre data-code-wrap="python"><code class="lang-python">torch.nn.CrossEntropyLoss()(torch.tensor([[1.,0,0,0,0.0]]), torch.tensor([[1,0,0,0,0.0]]))
# tensor(0.9048)

a = torch.tensor([[1.,0,0,0,0.0]])
b = torch.tensor([[1,0,0,0,0.0]])
-1.0 * torch.log(torch.exp(a[0, b.argmax()]) / torch.exp(a).sum())
# tensor(0.9048)
</code></pre>
Replace values in tensor

Aryam_Inria: <p>Hey,<br>
Given a <code>before</code> and <code>after</code> tensor, I want to replace all instance of <code>before</code> in another tensor <code>A</code> with <code>after</code> without using loops.</p>
<p>Example:</p>
<pre data-code-wrap="python"><code class="lang-python">before = torch.Tensor([2,4,5])
after  = torch.Tensor([20,40,50])
A = torch.Tensor([1,2,3,4,5,6])

result = replace(A, before, after)
</code></pre>
<p>The result should be <code>torch.Tensor([1,20,3,40,50,6])</code>.</p>
<p>Thanks!</p>

vardan: <p>You can use torch.where to replace it:</p>
<pre><code class="lang-auto">for b, a in zip(before, after):
    A = torch.where(A == b, a, A)
</code></pre>

Aryam_Inria: <p>Hello <a class="mention" href="/u/vardan">@vardan</a>, thank you for the reply. I want to avoid loops. I have to replace a lot of values, so this may be inefficient.</p>

vardan: <p>If you are guaranteed to have all elements in before that are present in A then you can do:</p>
<pre><code class="lang-auto">idxs = torch.nonzero(A.unsqueeze(1) == before, as_tuple=True)[0]
A[idxs] = after
</code></pre>

Aryam_Inria: <p>Yes, all elements in <code>before</code> are present in <code>A</code>. This is a good solution, thank you! However, I am dealing with large 2D tensors, and the unsqueeze and compare step may be heavy on the memory if <code>before</code> is large.</p>

Aryam_Inria: <p>There is a minor bug in case the tensors are not sorted, this solves it:</p>
<pre data-code-wrap="python"><code class="lang-python">idx_0, idx_1 = torch.nonzero(A.unsqueeze(1) == before, as_tuple=True)
A[idx_0] = after[idx_1]
</code></pre>
DataLoader worker is killed by signal: Segmentation fault

JasonWang: <p>I am using 3090ti for training. I have chosen lmdb for reading the data. but during the training phase the dataloader is randomly interrupted by signals and generates segment errors.</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "train.py", line 138, in &lt;module&gt;
    train()
  File "train.py", line 109, in train
    train_loss, train_top1_acc, train_topk_acc, global_iter = trainer.train()
  File "/home/chem/rsmiles/utils/train_utils.py", line 101, in train
    loss_token = self.criterion_tokens(pred_token_logit, gt_token_label)
  File "/home/chem/.miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/chem/rsmiles/utils/loss_utils.py", line 24, in forward
    indices = torch.Tensor([[torch.arange(len(label))[i].item(),
  File "/home/chem/.miniconda3/envs/py38/lib/python3.8/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
    _error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 21706) is killed by signal: Segmentation fault.
</code></pre>
<p>Sometimes the computer will just shut down.But the strange thing is that if I remove the code for training and iterate only the dataset, it will not generate any error. Also if I open two processes, one to iterate only the dataset and one to train, both processes will run normally.</p>

ptrblck: <p>Check the stacktrace via:</p>
<pre><code class="lang-auto">gdb --args python script.py args
...
run
...
bt
</code></pre>
<p>to narrow down which code causes the segfault.</p>
Set Torch.backends.cudnn.benchmark = True consumes huge amount of memory

YoYoYo: <p>I am training a progressive GAN model with torch.backends.cudnn.benchmark = True. The ProGAN progressively add more layers to the model during training to handle higher resolution images.</p>
<p>I notice that at the beginning of the training the GPU memory consumption fluctuate a lot, sometimes it exceeds 48 GB memory and lead to the CUDNN_STATUS_INTERNAL_ERROR. However, after the period of fluctuation, the memory consumption is back to normal, which is only around 3~4 GB. I suspect the fluctuation is from the auto-tuning of the convolution algorithm by CuDNN, so I set torch.backends.cudnn.benchmark = False, then everything runs smoothly.</p>
<p>My question is (1) should I use cudnn.benchmark when the network, i.e. ProGAN, changes its structure after a few epochs, like 50 epochs. (2) Is the excessive memory consumption from the auto-tuning? Or is it a bug in my code?</p>

ptrblck: <ol>
<li>
<p>Yes, you could use it for better performance especially if you are dealing with static input shapes (or a few different ones). Each new input shape (and conv setup) will rerun the heuristics and will thus slow down your code. Afterwards this setup will be cached.</p>
</li>
<li>
<p>Yes, it’s coming from running different algorithms internally, which might have different memory requirements.</p>
</li>
</ol>

YoYoYo: <p>Thanks for your reply! I am a bit surprised that the benchmark consumes so much memory. I have a 48 GB GPU and still got OOM, while the training of the same network only cost 3~4 GB. What else I could do to lower the memory consumption on benchmark? Currently I could not turn it on without error. Thanks again.</p>

ptrblck: <p>You should not run into an OOM and yes, some algorithms might try to use a huge workspace, if they need to e.g. create copies to permute and realign data.<br>
As a workaround, you could set <code>CUDNN_CONV_WSCAP_DBG</code> to a lower value in MiB to skip algorithms requesting a larger workspace (<a href="https://docs.nvidia.com/deeplearning/cudnn/release-notes/rel_8.html#rel-805">description of this env var</a>, <a href="https://github.com/pytorch/pytorch/issues/49207">proposal to expose this mechanism</a>).</p>

bruceSz: <aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="131010">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>CUDNN_CONV_WSCAP_DBG</p>
</blockquote>
</aside>
<p>Is the setup will be recorded in pytorch or in cudnn itself? can we dump this setup out and reuse it from pytorch user side?</p>

ptrblck: <p>The environment variable is used in cuDNN directly and can be set in your terminal or before loading cuDNN in your script.</p>
Torch.nan not supported in int16

polo5: <p>Is it a bug that <code>torch.nan</code> throws an error in int16 ?</p>
<pre><code class="lang-auto">a = torch.tensor([1,2,torch.nan]) #works fine 
b = torch.tensor([1,2,torch.nan], dtype=torch.int16) #throws error: RuntimeError: value cannot be converted to type int16 without overflow
</code></pre>
<p>I don’t see why the <code>torch.nan</code> functionality wouldn’t be extended to int16.</p>

albanD: <p>Hi,</p>
<p>nan a floating point concept. none of the integer dtype define nans as they all represent only valid integers in a given range.</p>
Is it possible to initialize only one layer when multiple LayerNorm layers need to be used in Pytorch?

mst72: <p>Is it possible to initialize only one layer when multiple LayerNorm layers need to be used in Pytorch? For example: self.layernorm = nn.LayerNorm() instead of writing self.layernorm1 = nn.LayerNorm() self.layernorm2 = nn.LayerNorm()</p>
<p>And I would like to know which other ones are possible to initialize only one and which ones are not.</p>
<p>Thanks for reading and considering my queries！</p>

albanD: <p>Hi,</p>
<p>Any layer that has no state will have the same behavior if re-used multiple times.<br>
Anything that has some learnable parameters or tracks statistics.</p>
<p>In the case of layer norm, you can re-use it without changing behavior only if elementwise_affine=False (meaning you have no parameter).</p>

mst72: <p>Thanks for your answer, but I have some more questions, for dropout and relu, do they have learnable parameters?<br>
For example，in the Transfromer encoderlayer code below I have defined 3 dropout layers, but if dropout doesn’t have learnable parameters wouldn’t it be better for me to define just one, why is it so complicated to define three:</p>
<pre><code class="lang-auto">class DecoderLayer(nn.Module):
    def __init__(self, d_model, heads, dropout=0.1):
        super().__init__()
        self.norm_1 = Norm(d_model)
        self.norm_2 = Norm(d_model)
        self.norm_3 = Norm(d_model)
        
        self.dropout_1 = nn.Dropout(dropout)
        self.dropout_2 = nn.Dropout(dropout)
        self.dropout_3 = nn.Dropout(dropout)
        
        self.attn_1 = MultiHeadAttention(heads, d_model, dropout=dropout)
        self.attn_2 = MultiHeadAttention(heads, d_model, dropout=dropout)
        self.ff = FeedForward(d_model, dropout=dropout)

    def forward(self, x, e_outputs, src_mask, trg_mask):
        x2 = self.norm_1(x)
        x = x + self.dropout_1(self.attn_1(x2, x2, x2, trg_mask))
        x2 = self.norm_2(x)
        x = x + self.dropout_2(self.attn_2(x2, e_outputs, e_outputs, \
        src_mask))
        x2 = self.norm_3(x)
        x = x + self.dropout_3(self.ff(x2))
        return x
</code></pre>

albanD: <p>Hi,</p>
<p>ReLU and Dropout don’t have learnable parameters no.<br>
But there is no downside at defining 3 so you can do that if it is easier for you to use.</p>
Memory out issue in cuda

Anam: <p>Hi all,<br>
I am using GPU: RTX3080 with ubunto version 22.04<br>
Cuda Version 11.7<br>
when i try to run my training code i face this issue<br>
1525         or _global_backward_pre_hooks or _global_backward_hooks<br>
1526         or _global_forward_hooks or _global_forward_pre_hooks):<br>
 → 1527     return forward_call(*args, **kwargs)<br>
1529 try:<br>
1530     result = None</p>
<p>File ~/anaconda3/envs/13-Dec_DL_code/lib/python3.9/site-packages/torch/nn/modules/pooling.py:166, in MaxPool2d.forward(self, input)<br>
165 def forward(self, input: Tensor):<br>
 → 166     return F.max_pool2d(input, self.kernel_size, self.stride,<br>
167                         self.padding, self.dilation, ceil_mode=self.ceil_mode,<br>
168                         return_indices=self.return_indices)</p>
<p>File ~/anaconda3/envs/13-Dec_DL_code/lib/python3.9/site-packages/torch/_jit_internal.py:488, in boolean_dispatch..fn(*args, **kwargs)<br>
486     return if_true(*args, **kwargs)<br>
487 else:<br>
 → 488     return if_false(*args, **kwargs)</p>
<p>File ~/anaconda3/envs/13-Dec_DL_code/lib/python3.9/site-packages/torch/nn/functional.py:791, in _max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode, return_indices)<br>
789 if stride is None:<br>
790     stride = torch.jit.annotate(List[int], <span class="chcklst-box fa fa-square-o fa-fw"></span>)<br>
 → 791 return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)</p>
<p>OutOfMemoryError: CUDA out of memory. Tried to allocate 64.00 MiB. GPU 0 has a total capacty of 9.77 GiB of which 62.44 MiB is free. Process 1208 has 217.56 MiB memory in use. Process 47813 has 7.84 GiB memory in use. Including non-PyTorch memory, this process has 1.38 GiB memory in use. Of the allocated memory 1.12 GiB is allocated by PyTorch, and 9.97 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF<br>
Selection deleted</p>

ptrblck: <aside class="quote no-group" data-username="Anam" data-post="1" data-topic="194652">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/47e85d/48.png" class="avatar"> Anam:</div>
<blockquote>
<p>Process 47813 has 7.84 GiB memory in use.</p>
</blockquote>
</aside>
<p>It seems the majority of the memory is used by another process so check what it is.</p>

Anam: <p>How can i check from my understanding no other process is running except the code</p>
TypeError: 'Compose' object is not iterable

Elton_Lobo: <p>I am trying to create a custom class for my dataset. Can anyone figure out what is wrong in the code given below?</p>
<pre><code class="lang-auto">class Map_dataset(Dataset):
    def __init__(self,root_dir,transforms=None):
        self.transforms=transforms
        self.root_dir=root_dir
        self.list_dir=os.listdir(self.root_dir)
    def __getitem__(self,idx):
        img_path=os.path.join(self.root_dir,self.list_dir[idx])
        whole_image=np.array(Image.open(img_path))
        input_image=torch.tensor(whole_image[:,:600,:])
        target_image=torch.tensor(whole_image[:,600:,:])
        if (self.transforms):
            for t in self.transforms:
                input_image=t(input_image)
                target_iage=t(target_image)
        return input_image,target_image
    def __len__(self):
        return len(self.list_dir)

img_size=256
batch_size=16
composed=torchvision.transforms.Compose([torchvision.transforms.Resize((img_size,img_size))])
data=Map_dataset("../input/pix2pix-maps/train",composed)
data_loader=DataLoader(data,batch_size,shuffle=True)
input_images,target_images=iter(data_loader).next()
print(input_images.shape,target_images.shape)
</code></pre>

ptrblck: <p>Based on the error message, the issue is in trying to iterate <code>transforms.Compose</code>, which is not working:</p>
<pre><code class="lang-python">for t in self.transforms:
    input_image=t(input_image)
    target_iage=t(target_image)
</code></pre>
<p><code>transforms.Compose</code> will apply the transformation in the specified order, so you can use it directly:</p>
<pre><code class="lang-python">input_image = self.transforms(input_image)
</code></pre>

franciskasara: <p>How can I make sure to apply the same transforms to the <em>input_image</em> and the <em>target_image</em>? For example if I have RandomFlip or something other “random” transformation.</p>
Register_buffer extra memory

Lakshya1: <p>Does using register_buffer lead to extra memory usage at runtime? For a specific case consider the following:</p>
<pre data-code-wrap="class"><code class="lang-class">
def __init__(self, d_model, max_len=5000):
    """
    Inputs
        d_model - Hidden dimensionality of the input.
        max_len - Maximum length of a sequence to expect.
    """
    super().__init__()

    # Create matrix of [SeqLen, HiddenDim] representing the positional encoding for max_len inputs
    pe = torch.zeros(max_len, d_model)
    position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
    div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model))
    pe[:, 0::2] = torch.sin(position * div_term)
    pe[:, 1::2] = torch.cos(position * div_term)
    pe = pe.unsqueeze(0)

    # register_buffer =&gt; Tensor which is not a parameter, but should be part of the modules state.
    # Used for tensors that need to be on the same device as the module.
    # persistent=False tells PyTorch to not add the buffer to the state dict (e.g. when we save the model)
    self.register_buffer('pe', pe, persistent=False)

def forward(self, x):
    x = x + self.pe[:, :x.size(1)]
    return x

</code></pre>
<p>As per the documentation, <a href="https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.modules.module.Module.register_buffer" class="inline-onebox" rel="noopener nofollow ugc">Module — PyTorch 2.1 documentation</a>, I think maybe not as it uses:</p>
<pre><code class="lang-auto">self._buffers[name] = tensor
</code></pre>
<p>But I’m not sure about the hook part. Thank you</p>
RNN's and imbalanced data

Sai1: <p>Hi all, I have signal data on which I’m doing binary classification. I trained a Resnet 50 with 1D convs and achieved 85% accuracy. In the process of improving the performance, I’m currently training a Bidirectional RNN with LSTM. I heard a couple of times that RNNs perform really well for sequential data.  But, I was able to only achieve 75% with RNN model. I tried different settings for hidden layers and num_layers in LSTM, but nothing seems to have worked. Any suggestions that could help me? The data is imbalanced with 80-20 distribution. Accuracy is not my primary metric. AUC was 81 for RNN and 93 for RESNET.<br>
Thanks in Advance.</p>

J_Johnson: <p>Standard accuracy scores mean very little with a highly unbalanced dataset.</p>
<p>If you have an 80:20 distribution, we can get 80% accuracy with standard scoring methods by just writing a few lines of code to always return the 80% class.</p>
<p>Here is an accuracy method which allows you to get class specific accuracy, or a mean of each class accuracy:</p>
<aside class="quote quote-modified" data-post="1" data-topic="194579">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/j_johnson/48/55494_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-do-i-get-the-accuracy-of-an-unbalanced-dataset-or-segmentation-task/194579">How do I get the accuracy of an unbalanced dataset or segmentation task?</a> <a class="badge-category__wrapper " href="/c/faq/30"><span data-category-id="30" data-drop-close="true" class="badge-category " title="The FAQ category contains commonly-asked questions and their answers. Please refer to this section before you post your query."><span class="badge-category__name">FAQ</span></span></a>
  </div>
  <blockquote>
    This post is to define a Class Weighted Accuracy function(WCA). As questions related to this get asked often, I thought it might help people to post a tool torchers can use and reference here. 
Why? 
Take, for example, a classification dataset of kittens and puppies with a ratio of 0.2:0.8 kittens to puppies. A model trained on this dataset might show an overall accuracy of 80% by just predicting “puppies” every time. 
A WCA function resolves this problem by checking the accuracy of each class s…
  </blockquote>
</aside>

<p>Regarding training on an unbalanced dataset, loss functions allow you to pass in a <code>weight</code>(for binary classes) or <code>pos_weight</code>(for multi-class) argument. These will in turn increase the loss ratio for the minority classes and/or decrease the loss ratio for the majority classes.</p>

Sai1: <p>Thanks Johnson. I’ve tried weighted loss function quite a few times in the past but never had any success with it. But RNN seems to be doing a lot better with weighted loss function than plain loss function.</p>
Suggestion for Small Sized Video Object Detection Dataset

weight_theta: <p>Dear Community,</p>
<p>Due to the large size of the ImageNet VID 2017 dataset (train: 800GB +, test: 800GB +), i am looking for a dataset that is smaller (i.e., 200 GB max atm) for sanity checking. Currently I do not have the resources to train one ImageNet Vid, but once results on a smaller dataset seem compelling, the necessary arrangements can be made.</p>
<p>Please let me know if a Dataset comes to mind.</p>
<p>Best,</p>
<p>Weight Theta</p>
Intellectual Property Concerns Regarding Private Deployment of LLM for Customers

incerto: <p>Dear PyTorch Community,</p>
<p>I am reaching out to seek your guidance on an intellectual property concern that I have encountered while attempting to conduct a private deployment of the LLM for my customers. The issue pertains to the potential vulnerability of the model weights and associated server codes when they are deployed on the customers’ hardware.</p>
<p>As the customers possess root permissions to the hardware, I am uncertain about the level of protection that can be provided to prevent the model weights from being leaked or compromised. I am particularly concerned about the ease with which customers may be able to copy the model weights directly from the GPU memory.</p>
<p>I would greatly appreciate any insights or recommendations that you may have to address this concern.</p>