akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Kernel did not dies but no return any result after certain epochs

username: eatcah
cooked: <p>Hi there,<br>
I am facing issue with my code. Here are the explanations:</p>
<ol>
<li>
<p>Output from anaconda prompt:<br>
Traceback (most recent call last):<br>
File “C:\Users\User\anaconda3\envs\pytorch-gpu\lib\multiprocessing\queues.py”, line 244, in _feed<br>
obj = _ForkingPickler.dumps(obj)<br>
File “C:\Users\User\anaconda3\envs\pytorch-gpu\lib\multiprocessing\reduction.py”, line 51, in dumps<br>
cls(buf, protocol).dump(obj)<br>
File “C:\Users\User\anaconda3\envs\pytorch-gpu\lib\site-packages\torch\multiprocessing\reductions.py”, line 261, in reduce_tensor<br>
event_sync_required) = storage.<em>share_cuda</em>()<br>
File “C:\Users\User\anaconda3\envs\pytorch-gpu\lib\site-packages\torch\storage.py”, line 920, in <em>share_cuda</em><br>
return self._untyped_storage.<em>share_cuda</em>(*args, **kwargs)<br>
RuntimeError: CUDA error: mapping of buffer object failed<br>
Compile with <code>TORCH_USE_CUDA_DSA</code> to enable device-side assertions.</p>
</li>
<li>
<p>Output from Jupyter Notebook: it runs until certain epoch then eventually not returning anything until the error above shows up.</p>
</li>
<li>
<p>My model:<br>
class CNNNetwork(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(CNNNetwork, self).<strong>init</strong>()<br>
self.conv1 = nn.Sequential(nn.Conv2d<br>
(in_channels=1, out_channels=16,<br>
kernel_size=3, stride=1,<br>
padding=2),<br>
nn.ReLU(),<br>
nn.MaxPool2d(kernel_size=2))<br>
self.conv2 = nn.Sequential(nn.Conv2d<br>
(in_channels=16, out_channels=32,<br>
kernel_size=3, stride=1,<br>
padding=2),<br>
nn.ReLU(),<br>
nn.MaxPool2d(kernel_size=2))<br>
self.conv3 = nn.Sequential(nn.Conv2d<br>
(in_channels=32, out_channels=64,<br>
kernel_size=3, stride=1,<br>
padding=2),<br>
nn.ReLU(),<br>
nn.MaxPool2d(kernel_size=2))<br>
self.conv4 = nn.Sequential(nn.Conv2d<br>
(in_channels=64, out_channels=128,<br>
kernel_size=3, stride=1,<br>
padding=2),<br>
nn.ReLU(),<br>
nn.MaxPool2d(kernel_size=2))<br>
self.flatten = nn.Flatten()<br>
self.fc1 = nn.Linear(128<em>2</em>20, 120)<br>
self.fc2 = nn.Linear(120, 60)<br>
self.fc3 = nn.Linear(60, 2)</p>
<p>def forward(self, x):<br>
x = self.conv1(x)<br>
x = self.conv2(x)<br>
x = self.conv3(x)<br>
x = self.conv4(x)<br>
x = self.flatten(x)<br>
x = x.view(-1, 128<em>2</em>20)<br>
x = F.relu(self.fc1(x))<br>
x = F.relu(self.fc2(x))<br>
x = self.fc3(x)<br>
return x</p>
</li>
</ol>
<p>I am still new to this, hope anyone can help me.<br>
Thanks!</p>

username: ptrblck
cooked: <p>Try to remove the usage of multiple processes, e.g. by setting <code>num_workers=0</code>, and also make sure the if-clause guard is used as explained <a href="https://pytorch.org/docs/stable/notes/windows.html#usage-multiprocessing">here</a>.</p>

username: eatcah
cooked: <p>Thank you so much for your response!<br>
It took me a day to solve this issue.<br>
Thank you!!!</p>
What is NFCC mentioned in the tutorial?

username: Mehran_Ziadloo
cooked: <p>Continuing my journey reading the PyTorch tutorials, I’m at “<a href="https://pytorch.org/audio/stable/tutorials/audio_feature_extractions_tutorial.html" rel="noopener nofollow ugc">AUDIO FEATURE EXTRACTIONS</a>” page.</p>
<p>At the bottom of the page, there’s a section titled “<a href="https://pytorch.org/audio/stable/tutorials/audio_feature_extractions_tutorial.html#kaldi-pitch-beta" rel="noopener nofollow ugc">Kaldi Pitch (beta)</a>”. The code accompanying this section shows how to generate two new features, “pitch” and “nfcc”. This was my first time hearing the term “nfcc” so I had to search it online. And I must say, I could not find anything. But the paper they link to in the same section has another acronym, NCCF.</p>
<p>I wonder, is this a typo in the tutorial? Or there’s actually an acronym called “NFCC”? If it’s the latter case, what is “NFCC”?</p>
<p>One thing that makes me think that this is not a typo and that the value calculated in the tutorial is not NCCF is the fact that NCCF stands for Normalized Cross Correlation Function. And as the result, by nature, it’s a value between -1 and 1. But the tensor calculated in the tutorial is way beyond that (125 to 250).</p>
<p>Appreciate it if someone could help me understand what this NFCC is and how can I learn more about it. Thanks.</p>

username: ptrblck
cooked: <p>I think it is a typo as the <a href="https://pytorch.org/audio/main/generated/torchaudio.functional.compute_kaldi_pitch.html#torchaudio-functional-compute-kaldi-pitch">docs</a> show that <code>torchaudio.functional.compute_kaldi_pitch</code> returns:</p>
<blockquote>
<p>Pitch feature. Shape: (batch, frames 2) where the last dimension corresponds to pitch and NCCF.</p>
</blockquote>
<p>However, I don’t know why the values are apparently not normalized to <code>[-1, 1]</code>.</p>
Can not load GPT-J6B on 32 GB instance

username: Aayush_Shah
cooked: <p>I have used <strong>ml.g4dn.2xlarge</strong> instance on SageMaker to test GPT-J6B model from HuggingFace using Transformer.</p>
<p>I am using <code>revision=float16</code> and low_cpu_usage=True so that the model is only of 12GB.</p>
<p>It is downloaded but after that it suddenly crashes the kernel.</p>
<p>Please share the workaround. The memory of that instance is 32 GB wit 4 vCPU.</p>
<p>Thanks</p>

username: eqy
cooked: <p>If the crash is due to OOM on the CPU side during model loading rather than inference, I would check if e.g., increasing the swap size for loading could help:</p><aside class="onebox stackexchange" data-onebox-src="https://stackoverflow.com/questions/17173972/how-do-you-add-swap-to-an-ec2-instance">
  <header class="source">

      <a href="https://stackoverflow.com/questions/17173972/how-do-you-add-swap-to-an-ec2-instance" target="_blank" rel="noopener nofollow ugc">stackoverflow.com</a>
  </header>

  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/1223293/ajtrichards" target="_blank" rel="noopener nofollow ugc">
    <img alt="ajtrichards" src="https://i.stack.imgur.com/64ImH.png?s=256&amp;g=1" class="thumbnail onebox-avatar" width="256" height="256">
  </a>

<h4>
  <a href="https://stackoverflow.com/questions/17173972/how-do-you-add-swap-to-an-ec2-instance" target="_blank" rel="noopener nofollow ugc">How do you add swap to an EC2 instance?</a>
</h4>

<div class="tags">
  <strong>amazon-web-services, amazon-ec2, out-of-memory</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/1223293/ajtrichards" target="_blank" rel="noopener nofollow ugc">
    ajtrichards
  </a>
  on <a href="https://stackoverflow.com/questions/17173972/how-do-you-add-swap-to-an-ec2-instance" target="_blank" rel="noopener nofollow ugc">04:17PM - 18 Jun 13 UTC</a>
</div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


username: matt.carlson
cooked: <p>Are the crashes happening before data is loaded?</p>
<p>One issue I have had is even if you move tensors to the gpu, they still take up system memory. If that is the issue, maybe streaming the data could help. <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.IterableDataset" class="inline-onebox" rel="noopener nofollow ugc">torch.utils.data — PyTorch 2.0 documentation</a></p>
Assignment to Tensor in Similar Way to Torch.select

username: bchen
cooked: <p>Is there an easy way to assign values to a tensor in a way that’s similar to torch.select?</p>
<p>For instance, say I have a tensor with n dimensions and I want to change the mth “row” of the kth dimension to be 1. If n = 5, m = 2, and k = 3, I could do something like tensor[:, :, :, 2, :] = 1 - but let’s say that m, n, and k are not known beforehand, or I want to loop through different values of m and k, etc. If I just wanted to retrieve the values of these slices, torch.select(tensor, 3, 2) would work, but I want to be able to make changes to this slice.</p>

username: ptrblck
cooked: <p>Could you post a (slow) example how exactly this assignment should look like which we could use as a reference?</p>

username: bchen
cooked: <p>Sure - the exact use case I’m thinking of looks like this, except the size of the tensor can be different in different cases.</p>
<pre><code class="lang-auto">import torch
tensor = torch.ones([5,6,7])

tensor[0] *= 0.5
tensor[-1] *= 0.5
tensor[:, 0] *= 0.5
tensor[:, -1] *= 0.5
tensor[:, 0] *= 0.5
tensor[:, -1] *= 0.5
tensor[..., 0] *= 0.5
tensor[..., -1] *= 0.5
</code></pre>
<p>I’m hoping there’s a way to somehow loop through the dimensions, sort of like</p>
<pre><code class="lang-auto">for dim in range(len(tensor.shape)):
    torch.select(tensor, dim, 0) *= 0.5 # But with a different function than torch.select
    torch.select(tensor, dim, -1) *= 0.5
</code></pre>
Dot product of two 4D tensors

username: Surekha_Gaikwad
cooked: <p>I have two tensors of size, t1 as [16,64,56,56] and t2 as [16,64,56,56]. Now I wanted to do perform dot product between two tensors to get final tensor with size as [16,64,16,64].</p>
<p>I needed this final tensor of required size so that I can again dot product this with t1 (as original features) in order to get refined features of original size [16,64,56,56]</p>
<p>How will I perform dot product between two tensors t1 and t2?<br>
Note:- Something like in snapshot with attention matrix as CxC/2</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/e/0e1081af545aa5fa768ac7af0765881a0c9cece9.png" data-download-href="https://discuss.pytorch.org/uploads/default/0e1081af545aa5fa768ac7af0765881a0c9cece9" title="Screen Shot 2020-02-13 at 4.29.50 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/e/0e1081af545aa5fa768ac7af0765881a0c9cece9_2_690x262.png" alt="Screen Shot 2020-02-13 at 4.29.50 PM" data-base62-sha1="20q35kA93DNLnMAyp3AURTtkAbv" width="690" height="262" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/e/0e1081af545aa5fa768ac7af0765881a0c9cece9_2_690x262.png, https://discuss.pytorch.org/uploads/default/optimized/3X/0/e/0e1081af545aa5fa768ac7af0765881a0c9cece9_2_1035x393.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/e/0e1081af545aa5fa768ac7af0765881a0c9cece9.png 2x" data-dominant-color="EAEAE9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-02-13 at 4.29.50 PM</span><span class="informations">1218×464 74.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: JuanFMontesinos
cooked: <p>Could you clarify what is dot product of 4D tensors?</p>

username: Surekha_Gaikwad
cooked: <p>Something like shown in snapshot dot product of CxH/2XW/2 and C/2xH/2xW/2 to get CxC/2.<br>
Here it is just 3D tensor but in my case there is extra dimension as batch.</p>

username: Yochanan_Scharf
cooked: <p>For dot_product you have several options:</p>
<p><code>torch.dot</code> expects a <code>1D</code> <code>tensor</code>.</p>
<p><code>torch.mm</code> expects a <code>1D</code> or <code>2D</code> <code>tensor</code>.</p>
<p>You can do  torch.flatten(x) to get a 1D tensor.</p>
<p><code>torch.matmul</code> works for any rank but the output depends on the rank as expounded on in the docs.</p>
<p><a href="https://pytorch.org/docs/stable/generated/torch.matmul.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.matmul.html</a></p>
If we have any detail information of each CI in PyTorch's Github?

username: sanbuphy
cooked: <p>Hello, I have been studying the actions (CI) of various open source projects on Github recently.<br>
I noticed that PyTorch has a well-established CI, so I would like to further understand its composition and structure. Do you have any relevant materials that I can study and refer to?<br>
Thank you very much.</p>

username: ptrblck
cooked: <p><a href="https://github.com/pytorch/pytorch/wiki/Continuous-Integration">This Wiki page</a> might be helpful.</p>

username: sanbuphy
cooked: <p>Thank you very much!</p>
TraceError: symbolically traced variables cannot be used as inputs to control flow

username: Arturas_Druteika
cooked: <p>Hello,<br>
I am getting an error when trying to create a feature extractor using the <code>create_feature_extractor()</code> function (<code>from torchvision.models.feature_extraction import create_feature_extractor</code>). What I wanna do is:</p>
<ol>
<li>I create a model using timm library like this</li>
</ol>
<p><code>model = timm.create_model('maxvit_xlarge_tf_512.in21k_ft_in1k', pretrained=True)</code></p>
<ol start="2">
<li>I take attention layers</li>
</ol>
<pre><code class="lang-auto">attention_layer_names = []
layer_name = 'attn_drop'

for name, module in model.named_modules():
    if layer_name in name:
        attention_layer_names.append(name)
</code></pre>
<ol start="3">
<li>Try to create a feature extractor<br>
<code>feature_extractor = create_feature_extractor(model, return_nodes=attention_layer_names)</code>
</li>
</ol>
<p>I am getting this error</p>
<pre><code class="lang-auto">TraceError: symbolically traced variables cannot be used as inputs to control flow
</code></pre>
<p>How could I solve it?</p>

username: ptrblck
cooked: <p>The error is raised because <code>torch.fx</code> has trouble tracing <code>x = pad_same(x, self.kernel_size, self.stride)</code> which is used in this <code>timm</code> model.<br>
You could add this method to <code>autowrap_functions</code> as seen here:</p>
<pre><code class="lang-python">feature_extractor = torchvision.models.feature_extraction.create_feature_extractor(
    model, return_nodes=attention_layer_names, tracer_kwargs={'autowrap_functions': [timm.layers.padding.pad_same]})
</code></pre>
<p>which will then fail with:</p>
<pre><code class="lang-python">ValueError: node: 'stages.0.blocks.0.attn_block.attn.attn_drop' is not present in model. Hint: use `get_graph_node_names` to make sure the `return_nodes` you specified are present. It may even be that you need to specify `train_return_nodes` and `eval_return_nodes` separately.
</code></pre>
<p>as it seems this module is not used.<br>
It may be easier using forward hooks as described in <a href="https://discuss.pytorch.org/t/how-can-l-load-my-best-model-as-a-feature-extractor-evaluator/17254/6">this post</a> instead.</p>
Custom IterDataPipe for multiprocessing

username: ChriMeiTorch
cooked: <p>Hello everyone.</p>
<p>I’m not sure if i understand MultiProcessingReadingService correctly.<br>
Below is a small test with a custom IterDataPipe which should be spawned to 4 workers.</p>
<p>Without MultiProcessingReadingService it takes ~ 110ms on my PC.<br>
With MultiProcessingReadingService it takes minutes!<br>
I mean it’s not a little bit slower, its like… crazy <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>My versions: torchdata 0.6.0, Python 3.9.16<br>
I’m testing on a Win11 machine.</p>
<p>Am i doing it wrong?<br>
Is the slow down caused by IPC and multiprocessing should only be used for very heavy-CPU related stuff?<br>
Is it Windows related?</p>
<p>Thank you for any help!</p>
<pre><code class="lang-auto">from io import IOBase
import time
from typing import IO, Tuple
from torchdata.dataloader2 import DataLoader2, MultiProcessingReadingService
from torchdata.datapipes.iter import IterDataPipe


class TestIterDataPipe(IterDataPipe[Tuple[str, IOBase]]):
    def __init__(self, source_datapipe: IterDataPipe[Tuple[str, IO]]) -&gt; None:
        self.source_datapipe: IterDataPipe[Tuple[str, IO]] = source_datapipe

    def __iter__(self):
        for i in range(100000):
            yield 'foo', i


if __name__ == '__main__':

    tStep1 = time.perf_counter() * 1000.0
    tStart = tStep1

    pipe = TestIterDataPipe(None)
    mpReader = MultiProcessingReadingService(num_workers=4)
    dl2 = DataLoader2(pipe, reading_service=mpReader)

    tStep2 = time.perf_counter() * 1000.0
    tDiff1 = tStep2 - tStep1
    tStep1 = tStep2

    ndx = 0
    for dt1, dt2 in dl2:
        ndx += 1
        if not ndx % 123:
            tStep2 = time.perf_counter() * 1000.0
            tDiff2 = tStep2 - tStep1
            print(f'{ndx}: {dt2}, {tDiff2} ms')

    tStep2 = time.perf_counter() * 1000.0
    tDiff2 = tStep2 - tStep1
    tStep1 = tStep2

    print(f'pipeline: {tDiff1} ms, loop: {tDiff2} ms')
    print(f'total: {tStep2 - tStart} ms')

    dl2.shutdown()
</code></pre>

username: ChriMeiTorch
cooked: <p>Small update: looks like a Windows (native) problem.<br>
I setup WSL2 and recreated my conda-env in WSL2 (Ubuntu).<br>
It works much better now.<br>
I worked with Threads before on Windows, but not multiprocessing, and never had any problems.<br>
So i’m not sure if it’s related to torchdata oooor python itself?</p>
Computer Vision Project.Send me help for my FYP

username: Ethanlyt
cooked: <p>Hi PyTorch community, I am recently planninf for my FYP and facing some hurdle, hoping the community will give me some answer. I am planning to build a system and consist a feature to calculate larvae in water. I want to use cv algorithm to get the number of larvae, and the larvae will be small black dots. Will it be a good idea to use cv since the larvae is so small or rather do it manually using microscope?  Moreover, I will also need the larvae dataset for the model training which I can’t found it.<br>
I hope there are people out there that can help me, thank you so much!</p>

username: tiramisuNcustard
cooked: <p>You didn’t mention anything about the resolution of images. If the water sample is untreated then there will be other organisms apart from larvae and if the resolution is too small then all the organisms/particles might not be differentiable.</p>

username: Ethanlyt
cooked: <p>lets assume the water is still, the water only have all black dot (other particles) treat as the larvae, and using a high solution camera.</p>

username: tiramisuNcustard
cooked: <p>Assuming the camera is powerful enough to capture the differentiating features of the larvae and other particles, I think you will end up using image segmentation (i.e. drawing boxes or creating segmentation masks) to identify larvae. If you are not already familiar with image segmentation please do some background research in that area. It is widely used in CV.</p>

username: Ethanlyt
cooked: <p>Sure! Thank you so much! I believe this will be helpful!</p>
RuntimeError: Error(s) in loading state_dict for FasterRCNN

username: bugramurat
cooked: <p>I’m getting an error when i try to make predicts for my FasterRCNN object detection model. I was able to make predicts with same model file 2 days ago. But it gives error now.</p>
<p>Error:</p>
<pre><code class="lang-auto">RuntimeError: Error(s) in loading state_dict for FasterRCNN:
        size mismatch for roi_heads.box_predictor.cls_score.weight: copying a param with shape torch.Size([6, 1024]) from checkpoint, the shape in current model is torch.Size([13, 1024]).
        size mismatch for roi_heads.box_predictor.cls_score.bias: copying a param with shape torch.Size([6]) from checkpoint, the shape in current model is torch.Size([13]).
        size mismatch for roi_heads.box_predictor.bbox_pred.weight: copying a param with shape torch.Size([24, 1024]) from checkpoint, the shape in current model is torch.Size([52, 1024]).
        size mismatch for roi_heads.box_predictor.bbox_pred.bias: copying a param with shape torch.Size([24]) from checkpoint, the shape in current model is torch.Size([52]).
</code></pre>
<p>Code from test.py:</p>
<pre><code class="lang-auto"># load the best model and trained weights
model = create_model(num_classes=NUM_CLASSES)
checkpoint = torch.load('C:\\Users\\lemon\\Desktop\\outputs_first\\last_model.pth', map_location=DEVICE)
model.load_state_dict(checkpoint['model_state_dict'])
model.to(DEVICE).eval()
</code></pre>

username: thecho7
cooked: <p>The model you created has 13 classes where the state_dict you loaded has 6.</p>
<p>You should make them same.</p>

username: bugramurat
cooked: <p>Ohhhh i got it now, i was just changed dataset so it has 13 classes. I missed it. Thank you so much!</p>
How to get batch norm's running stats such as running_var and running_mean in pytorch

username: Cuiqing_Li
cooked: <p>Hi friends:<br>
I have a question. Suppose I have a model which contains batch norm layers. Then， due to some tasks’ requirements, I need to get the batch norm layers’ running_var and running_mean at the end of training or evaluation process.<br>
for example, here is a simple code:</p>
<pre><code class="lang-auto">    class Net(nn.Module):
        def __init__(self):
            super().__init__()
            self.conv1 = nn.Conv2d(in_channels = in_channels, out_channels = out_channels,
                        kernel_size = 3, padding=1, bias = False)
            self.bn = nn.BatchNorm2d(out_channels)

        def forward(self, x):
            x = self.conv1(x)
            x = self.bn(x)
            return x
    
    my_net = Net()
</code></pre>
<p>I want to get my_net’s running var and running_mean of batch norm layers, how should I do? Thank you!!!</p>

username: ptrblck
cooked: <p>You can directly access them via:</p>
<pre><code class="lang-python">my_net.bn.running_mean
my_net.bn.running_var
</code></pre>

username: Alva-2020
cooked: <p>Hello, I encounter a problem. <code>(Pdb) p self.norm.norm.running_var tensor([nan, nan, nan,  ..., nan, nan, nan], device='cuda:0') (Pdb) p self.norm.norm.running_mean tensor([nan, nan, nan,  ..., nan, nan, nan], device='cuda:0')</code><br>
Could you please give me some advices to debug this problem?<br>
Thanks  very much.</p>

username: ptrblck
cooked: <p>Running stats can be invalid in case an input activation contained invalid values, such as Infs or NaNs.<br>
You could add checks to your code to make sure the input activation to all batchnorm layers contain finite values only and narrow down which layer is causing the invalid activations.</p>
Why we need softmax with dim=-1 in attention mechanism?

username: omid_khalafbeigi
cooked: <p>Hi there.<br>
As mentioned in <a href="https://arxiv.org/abs/1706.03762" rel="noopener nofollow ugc">Attention Is All You Need</a>, we should apply softmax function on result of (QK/sqrt(dk)) to achieve weights or attention score for each sequence element (like words). It’s unclear for me why we need to apply softmax on columns of feature vectors? I mean, according to PyTorch implementation of <code>multi_head_attention_forward</code> softmax is applied with dim=-1. From what I understood, we need to get attention score for each word using softmax, so we should apply softmax on rows to multiply it by <strong>Value Embedding</strong> of each word. Why is this not happening?<br>
For example in <a href="https://arxiv.org/abs/1512.08756" rel="noopener nofollow ugc">FEED-FORWARD NETWORKS WITH ATTENTION CAN SOVLE SOME LONG-TERM MEMORY PROBLEMS</a>, researchers proposed attention mechanism which apply softmax on hidden states. in other words, softmax is applying on rows of result matrix (each row assumed to be hidden state output).<br>
I appreciate for any guidance.</p>
Is there a layer in torch.nn module to rescale images in the range [0-255]?

username: shrbrh
cooked: <p>Hi, I have a decoder architecture as given:</p>
<pre><code class="lang-auto">     nn.Sequential(
			nn.ReflectionPad2d((1, 1, 1, 1)),
			nn.Conv2d(512, 256, (3, 3)),
			nn.ReLU(),
			nn.Upsample(scale_factor=2, mode='nearest'),
			nn.ReflectionPad2d((1, 1, 1, 1)),
			nn.Conv2d(256, 256, (3, 3)),
			nn.ReLU(),
			nn.ReflectionPad2d((1, 1, 1, 1)),
			nn.Conv2d(256, 256, (3, 3)),
			nn.ReLU(),
			nn.ReflectionPad2d((1, 1, 1, 1)),
			nn.Conv2d(256, 256, (3, 3)),
			nn.ReLU(),
			nn.ReflectionPad2d((1, 1, 1, 1)),
			nn.Conv2d(256, 128, (3, 3)),
			nn.ReLU(),
			nn.Upsample(scale_factor=2, mode='nearest'),
			nn.ReflectionPad2d((1, 1, 1, 1)),
			nn.Conv2d(128, 128, (3, 3)),
			nn.ReLU(),
			nn.ReflectionPad2d((1, 1, 1, 1)),
			nn.Conv2d(128, 64, (3, 3)),
			nn.ReLU(),
			nn.Upsample(scale_factor=2, mode='nearest'),
			nn.ReflectionPad2d((1, 1, 1, 1)),
			nn.Conv2d(64, 64, (3, 3)),
			nn.ReLU(),
			nn.ReflectionPad2d((1, 1, 1, 1)),
			nn.Conv2d(64, 3, (3, 3)),
			nn.Sigmoid()
</code></pre>
<p>I want to re-scale the output of the last nn.Sigmoid() layer in the range [0-255]. Is there a layer in torch.nn module to perform this re-scaling?</p>

username: eqy
cooked: <p>There isn’t a specialized layer to do this, but as sigmoid will already clamp values to [0, 1] you can simply multiply by 255.</p>
I want to install deepspeed, but I run into this error below

username: Hunter_Lee
cooked: <p>Collecting deepspeed<br>
Downloading <a href="http://pypi.doubanio.com/packages/b4/7f/f81e601329fd19c5e7beabb472e0a4cb1ede8292b10df715b261f4d3a16b/deepspeed-0.8.2.tar.gz" rel="noopener nofollow ugc">http://pypi.doubanio.com/packages/b4/7f/f81e601329fd19c5e7beabb472e0a4cb1ede8292b10df715b261f4d3a16b/deepspeed-0.8.2.tar.gz</a> (763 kB)<br>
---------------------------------------- 763.5/763.5 kB 595.5 kB/s eta 0:00:00<br>
Preparing metadata (setup.py) … error<br>
error: subprocess-exited-with-error</p>
<p>× python setup.py egg_info did not run successfully.<br>
│ exit code: 1<br>
╰─&gt; [13 lines of output]<br>
Traceback (most recent call last):<br>
File “”, line 2, in <br>
File “”, line 34, in <br>
File “C:\Users\abc\AppData\Local\Temp\pip-install-va9has0i\deepspeed_6b31b598f2254e2f9f54c795d8aec1b0\setup.py”, line 156, in <br>
abort(f"Unable to pre-compile {op_name}")<br>
File “C:\Users\abc\AppData\Local\Temp\pip-install-va9has0i\deepspeed_6b31b598f2254e2f9f54c795d8aec1b0\setup.py”, line 48, in abort<br>
assert False, msg<br>
AssertionError: Unable to pre-compile async_io<br>
DS_BUILD_OPS=1<br>
e[93m [WARNING] e[0m async_io requires the dev libaio .so object and headers but these were not found.<br>
e[93m [WARNING] e[0m If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.<br>
e[93m [WARNING] e[0m One can disable async_io with DS_BUILD_AIO=0<br>
e[31m [ERROR] e[0m Unable to pre-compile async_io<br>
[end of output]</p>
<p>note: This error originates from a subprocess, and is likely not a problem with pip.<br>
error: metadata-generation-failed</p>
<p>× Encountered error while generating package metadata.<br>
╰─&gt; See above for output.</p>

username: ptrblck
cooked: <p>This seems to be a Windows-specific issue in <code>DeepSpeed</code> and also discussed <a href="https://github.com/microsoft/DeepSpeed/issues/1769">here</a>.</p>
Where to find _fft_c2c_mkl implementation?

username: torchwang
cooked: <p>I would like to know how _fft_c2c_mkl is implemented?</p>
<p>SpectralOps.cpp has only the interface but not the implementation inside, I want to know how it is generated.</p>

username: ptrblck
cooked: <p>Based on <a href="https://github.com/pytorch/pytorch/blob/b5edf183343c8588bfb4105f7545525b644546be/aten/src/ATen/native/mkl/SpectralOps.cpp"><code>SpectralOps.cpp</code></a> this call redirects to <a href="https://github.com/pytorch/pytorch/blob/b5edf183343c8588bfb4105f7545525b644546be/aten/src/ATen/native/mkl/SpectralOps.cpp#L482-L486">MKL methods</a> such as <a href="https://www.intel.com/content/www/us/en/docs/onemkl/developer-reference-fortran/2023-0/dfticomputeforward.html"><code>DftiComputeForward</code></a> or <a href="https://github.com/pytorch/pytorch/blob/b5edf183343c8588bfb4105f7545525b644546be/aten/src/ATen/native/mkl/SpectralOps.cpp#L308-L321"><code>pocketfft</code></a> depending on the CPU architecture and how PyTorch was built.<br>
At least MKL is closed source so you won’t be able to check its source code. <code>PocketFFT</code> seems to be a modification of <code>FFTPack</code> so you might be able to find the source.</p>

username: torchwang
cooked: <p>Thank you so much for your enthusiastic response, I found it and it was indeed just what I needed!</p>
Random Sampler iterator function

username: pararthdave
cooked: <p>I’m trying to customise the RandomSampler subclass to prevent samples below a certain provided loss threshold from going into training. I’m confused about the <code>__iter__()</code> function’s utility and hence unable to make appropriate modifications to it. The <code>__iter__()</code> function from source code looks like this:</p>
<pre><code class="lang-auto">def __iter__(self) -&gt; Iterator[int]:
        n = len(self.data_source)
        if self.generator is None:
            seed = int(torch.empty((), dtype=torch.int64).random_().item())
            generator = torch.Generator()
            generator.manual_seed(seed)
        else:
            generator = self.generator

        if self.replacement:
            for _ in range(self.num_samples // 32):
                yield from torch.randint(high=n, size=(32,), dtype=torch.int64, generator=generator).tolist()
            yield from torch.randint(high=n, size=(self.num_samples % 32,), dtype=torch.int64, generator=generator).tolist()
        else:
            for _ in range(self.num_samples // n):
                yield from torch.randperm(n, generator=generator).tolist()
            yield from torch.randperm(n, generator=generator).tolist()[:self.num_samples % n]
</code></pre>
<p>I was wondering how the iterator function actually works in choosing the samples each time. Also, what’s the utility of having <strong>32</strong> when replacement is true. The use of <strong>yield</strong> functions and how <strong>num_samples</strong> is calculated in this function. What happens when value of num_samples and n is same and how the RandomSampler handles such cases. As in my case, the value of n and num_samples seems to be same which is making it loop over only a single sample one-by-one.</p>
<p>Additionally, I also want to add a validation mode where the forbidden list (samples prevented from going into training) are checked for loss after every epoch and the higher loss samples are sent to training again.</p>

username: nivek
cooked: <ol>
<li>The constant <code>32</code> prevents generating too large of a list at time (it only draws 32 numbers at a time, and draw more if needed), reducing memory usage</li>
<li>The usage of <code>yield</code> creates a generator when the function is called and returns one index at a time</li>
<li>
<code>num_samples</code> is computed within the same file:</li>
</ol>
<pre><code class="lang-python">@property
def num_samples(self) -&gt; int:
    # dataset size might change at runtime
    if self._num_samples is None:
        return len(self.data_source)
    return self._num_samples
</code></pre>

username: pararthdave
cooked: <p>Hi, thanks for responding. I have another query that how <strong>num_samples</strong> differs from <strong>n</strong> in the code I attached with question.</p>

username: nivek
cooked: <p>It simply yields <code>n</code> indices from <code>torch.randperm(n, generator=generator).tolist()</code></p>
Torch.no_grad vs requires_grad = false

username: YoYoYo
cooked: <p>I understand that using torch.no_grad has the benefit of saving memory and computation because the network won’t backpropagate gradients to layers before torch.no_grad.<br>
So, is the following statement correct: If I have a network and only want to update the first and last layers, I cannot simply put all the middle layers inside the context manager torch.no_grad. Instead, I should set “requires_grad = False” in all the middle layers. However, due to the chain rule, the gradients in the middle layers still need to be computed in order to compute the gradient of the first layer. Therefore, the conclusion is that I will not save memory or computation when using “requires_grad = False” compared to using torch.no_grad.</p>

username: soulitzer
cooked: <blockquote>
<p>Instead, I should set “requires_grad = False” in all the middle layers.</p>
</blockquote>
<p>Yes this is the right approach</p>
<blockquote>
<p>However, due to the chain rule, the gradients in the middle layers still need to be computed in order to compute the gradient of the first layer.</p>
</blockquote>
<p>Correct, but not entirely. You will still need to compute gradients wrt the inputs so that the gradients flow to lower layers, but you won’t need to compute gradients wrt all the parameters that you’ve set requires_grad=False to, and that saves computation.</p>
<p>You will also save memory as a result of this. Because depending on which inputs you need to compute gradients, that will change which inputs you will need to save for backward. For example, if I have z = x * y. if I only care about dz/dy, I’d only need to save x.</p>

username: ptrblck
cooked: <p>As a small addition: <a href="https://discuss.pytorch.org/t/for-freezing-certain-layers-why-do-i-need-a-two-step-process/175289/2">this code</a> shows what <a class="mention" href="/u/soulitzer">@soulitzer</a> described.</p>