akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How to get non negative, nonzero min value from tensor?

shashimalcse: <p>I want to get a non-negative, nonzero min value from the tensor. Can someone help me to do this in an easy and efficient way</p>

Sayed_Nadim: <p>Will this work?</p>
<pre><code class="lang-auto">&gt;&gt;&gt; x = (torch.rand((1,1,4,4)) * 2) - 1  # creating a tensor with a mixture of positive and negative value
&gt;&gt;&gt; x
tensor([[[[ 0.8101, -0.5538,  0.2049, -0.8969],
          [-0.5394, -0.8587, -0.5657,  0.0481],
          [-0.9651,  0.8118, -0.3242, -0.5439],
          [ 0.0358, -0.1856, -0.1928,  0.7507]]]])
&gt;&gt;&gt; min_x = torch.min(x[x&gt;0]) # taking non-negative non-zero min value from the tensor
&gt;&gt;&gt; min_x
tensor(0.0358)

</code></pre>

markl: <p>I discovered a trick for this which avoids construction of a mask.</p>
<pre><code class="lang-auto"># how to determine the smallest positive value in the tensor x

# by shifting and taking inverse, we accomplish the following
# - zero maps to "negative infinity"
# - small positive numbers go to large positive numbers
# - negative numbers remain negative
small_shift = 0.001
invs = 1.0/(x - small_shift)

# undo the mapping
min_value = 1.0/invs.max().values() + small_shift
</code></pre>
Verbose in dassl

kendal: <p>I had to download new version of pytorch because of the Slurm server’s limitation.<br>
Then I planed to use dassl to train a model, but got warning “UserWarning: The verbose parameter is deprecated” and it seems that the config VERBOSE have been deprecated in recent versions of Pytorch.<br>
I’m a rookie of coding in Machine Learning so I have to send this topic for help.<br>
I’d be appreciated for your answers!!!</p>

ptrblck: <p>I’m not familiar with <code>dassl</code> but could you check which line of code raises this warning?</p>

kendal: <p>torch/optim/lr_scheduler.py:62: UserWarning: The verbose parameter is deprecated. Please use get_last_lr() to access the learning rate.</p>

ptrblck: <p>Thank you!<br>
It seems the <code>verbose</code> argument is directly passed to all learning rate schedulers as seen e.g. <a href="https://github.com/KaiyangZhou/Dassl.pytorch/blob/c61a1b570ac6333bd50fb5ae06aea59002fb20bb/dassl/optim/lr_scheduler.py#L22">here</a>. I don’t know if <code>dassl</code> is still maintained, but you could create an issue in their GitHub repo or submit a fix.</p>
How to run PyTorch DDP with Hydra + Optuna?

deepakpokkalla: <p>I am trying to use Hydra + Optuna Sweeper with my PyTorch + DDP (mp.spawn) setup. I have the PyTorch + DDP running properly. However, when I’m trying to run the hydra sweeper, it;s not working - as in, it stops after just one case. So, I am wondering how to make it work and tune the hyperparameters?</p>
<p>In the below, the main function is launched by the manual_proc_runner spawned across GPUs.</p>
<pre><code class="lang-auto">def manual_proc_runner(rank, cfg, m, results): # for current process
    world_size = m.get_world_size() # 2
    print('world_size:',world_size)

    manual_proc_setup(rank=rank, world_size=world_size)

    # ----- DDP Manual Env Setup -----
    set_seed(seed=42)
    env = CustomManualEnvironment(world_size=world_size, rank=rank)
    
    # main(m=m, env=env, args=args)
    loss = main(cfg=cfg, m=m, env=env)

    results[rank] = loss
    print(f'results:{results} in rank:{rank}')

    # clean up DDP process group after completion
    manual_proc_cleanup()

@hydra.main(version_base=None, config_path='../config_files', config_name='config')
def hydra_manual_proc_launcher(cfg): # no arguments other than 'cfg' since hydra
    m = ModelConfig('./config_files/drivaer_domains_ddp.ini')
    world_size = m.get_world_size() # 2
    print('world_size:',world_size)

    out_dir = hydra.core.hydra_config.HydraConfig.get().runtime.output_dir
    print(f'out_dir:{out_dir}')

    with mp.Manager() as manager:
        results = manager.list([None]* world_size)
        
        mp.spawn(manual_proc_runner, args=(cfg,m,results), nprocs=world_size, join=True)
        
        print(f'results:{results}')
        aggregated_loss = sum(filter(None,results)) / len(results)
        print(f'Aggregated loss: {aggregated_loss}')

        return aggregated_loss

def main(cfg, m, env):
</code></pre>
<p>This is what I have, along with the following config file. When I run this, default hyper-parameters are chosen and then it stops instead of running the whole optimization loop.</p>
<pre><code class="lang-auto">defaults:
  - override hydra/sweeper: optuna
  - override hydra/sweeper/sampler: tpe

storage: 'sqlite:///../output/train/checkpoints/subdomains_211_v2/optuna_dashboard.db'

hydra:
  sweeper:
    sampler:
      seed: 123
    direction: minimize
    study_name: hydra-1
    # storage: null
    n_trials: 4 #50
    n_jobs: 1
    params:
      mp_iterations: range(1, 15)
      mlp_dim: range(64,256)
      mlp_layers: range(1,4)

mp_iterations: 15
mlp_dim: 64
mlp_layers: 2

# if true, simulate a failure by raising an exception
error: false
</code></pre>
`torch.linalg.svd` uses `cudaMemcpyAsync` that syncs between host and device

Rui_Wang: <p>As I understand and profile, I believe it might perform some checks on the numerical issue like <a href="https://pytorch.org/docs/stable/generated/torch.linalg.cholesky.html#torch.linalg.cholesky" rel="noopener nofollow ugc">cholesky</a>, but it does not have a <code>_ex</code> variant that does not sync.</p>
<p>Do we have a solution to this?</p>
How to use FSDP with LoRA?

Vedant_Roy: <p>I am trying to use Microsoft’s loralib: <a href="https://github.com/microsoft/LoRA" class="inline-onebox" rel="noopener nofollow ugc">GitHub - microsoft/LoRA: Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"</a> inside of an FSDP-wrapped model. However, this does not work because the parameter tensors: <a href="https://github.com/microsoft/LoRA/blob/4c0333854cb905966f8cc4e9a74068c1e507c7b7/loralib/layers.py#L110" class="inline-onebox" rel="noopener nofollow ugc">LoRA/loralib/layers.py at 4c0333854cb905966f8cc4e9a74068c1e507c7b7 · microsoft/LoRA · GitHub</a>, get sharded, which causes issues when computing the LoRA update: <a href="https://github.com/microsoft/LoRA/blob/4c0333854cb905966f8cc4e9a74068c1e507c7b7/loralib/layers.py#L208" class="inline-onebox" rel="noopener nofollow ugc">LoRA/loralib/layers.py at 4c0333854cb905966f8cc4e9a74068c1e507c7b7 · microsoft/LoRA · GitHub</a>.</p>
<p>Ideally – I would like to avoid using FSDP on the matrices that are being lora tuned (in this case, the q/k/v projections in a transformer).</p>
<p>How can I do this?</p>

tianyu: <p><a class="mention" href="/u/weifengpy">@weifengpy</a> may be the right person to help</p>

weifengpy: <blockquote>
<p>Ideally – I would like to avoid using FSDP on the matrices that are being lora tuned</p>
</blockquote>
<p>is this from torch.distributed.fsdp.FullyShardedDataParalle? If yes, you can specify <code>ignored_states</code> to not shard qkv projections. example code: <a href="https://github.com/pytorch/pytorch/blob/dbe6fce185afc3a59f7a2063289c809f39d36c32/test/distributed/fsdp/test_fsdp_ignored_modules.py#L292-L303" class="inline-onebox" rel="noopener nofollow ugc">pytorch/test/distributed/fsdp/test_fsdp_ignored_modules.py at dbe6fce185afc3a59f7a2063289c809f39d36c32 · pytorch/pytorch · GitHub</a></p>
<p>I am also curious about your wrapping policy. if wrapping transformer layer, FSDP should unshard <code>layer.parameters()</code> before <code>layer.forward</code>. Technically it’s ok to shard qkv with lora adapters. Here is an example of LoRA + FSDP2 in TorchTune: <a href="https://github.com/pytorch/torchtune/blob/main/README.md" class="inline-onebox" rel="noopener nofollow ugc">torchtune/README.md at main · pytorch/torchtune · GitHub</a></p>
LSTM incompatible with vmap? "accessing '.data' under vmap transform is not allowed"

Clion: <p>Been trying to figure out this problem for quite some time now and it’s proving quite intractable, and searches on the error return no results…apparently a very rare problem, as it seems no one anywhere has ever posted anything about this particular error message (at least that I’ve been able to find).</p>
<p>Basically the problem is that a particular step in executing the forward pass through an LSTM module seems to be fundamentally incompatible with Torch’s vmap apparatus.</p>
<p>A full description of the problem with a minimal reproducible case and full stack trace can be found here: <a href="https://github.com/pytorch/pytorch/issues/105982" class="inline-onebox" rel="noopener nofollow ugc">vmap and rnn/lstm "accessing '.data' under vmap transform is not allowed" · Issue #105982 · pytorch/pytorch · GitHub</a></p>
<p>Any takers? This problem has proven intractable for me for months at this point.</p>

gregarityNow: <p>I’m having the same issue, did you ever solve this?</p>
Fasterrcnn - Rotation augmentation makes model not converging

stendus: <p>I am fine-tuning a Faster R-CNN model for object detection on a custom dataset. The model is declared as follows:</p>
<pre><code class="lang-auto">model = fasterrcnn_resnet50_fpn(weights="DEFAULT")
in_features = model.roi_heads.box_predictor.cls_score.in_features
model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)
</code></pre>
<p>Due to the nature of my data, I decided to add an augmentation to rotate the image by 180 degrees. I added the following transform:</p>
<pre><code class="lang-auto">transforms.append(T.RandomApply(torch.nn.ModuleList([T.RandomRotation(degrees=(180, 180), expand=1)]), p=0.5)
</code></pre>
<p>However, using a probability of 0.5 led to oscillating training performance, not reaching the baseline results on the same test set. Increasing the probability to 1 made it even worse.</p>
<p>I have checked the images passed through the data loader, and the bounding boxes are correct after the augmentation step.</p>
<p>It seems like the model is receiving incorrect data and training on random coordinates. What am I missing?</p>
Complex derivative of real-to-real function

sadda: <p>Hello, can anyone please point out what I do wrong in the computation of the following two functions? Both <code>f1</code> and <code>f2</code> should be identical and real-to-real functions and therefore should give <a href="https://pytorch.org/docs/stable/notes/autograd.html#complex-autograd-doc" rel="noopener nofollow ugc">the same derivative</a>. However, they give a different output <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=12" title=":frowning:" class="emoji" alt=":frowning:" loading="lazy" width="20" height="20"> Thanks a lot.</p>
<pre><code class="lang-auto">import torch

def f1(x):
    return torch.view_as_complex(torch.Tensor([x, 0])).unsqueeze(0).abs()

def f2(x):
    return torch.Tensor(x).abs()

x = torch.Tensor([1])
print(torch.autograd.functional.jacobian(f1, x))
print(torch.autograd.functional.jacobian(f2, x))
</code></pre>
<p>outputs</p>
<pre><code class="lang-auto">tensor([[0.]])
tensor([[1.]])
</code></pre>

sadda: <p>Problem solved by replacing <code>f1</code> by</p>
<pre><code class="lang-auto">def f1(x):
    return torch.view_as_complex(torch.hstack((x,torch.zeros_like(x)))).unsqueeze(0).abs()
</code></pre>
Indexing a large tensor is very slow

gradientlord: <p>Hey all, we have two tensors. The first is of size <code>(S1, N)</code> and the other is <code>(N, S2)</code>. Both are considerably large matrices, with <code>S1 =  ~200k</code>, <code>N = 10000</code> and <code>S2 = ~200k</code><br>
Requirement is to do dot product of a few rows (a couple 10s) of the first tensor with the complete second tensor. The few rows are given by a mask, <code>mask</code>. And there are many such masks for which we need to calculate the dot product.</p>
<p>Right now, if we’re doing.</p>
<pre><code class="lang-auto">for mask in masks:
    t = S1[mask]
    result = torch.matmul(t, S2)
</code></pre>
<p>But it’s very slow.<br>
We found that indexing through a tensor actually copies the result as opposed to indexing through an integer or a slice which returns a view.</p>
<p>We guess that <code>S1[mask]</code> creates another tensor which seems to be the bottleneck since <code>S1</code> is a pretty large matrix.</p>
<p>What are our options?<br>
Sparse matrices? Writing custom CUDA code? Any other way?</p>
<p>Thanks!</p>

ptrblck: <p>I don’t know if the copy is the actual bottleneck or the synchronization if a <code>BoolTensor</code> is used as a mask since you are adding data-dependent logic to your code (the output shape depends on the content of the <code>mask</code>, not its metadata).<br>
You could try to replace the <code>mask</code> with an index tensor and could then profile your code again to ee if these copies are actually so expensive as you think.</p>

GF-Huang: <p>Hi bro, did you solve this problem?</p>
What's the theoreticl basis of torch.testing tolerance table?

Jiexin-Zheng: <p>Hi, Pytorch Team, could you pls tell me how did you get this tolerance table? what’s the theoreticl basis of torch.testing tolerance table?<br>
<a href="https://pytorch.org/docs/stable/testing.html#torch.testing.assert_close" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/testing.html#torch.testing.assert_close</a><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/0/704c9ddf26dc72ffe8f69312a123846fdab78ded.png" data-download-href="https://discuss.pytorch.org/uploads/default/704c9ddf26dc72ffe8f69312a123846fdab78ded" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/704c9ddf26dc72ffe8f69312a123846fdab78ded_2_690x496.png" alt="image" data-base62-sha1="g1rCSTppCsZm5PVTt2nKuq79UUl" width="690" height="496" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/704c9ddf26dc72ffe8f69312a123846fdab78ded_2_690x496.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/704c9ddf26dc72ffe8f69312a123846fdab78ded_2_1035x744.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/0/704c9ddf26dc72ffe8f69312a123846fdab78ded.png 2x" data-dominant-color="FAF9FA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1098×790 24 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
How to extract figures with labels from image

stunning_World: <p>Here is the image, from that i want to extract the figure with labels<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/8/3892b62f3a6f8e8d592d805f24dabb19c0533d44.png" data-download-href="https://discuss.pytorch.org/uploads/default/3892b62f3a6f8e8d592d805f24dabb19c0533d44" title="page_11 (1)"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/8/3892b62f3a6f8e8d592d805f24dabb19c0533d44_2_353x500.png" alt="page_11 (1)" data-base62-sha1="84t4868HkLByptrUJDVV12NJf3C" width="353" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/8/3892b62f3a6f8e8d592d805f24dabb19c0533d44_2_353x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/8/3892b62f3a6f8e8d592d805f24dabb19c0533d44_2_529x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/8/3892b62f3a6f8e8d592d805f24dabb19c0533d44_2_706x1000.png 2x" data-dominant-color="F9F9F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">page_11 (1)</span><span class="informations">1655×2340 124 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Has no gradient in bias

olegkufa: <p>I wrote a layer in the EMA style for v at attention, but the gradient does not pass through the bias and I don’t understand what the problem is, since everything is fine according to the weights.<br>
Can someone tell me what’s the problem</p>
<pre><code class="lang-auto">class ValueResidualLayer(nn.Module):
    def __init__(self, dim, n_heads):
        super().__init__()
        self.to_gates = nn.Linear(dim, n_heads, bias=True)
        self.act = nn.Sigmoid()
        self.h = n_heads
        

    def forward(self, x, v, res):
        
        gates = self.act(self.to_gates(x))
        
        gates = rearrange(gates, 'b n (h d) -&gt; b h n d', h=self.h)

        out =  v * gates + (1 - gates) * res

        return out
</code></pre>

ptrblck: <p>I don’t know which inputs you are using but testing any random tensors works for me:</p>
<pre data-code-wrap="python"><code class="lang-python">model = ValueResidualLayer(10, 10)
x = torch.randn(1, 10, 10)
v = torch.randn_like(x)
res = torch.randn_like(x)
out = model(x, v, res)
out.mean().backward()

print({name: p.grad for name, p in model.named_parameters()})
# {'to_gates.weight': tensor([[ 3.3208e-03,  1.7235e-03, -9.4052e-03,  2.4368e-03, -2.5074e-03,
#          -1.6132e-03,  3.9941e-03, -5.1434e-03,  3.4118e-03, -3.5914e-03],
#         [ 3.3302e-03,  2.2675e-03, -1.0479e-02,  2.4576e-03, -1.9038e-03,
#          -5.3038e-04,  3.9044e-03, -4.5693e-03,  4.7626e-03, -3.5518e-03],
#         [ 2.9834e-03,  2.3057e-03, -9.5416e-03,  2.0493e-03, -2.2837e-03,
#          -7.6640e-04,  3.9544e-03, -4.7697e-03,  4.6088e-03, -3.2332e-03],
#         [ 2.1757e-03,  2.4342e-03, -8.1161e-03,  1.5353e-03, -2.5703e-03,
#          -2.7348e-04,  3.4181e-03, -4.1390e-03,  5.2883e-03, -2.5688e-03],
#         [ 2.2552e-03,  2.4213e-03, -8.6571e-03,  1.8172e-03, -2.0862e-03,
#          -4.0808e-04,  3.5837e-03, -4.1661e-03,  5.4443e-03, -2.3937e-03],
#         [ 3.3070e-03,  2.4589e-03, -1.0083e-02,  2.0671e-03, -2.2453e-03,
#          -1.7378e-04,  4.0691e-03, -4.6217e-03,  4.6735e-03, -3.8617e-03],
#         [ 2.7600e-03,  3.3665e-03, -9.4790e-03,  2.0118e-03, -3.1184e-03,
#           1.3336e-04,  4.0494e-03, -4.4689e-03,  4.7932e-03, -4.1376e-03],
#         [ 3.3488e-03,  2.2973e-03, -9.7965e-03,  2.0508e-03, -2.2581e-03,
#          -5.0712e-04,  4.0553e-03, -4.8201e-03,  4.1354e-03, -3.8284e-03],
#         [ 3.3092e-03,  2.2325e-03, -9.7376e-03,  1.8958e-03, -1.5461e-03,
#           1.4049e-04,  4.0094e-03, -4.3248e-03,  4.7798e-03, -3.5617e-03],
#         [ 3.4868e-03,  1.9710e-03, -9.9802e-03,  2.1578e-03, -8.9903e-04,
#           8.1139e-05,  3.9524e-03, -3.9340e-03,  4.4010e-03, -3.4717e-03]]), 'to_gates.bias': tensor([0.0014, 0.0006, 0.0014, 0.0010, 0.0015, 0.0006, 0.0005, 0.0011, 0.0007,
#         0.0002])}
</code></pre>
<p>and shows valid gradients in the <code>weight</code> and <code>bias</code>.</p>
How to resolve with the issue of adjusting the gaps in my 'time_idx' feature in my stock market dataset?

Satyam_sharma: <p>allow_missing_timesteps (bool): if to allow missing timesteps that are automatically filled up. Missing values refer to gaps in the <code>time_idx</code>, e.g. if a specific timeseries has only samples for 1, 2, 4, 5, the sample for 3 will be generated on-the-fly.Allow missings does not deal with <code>NA</code> values. You should fill NA values before passing the dataframe to the TimeSeriesDataSet.</p>
<p>ok I guess i can use this parameter for my gap problem,<br>
but how will it help me what data will it take,<br>
and will it actually going to reduce my past sequence length?<br>
will this artificially filled gap values be considered in the max_encoder_length ?</p>
<p>please suggest me the approach (a consistent approach, throughout the process from training the model till live real data prediction) .<br>
since when i will be seeking the next hour prediction value of NIFTY(Indian stock exchange) , then I have to give the new time_idx values (never seen by model)</p>
Numpy is not avaiable when transfrom Torch tensor to numpy

miraboreasu: <p>I have learned that Numpy 2 will cause such problem, so I have<br>
Pytorch 2.5.1<br>
Numpy 1.26.4<br>
Python 3.12.4</p>
<p>It has error when I try to use</p>
<pre><code class="lang-auto">.numpy()

</code></pre>

ptrblck: <p>What kind of error are you seeing?</p>
Struggling to understand how mAP is calculated

Soumya_Kundu: <pre><code class="lang-auto">from torch import tensor
from torchmetrics.detection import MeanAveragePrecision
preds = [
  dict(
    boxes=tensor([[215.0, 41.0, 562.0, 285.0], [214.0, 41.0, 562.0, 285.0]]),
    scores=tensor([0.99, 0.09]),
    labels=tensor([0, 1]),
  )
]
target = [
  dict(
    boxes=tensor([[214.0, 41.0, 562.0, 285.0]]),
    labels=tensor([0]),
  )
]
metric = MeanAveragePrecision(iou_type="bbox", class_metrics=True, iou_thresholds=[0.5, 0.75], average="micro")
metric.update(preds, target)
from pprint import pprint
pprint(metric.compute())
</code></pre>
<p>This code gives:</p>
<pre><code class="lang-auto">{'classes': tensor([0, 1], dtype=torch.int32),
 'map': tensor(1.),
 'map_50': tensor(1.),
 'map_75': tensor(1.),
 'map_large': tensor(1.),
 'map_medium': tensor(-1.),
 'map_per_class': tensor([ 1., -1.]),
 'map_small': tensor(-1.),
 'mar_1': tensor(1.),
 'mar_10': tensor(1.),
 'mar_100': tensor(1.),
 'mar_100_per_class': tensor([ 1., -1.]),
 'mar_large': tensor(1.),
 'mar_medium': tensor(-1.),
 'mar_small': tensor(-1.)}
</code></pre>
<p>But according to my understanding, should it not give 0.5?</p>
<p>I am simulating two preds masks of 2 seperate classes having 100% overlap with one of the classes.</p>

Soumya_Kundu: <p>Another Case which should give 0, notice no classes are matched.</p>
<pre><code class="lang-auto">from torch import tensor
from torchmetrics.detection import MeanAveragePrecision
preds = [
  dict(
    boxes=tensor([[215.0, 41.0, 562.0, 285.0], [214.0, 41.0, 562.0, 285.0]]),
    scores=tensor([0.99, 0.09]),
    labels=tensor([6, 7]),
  )
]
target = [
  dict(
    boxes=tensor([[214.0, 41.0, 562.0, 285.0]]),
    labels=tensor([0]),
  )
]
metric = MeanAveragePrecision(iou_type="bbox", class_metrics=True, iou_thresholds=[0.5, 0.75], average="micro")
metric.update(preds, target)
from pprint import pprint
pprint(metric.compute())
</code></pre>
<p>Gives:</p>
<pre><code class="lang-auto">{'classes': tensor([0, 6, 7], dtype=torch.int32),
 'map': tensor(1.),
 'map_50': tensor(1.),
 'map_75': tensor(1.),
 'map_large': tensor(1.),
 'map_medium': tensor(-1.),
 'map_per_class': tensor([ 0., -1., -1.]),
 'map_small': tensor(-1.),
 'mar_1': tensor(1.),
 'mar_10': tensor(1.),
 'mar_100': tensor(1.),
 'mar_100_per_class': tensor([ 0., -1., -1.]),
 'mar_large': tensor(1.),
 'mar_medium': tensor(-1.),
 'mar_small': tensor(-1.)}
</code></pre>
Extracting Individual Tensors from a PyTorch .pt File for Custom CUDA/C++ Inference

Saad_Amin: <p>Hello,</p>
<p>I am training a model in PyTorch and would like to load it into a custom CUDA/C++ library I’ve written for fast inference. I am aware of solutions that use libtorch to load and run pre-trained PyTorch models in C++ (e.g., <a href="https://discuss.pytorch.org/t/is-there-a-way-to-read-pt-files-in-c/34392">this discussion</a>). However, my use case involves bypassing libtorch and directly extracting the individual tensors from the <code>.pt</code> file to load them into my custom library.</p>
<p>Does a library or utility exist that can help me parse a <code>.pt</code> file to extract its tensor data, or would I need to implement this functionality manually? Any guidance or references would be greatly appreciated.</p>
<p>Thank you!</p>
RuntimeError: unable to mmap 128 bytes from file </torch_31439_2312018605_63322>: Cannot allocate memory (12)

111466: <p>I ran into a problem when using the torch multiprocessing library. The error message is as follows:</p>
<p>I’ve changed the num_workers parameter in dataloader to 0, but it doesn’t solve it.<br>
I have monitored the memory of the host machine and it is sufficient.<br>
How can I solve it?</p>
<pre><code class="lang-auto">import torch
import torch.multiprocessing
torch.multiprocessing.set_sharing_strategy('file_system')#
</code></pre>
<pre><code class="lang-auto">
Traceback (most recent call last):
  File "run.py", line 53, in &lt;module&gt;
    main()
  File run.py", line 49, in main
    slam.run()
  File /NICE_SLAM.py", line 304, in run
    p.start()
  File anaconda3/envs/nice-slam/lib/python3.7/multiprocessing/process.py", line 112, in start
    self._popen = self._Popen(self)
  Fileanaconda3/envs/nice-slam/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  Fileanaconda3/envs/nice-slam/lib/python3.7/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File anaconda3/envs/nice-slam/lib/python3.7/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File anaconda3/envs/nice-slam/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
  File anaconda3/envs/nice-slam/lib/python3.7/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File anaconda3/envs/nice-slam/lib/python3.7/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
  File anaconda3/envs/nice-slam/lib/python3.7/site-packages/torch/multiprocessing/reductions.py", line 338, in reduce_storage
    metadata = storage._share_filename_()
RuntimeError: unable to mmap 128 bytes from file &lt;/torch_31439_2312018605_63322&gt;: Cannot allocate memory (12)
[W CudaIPCTypes.cpp:15] Producer process has been terminated before all shared CUDA tensors released. See Note [Sharing CUDA tensors]

Process finished with exit code 1
</code></pre>

HELLORPG: <p>Hi, have you solved this issue? It seems like I encounter the same problem.</p>
Mask value in the range 0-255

Kelvin: <p>Hi, I would like to ask about image segmentation. I am currently working on multi-class segmentation. However, when I check the unique value of the mask using <code>np.unique(mask)</code>, I get a value of 0-255 instead of the class index (0-34). This also happens when I am working on binary segmentation. Can anyone explain this, why this happens and what should I do to convert the 0-255 value to a class index. Thanks.</p>
<pre><code class="lang-auto">class CityScapesDataset(Dataset):
    def __init__(self, dir_path:str, dir_array: List, transform=None):
        self.dir_path = dir_path
        self.images_mask_array = dir_array
        self.transform = transform

    def __len__(self):
        return len(self.images_mask_array)

    def __getitem__(self, idx):
        image_mask = self.images_mask_array[idx]
        image_mask_path = os.path.join(self.dir_path, image_mask)
        image_mask_read = cv2.imread(image_mask_path, cv2.IMREAD_COLOR)
        image_mask_read = cv2.cvtColor(image_mask_read, cv2.COLOR_BGR2RGB)

        image = image_mask_read[:, :256, :]
        image = np.transpose(image, (2, 0, 1))

        mask = image_mask_read[:, 256:, :]
        mask = np.transpose(mask, (2, 0, 1))
        
        return image, mask
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/a/5a2be397bcc6af01e714b40e5e55b3fa3ca91eda.png" data-download-href="https://discuss.pytorch.org/uploads/default/5a2be397bcc6af01e714b40e5e55b3fa3ca91eda" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/a/5a2be397bcc6af01e714b40e5e55b3fa3ca91eda.png" alt="image" data-base62-sha1="cRH0fZYWclGppJ20MAGwKmUCthU" width="690" height="401" data-dominant-color="F1F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">836×486 18.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Why does the inductor reduction Triton Codegen use the Welford algorithm instead of the Naive?

Rrrrrice: <p>In my AMD GPU testing, I replaced the Welford algorithm in the batchnorm operator with the Native algorithm to generate Triton DSL, and achieved better performance in some cases. At the same time, I found that compared to Welford, an additional iteration count variable needs to be maintained, and Native uses fewer registers in a kernel, which helps to achieve better parallelism. I am confused as to why Conductor defaults to using Welford algorithm? The formula for Native is as follows:</p>
<pre><code class="lang-auto">D(x) = E(x^2) - (E(x))^2
</code></pre>

Rrrrrice: <p><span class="mention">@peterbell10</span> Excuse me, can you help me take a look at this <img src="https://discuss.pytorch.org/images/emoji/apple/grinning.png?v=12" title=":grinning:" class="emoji" alt=":grinning:" loading="lazy" width="20" height="20">？</p>
Integrating the Laplacian of a Neural Network

Giorgio_c: <p>Hi everybody,<br>
I’m struggling with the integration of a neural network’s Laplacian. The problem is related to the speed of the training.<br>
In particular, i’ve implemented an hp-vPINN(Partial Differential Equation solver), which requires to do a refinement of the domain. In my case, the domain is a simply square [-1,1]^2. I divided the domain into 9 different squares and, for every square, i had to integrate the laplacian of the neural network.<br>
I’ve implemented a GaussianLegendre quadrature function in 2D, which is currently working but the training is really too slow. Any tips?</p>

Grant_Norman: <p>It’s hard to provide more specific recommendations without seeing your code. To begin, does your code use <em>batching</em> for parts of the code that are repeated? In other words, is the computation of the function at the quadrature points done with a for loop, or handled with batched operations?</p>
Stft behavior with center=False and n_fft > win_length

nedlrichards: <p>I am implimenting a stft library that uses torch under the hood but uses the parameter conventions of scipy’s ShortTimeFFT. I am having trouble understanding the torch behavior when the stft fft is zero padded and center=False. My basic expectation is zero padding should only change the frequency axis of the output, but the time axis should not change because the data sampling (window and hop) has remained the same.</p>
<p>When center=True this holds, but doesn’t when center=False. Is this behavior expected? Any help is appreciated.</p>
<p>Expect True:</p>
<pre><code class="lang-auto">timeseries = torch.randn((8, 2300))

# OK
torch.stft(timeseries, 200, hop_length=100, win_length=200, window=torch.ones(200), return_complex=True).shape[2] == torch.stft(timeseries, 400, hop_length=100, win_length=200, window=torch.ones(200), return_complex=True).shape[2]

# False
torch.stft(timeseries, 200, hop_length=100, win_length=200, window=torch.ones(200), return_complex=True, center=False).shape[2] == torch.stft(timeseries, 400, hop_length=100, win_length=200, window=torch.ones(200), return_complex=True, center=False).shape[2]
</code></pre>
Scaling of pytorch profiler with large number of nodes

HariSeldon11988: <p>Hey there,</p>
<p>I’m using the pytorch profiler (<code>profiler.step()</code>) function to analyze my code for different quite large Models on 1,2,4,8,16 nodes each 8 GPUs. I store the data for the traces in json files and use my own parser and additionally tensorboard to evaluate the files.</p>
<p>I got strange behaviours and exploding runtimes for 8 and especially 16 nodes. The profiler stores the trace for each GPU so if I have 64 GPUs I have 64 json files and average the times. I run the test multiple times.</p>
<p>Is it possible that the profiler scales not very well using a large number of GPUs? I doubt it to be honest, but I’m not sure and searching for explanations for this behaviour.</p>
<p>Would appreciate any kind of help!</p>