akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Quickstart tutorial: __array__() takes 1 positional argument but 2 were given

GMSL: <p>I’ve tried the <a href="https://pytorch.org/tutorials/beginner/basics/quickstart_tutorial.html" rel="noopener nofollow ugc">quickstart tutorial</a> on two machines - one without a CUDA-compatible GPU running PyTorch 1.9.0-CPU, and the other with CUDA-compatible GPU running PyTorch 1.9.0+cu111. For the first one (with CPU), the tutorial works fine, however with the second one (with CUDA) it fails in the first section.</p>
<p>I copied and pasted the code in directly from the tutorial:</p>
<pre><code class="lang-auto">import torch
from torch import nn
from torch.utils.data import DataLoader
from torchvision import datasets
from torchvision.transforms import ToTensor, Lambda, Compose
import matplotlib.pyplot as plt

# Download training data from open datasets.
training_data = datasets.FashionMNIST(
    root="data",
    train=True,
    download=True,
    transform=ToTensor(),
)

# Download test data from open datasets.
test_data = datasets.FashionMNIST(
    root="data",
    train=False,
    download=True,
    transform=ToTensor(),
)

batch_size = 64

# Create data loaders.
train_dataloader = DataLoader(training_data, batch_size=batch_size)
test_dataloader = DataLoader(test_data, batch_size=batch_size)

for X, y in test_dataloader:
    print("Shape of X [N, C, H, W]: ", X.shape)
    print("Shape of y: ", y.shape, y.dtype)
    break
</code></pre>
<p>Running this gives me the error:</p>
<pre><code class="lang-auto">TypeError: __array__() takes 1 positional argument but 2 were given
</code></pre>
<p>It’s the for loop that causes the error, it runs fine if that for loop is commented out. Looking on this forum and google, it seems a common reason is ommitting the <strong>init</strong> method from a function, but obviously I haven’t defined any functions yet in this tutorial.</p>
<p>Thanks for any help.</p>

ptrblck: <p>The issue might be related to <code>PIL==8.3.0</code> as described <a href="https://github.com/pytorch/pytorch/issues/61125">here</a> and <a class="mention" href="/u/tom">@tom</a> already provided a fix <a href="https://github.com/python-pillow/Pillow/pull/5572">here</a>.<br>
If you are also using <code>PIL==8.3.0</code>, downgrade it to <code>8.2.0</code> for now. If that’s not the case, please let us know and we’ll take another look into your issue.</p>

Aimlessboy_Yuan: <p>Thanks，this solution does work！ <img src="https://discuss.pytorch.org/images/emoji/apple/+1.png?v=9" title=":+1:" class="emoji" alt=":+1:"> <img src="https://discuss.pytorch.org/images/emoji/apple/+1.png?v=9" title=":+1:" class="emoji" alt=":+1:"> <img src="https://discuss.pytorch.org/images/emoji/apple/+1.png?v=9" title=":+1:" class="emoji" alt=":+1:"></p>

GMSL: <p>This worked thank you. Just to confirm, it was <code>pillow</code> that I had to downgrade to 8.2.0, which is a fork of <code>PIL</code>.</p>

HackerCop: <p>This worked thanks pip uninstall pillow and pip install pillow==8.2.0 solved the problem</p>
Is it possible to hijack the storage of data needed in backward pass

Anders_G: <p>Is it possible to take of over the storage of the data used in the back propagation? For instance if I have little memory on the GPU and want to store it on CPU RAM and then move it back when it is needed in during back propagation.</p>

albanD: <p>Hi,</p>
<p>If you’re using nightly or in the upcoming 1.10, you will be able to register such hooks directly: <a href="https://pytorch.org/docs/master/notes/autograd.html#hooks-for-saved-tensors" class="inline-onebox">Autograd mechanics — PyTorch master documentation</a></p>
<p>And if you just want to move everything from GPU to CPU, we have a builtin here: <a href="https://pytorch.org/docs/master/autograd.html?highlight=cpu#torch.autograd.graph.save_on_cpu" class="inline-onebox">Automatic differentiation package - torch.autograd — PyTorch master documentation</a></p>

Anders_G: <p>Thank you it was exactly what I was looking for!</p>
Efficient way to divide samples' gradient contribution

Torcione: <p>Hi everyone,<br>
I have a classification problem that take an image dataset in input (Cifar10 for example); given a generic batch of the train_loader, which is the most efficient way to get back one gradient vector associated to each class in the batch (or to each sample of the batch)?<br>
I can do it ,for example, iterating over the samples of the batch computing the following steps:</p>
<pre><code class="lang-auto">output = model(data[i])
loss = criterion(output,label[i])
loss.backward()
</code></pre>
<p>(where is an index associated to a single sample of the batch) and finally copy the gradient associated to the specific sample into a second variable.</p>
<p>This approach seems very inefficient since the samples  of the batch are not computed in parallel but in sequence, one by one; is it possible to do it in a more clever way?</p>
How to input more than one image into a model

Trevor_Blythe: <p>I have images in a folder called 2. I want to train a model to output images from folder 3 when input images from folder 2.</p>
<p>For this, to work I have to train in batches or train on the whole set at once!</p>
<p>But how can I input more than one image into the model to train???</p>
<p>I looked everywhere and IDK how-to! I’ve tried making a tensor of tensors of the full data and doing<br>
model(data) but uh oh it doesn’t work cuz it’s not the right dimensions.</p>

eqy: <p>The MNIST example gives a relatively short example of batched training:<br>
<a href="https://github.com/pytorch/examples/blob/master/mnist/main.py" rel="noopener nofollow ugc">examples/main.py at master · pytorch/examples (github.com)</a>, primarily through the use of a <code>DataLoader</code>: <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader" rel="noopener nofollow ugc">torch.utils.data — PyTorch 1.9.1 documentation</a>.</p>

Trevor_Blythe: <p>what is data in the train function??</p>
Multi binary-class classification

lima: <p>Hello peeps,</p>
<p>I have some audio data for which i computed the audio features. The labels for each input feature is 20 binary classes. i.e. for each input element there are 20 classes (1 or 0).<br>
My initial understanding is that  this is a multi-label classification that can be addressed using <code>nn.BCELoss</code>.</p>
<p>Can you guys let me know if I got this right and to explain how <code>nn.BCELoss</code> works?</p>
<p>Thank you</p>

nateanl: <p>Hi <a class="mention" href="/u/lima">@lima</a>, <a href="https://pytorch.org/docs/stable/generated/torch.nn.BCELoss.html" rel="noopener nofollow ugc"><code>nn.BCELoss</code></a> is designed for binary classification task. The prediction and label are both of shape <code>(batch, ...)</code>.<br>
In your case, you have 20 classes, which is a multi-class classification task. You can use <a href="https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss" rel="noopener nofollow ugc"><code>nn.CrossEntropyLoss</code></a> where the prediction is a float tensor of shape <code>(batch, class, ...)</code> and the label is a long tensor of shape <code>(batch, ...)</code>.</p>

lima: <p>Hi <a class="mention" href="/u/nateanl">@nateanl</a>  thank you for your answer. but what about the fact that each class should either be 1 or 0?</p>

nateanl: <p>I see, if the input has more than one classes that are labeled as 1, then you should use <code>nn.BCELoss</code> and make your prediction and label both of shape <code>(batch, class)</code>.</p>

lima: <p>Hi <a class="mention" href="/u/nateanl">@nateanl</a> The idea is the following. I have computed the mfcc features for some audio data, and I have trained my model to do phoneme classification.<br>
Now instead of the phoneme classification, I would like to do a phonetic feature classification (i.e. classify the individual phonetic features (total 20) for each audio frame). I believe <code>nn.BCELoss</code> can do the job, but what about the prediction and labels shape?</p>

xdwang0726: <p>If you are dealing with multi-label classification <code>sigmoid</code> + <code>nn.BCELoss</code> should be what you are looking for; and if you are looking for multi-class classification (each of your sample will belong to only one of the 20 classes) then <code>softmax</code> + <code>nn.CrossEntropyLoss</code> are the things.</p>

lima: <p>Thank you <a class="mention" href="/u/xdwang0726">@xdwang0726</a></p>
U-Net resulting in absolutely binary classification

Tiago_Dagostini: <p>Hello. I  am rather new to Pytorch (used to work mostly with keras in previous jobs) and I am in need of some help.  I am facing some  issues when I implemented a segmentation model using the u-net from segmentation models</p>
<p>import segmentation_models_pytorch as smp</p>
<p>model_old = smp.Unet(<br>
encoder_name=“resnet34”,<br>
encoder_weights=None,<br>
in_channels=3,<br>
classes=8,<br>
).to(device)</p>
<p>I train the model and it get quite ok results quite fast, but each pixel has always 0 or 1 at the final result. There is no degree of belief/pertinence  of the pixel/area to the class. I really need the degree so I can apply some thresholds.</p>
<p>Is that  expected from an U-NET, is is characteristic from this implementation or should I be doing something different to achieve that? Could any kind soul give me some pointers?</p>
Sox_effects.apply_effects_f

Alva-2020: <p>Hello,<br>
import torchaudio.sox_effects as sox_effects<br>
wav, sr = sox_effects.apply_effects_file(<br>
wav_file,<br>
[[‘speed’, str(speed)],[‘pitch’, ‘0’], [‘rate’, str(sample_rate)]])</p>
<p>This method apply_effects_file is used to change the speed of speech, but the pitch is changed even I set the pitch shift is zero.  I want to know how to change the speed of audio but not change the pitch.<br>
In addition, I found another method “torchaudio.transforms.TimeStretch” to stretch on the stft and keep the pitch unchanged. If the “torchaudio.transforms.TimeStretch” can make the speed change but the pitch unchanged?</p>
<p>Thanks very much for your help.</p>

nateanl: <p>Hi <a class="mention" href="/u/alva-2020">@Alva-2020</a>, you can apply <code>tempo</code> argument for changing the speed while keeping the pitch. As described in <a href="http://sox.sourceforge.net/sox.html" rel="noopener nofollow ugc">sox document</a>.<br>
Yes, <code>torchaudio.transforms.TimeStretch</code> can change the speech while keeping the pitch. Note that the method applied is different from sox, the values may not be fully identical.</p>
Model output have wrong shape

Martina_Ragulikova: <p>Hi I am trying to make super ressolution model and i want it to upsaple image by 3 times with upsample layer in model. The problem is i am getting always wrong output from model.<br>
Output should be this - torch.Size([32, 3, 384, 384]) as I have my labels<br>
But model gives me - torch.Size([32, 128, 19, 19])</p>
<p>Can you please give me TIP what to change in this model to get correct output - ([32, 3, 384, 384]) ?<br>
My input to model is ([32, 3, 128, 128]) tensor of images</p>
<p>Thanks</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

class Super_ress_model(torch.nn.Module):
    def __init__(self, input_shape= (3, 256, 352), output_shape= (2, 256, 352)):
        super(Super_ress_model, self).__init__()

        self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

        self.layer1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=1)
        self.activation1 = nn.ReLU()

        self.layer2 = nn.Conv2d(64, 128, kernel_size=3, stride=2, padding=1)
        self.activation2 = nn.ReLU()

        self.upsample = torch.nn.Upsample(size=None, scale_factor=3, mode='nearest', align_corners=None)

        self.layer3 = nn.Conv2d(128, 128, kernel_size=3, stride=2, padding=1)
        self.activation3 = nn.ReLU()

        self.layer4 = nn.Conv2d(128, 128, kernel_size=3, stride=2, padding=1)
        self.activation4 = nn.ReLU()


    def forward(self,x):
        x = self.layer1(x)
        x = self.activation1(x)

        x = self.layer2(x)
        x = self.activation2(x)

        x = self.upsample(x)

        x = self.layer3(x)
        x = self.activation3(x)

        x = self.layer4(x)
        x = self.activation4(x)

        return x

</code></pre>
Torch.ops.torchaudio.sox_effects_apply_effects_file code

Alva-2020: <p>Hello, I want to know how to check the source code of torch.ops.torchaudio.sox_effects_apply_effects_file. I don’t know where I can see those codes?</p>

ptrblck: <p>I think you can find the definition <a href="https://github.com/pytorch/audio/blob/27e6779cdc2369a42a0e881f21e3644d0d8862e1/torchaudio/csrc/sox/effects.cpp#L92-L138">here</a>.</p>

Alva-2020: <p>Thank you very much.</p>
Can we calculate gradients for quantized models?

yyl-github-1896: <p>I tried to use torch.autograd.grad() to calculate gradients for a quantized model, just as what we usually do on full precision models:</p>
<pre><code class="lang-auto">    for idx, (inputs, targets) in enumerate(data_loader):
        with torch.enable_grad():
            inputs.requires_grad = True
            outputs = quantized_model(inputs)
            loss = criterion(outputs, targets)
            grads = torch.autograd.grad(loss, inputs)[0]
</code></pre>
<p>But I got a RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn<br>
Does models quantized with PyTorch Quantization currently do not support backpropagation? Is there some methods I can calculate the gradients for PyTorch quantized models?</p>

supriyar: <p>Hi <a class="mention" href="/u/yyl-github-1896">@yyl-github-1896</a></p>
<p>quantized models currently run only during inference so you can only call forward on them. If you are trying out quantization aware training <a href="https://pytorch.org/tutorials/recipes/quantization.html#quantization-aware-training" class="inline-onebox" rel="noopener nofollow ugc">Quantization Recipe — PyTorch Tutorials 1.9.1+cu102 documentation</a>, we do support back-propagation in that case during training.</p>

yyl-github-1896: <p>Thank you for the reply. I know that quantization aware training use fake quantization during training, which simulates quantization with fp32. I want to know that what is the difference between fake quantization and real quantization, especially when we do back-propagation on them?</p>

HDCharles: <p>fake quantization simulates quantization but uses high precision data types</p>
<p>so for example imagine if you were trying to quantize to integers.</p>
<p>mathematically a quantized linear op would be:</p>
<p>X = round(X).to(int)<br>
weight = round(weight).to(int)<br>
out = X*weight</p>
<p>whereas a fake_quantized linear would be<br>
X = round(X).to(fp32)<br>
weight = round(weight).to(fp32)<br>
out = X*weight</p>
<p>In practice quantized weights are stored as quantized tensors which are difficult to interact with in order to make them able to perform quantized operations quickly.</p>
<p>fake_quantized weights are stored as floats so you can interact with them easily in order to do gradient updates.</p>
<p>most quantized ops do not have a gradient function so you won’t be able to take a gradient of it. Note: even quantization aware training doesn’t really give gradients of the model, see: <a href="https://arxiv.org/abs/1308.3432" class="inline-onebox" rel="noopener nofollow ugc">[1308.3432] Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation</a></p>

yyl-github-1896: <p>Thank you so much for your clear explanation !   : )</p>
[c++] Cuda GPU mode isn't available on my env

JohnSidKim: <p>env : vs2019, libtorch 1.9.0+cu11.1 , cuda 11.1, cudnn 7.6.5</p>
<p>include dir<br>
: libtorch/include<br>
: libtorch/include/torch/csrc/api/incldue<br>
: opencv/build/incldue<br>
: cudnn/include<br>
: cuda/include</p>
<p>lib dir<br>
: libtorch/lib<br>
: cudnn/lib<br>
: opencv/build/x64/vc15/lib</p>
<p>add dependency<br>
c10.lib<br>
torch.lib<br>
torch_cpu.lib<br>
torch_cuda.lib<br>
opencv_world453.lib</p>
<p>also i tried to add -INCLUDE:?warp_size@cuda@at@<span class="mention">@YAHXZ</span> on dependency</p>
<p>but it occured an error related to added dependency with following message.</p>
<p>PyTorch is not linked with support for cuda devices<br>
Exception raised from getDeviceGuardImpl at …\c10/core/impl/DeviceGuardImplInterface.h:294 (most recent call first):</p>
<p>i can build/run the project with torch::cpu but i could’t run with gpu mode.</p>
<p>what can i do to handle the problem?</p>
<p>please answer me. thank you.</p>

JohnSidKim: <p>i solved it by changing a command from [-INCLUDE:?warp_size@cuda@at@<span class="mention">@YAHXZ</span>] to [/INCLUDE:"?ignore_this_library_placeholder@<span class="mention">@YAHXZ</span>"] the reason why cuda 11.1 ver has to use second command.</p>
RuntimeError: set_sizes_and_strides is not allowed on a Tensor created from .data or .detach(). when using Gradient checkpointing

minuet: <p>Hello,</p>
<p>I’m using a pre-trained VGG to extract features at certain layer to avoid the dreaded OOM. The forward pass seems to be fine but during the backward pass, I got the error:</p>
<pre><code class="lang-auto">RuntimeError: set_sizes_and_strides is not allowed on a Tensor created from .data or .detach().
If your intent is to change the metadata of a Tensor (such as sizes / strides / storage / storage_offset)
without autograd tracking the change, remove the .data / .detach() call and wrap the change in a `with torch.no_grad():` block.
For example, change:
    x.data.set_(y)
to:
    with torch.no_grad():
        x.set_(y)
</code></pre>
<p>When I test the same code using on my local machine with a cpu, this error does not happen. I’m not sure how to tackle this. Any thought is much appreciated <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>Full stack here:</p>
<pre><code class="lang-auto">    loss.backward()
  File "/opt/conda/lib/python3.8/site-packages/torch/_tensor.py", line 264, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/opt/conda/lib/python3.8/site-packages/torch/autograd/__init__.py", line 153, in backward
    Variable._execution_engine.run_backward(
  File "/opt/conda/lib/python3.8/site-packages/torch/autograd/function.py", line 87, in apply
    return self._forward_cls.backward(self, *args)  # type: ignore[attr-defined]
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/checkpoint.py", line 122, in backward
    outputs = ctx.run_function(*detached_inputs)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1056, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/container.py", line 139, in forward
    input = module(input)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1056, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 446, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 442, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: set_sizes_and_strides is not allowed on a Tensor created from .data or .detach().
If your intent is to change the metadata of a Tensor (such as sizes / strides / storage / storage_offset)
without autograd tracking the change, remove the .data / .detach() call and wrap the change in a `with torch.no_grad():` block.
For example, change:
    x.data.set_(y)
to:
    with torch.no_grad():
        x.set_(y)
</code></pre>

ptrblck: <p>Could you post a minimal executable code snippet so that we can check where this issue is triggered, please?</p>
Mean Teacher for semi supervised learning

mimpi: <p>Hi all,<br>
please can anyone tell me how to solve this issue? I am trying to train a net with the semi supervised method ‘mean teacher’ and have a problem to update the weight of the teacher.<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/1/c12f18388ef504327913e2950f1cc486b8d3623b.png" data-download-href="https://discuss.pytorch.org/uploads/default/c12f18388ef504327913e2950f1cc486b8d3623b" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/1/c12f18388ef504327913e2950f1cc486b8d3623b_2_690x107.png" alt="image" data-base62-sha1="ryZ3c7lYc2F5iTchXWrKHojdfmz" width="690" height="107" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/1/c12f18388ef504327913e2950f1cc486b8d3623b_2_690x107.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/1/c12f18388ef504327913e2950f1cc486b8d3623b_2_1035x160.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/1/c12f18388ef504327913e2950f1cc486b8d3623b_2_1380x214.png 2x" data-dominant-color="2F3032"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1515×235 33.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Based on the error message you are running into a shape mismatch while trying to update one of the parameters. Check the shape of the parameters and make sure <code>add_</code> would get the expected shapes.</p>
Image segmentation using Unet to generate binary masks using pytorch and cuda

Sanjana_Mudduluru: <p>Hi, I am new to pytorch and image segmentation(UNET). I am trying to train the UNet to generate binary masks(Grayscale) on retinal images(RGB) + masks (RGB). I am converting both the masks to grayscale for training. Please help me with the following questions:</p>
<ol>
<li>The Mean IOU and Pixel Acc values are not getting computed. I am not entirely sure why? or if the function calls are active.</li>
<li>How to implement pixel wise cross entropy loss? which loss function works efficient in this case?</li>
<li>My training and validation loss values are very poor. Should I do anything different here?</li>
<li>How should I save the model and use it for the testing loop.<br>
Thank you for your help.</li>
</ol>
<p>Please find my code below.<br>
<a href="https://colab.research.google.com/drive/1EpmvhT_4SgAdxB5PuiBpUJ6YfaBiWiqQ?usp=sharing" rel="noopener nofollow ugc">Unet_Colab</a></p>

ptrblck: <ol>
<li>You can add debug print statements to these functions and check what is calculated and if there might be a bug in the code.</li>
<li>You can use <code>nn.CrossEntropyLoss</code> with a model output shape of <code>[batch_size, nb_classes, height, width]</code> and a target in the shape <code>[batch_size, height, width]</code> containing class indices in the range <code>[0, nb_classes-1]</code>.</li>
<li>The accuracy and IOU looks a bit strange, so I would probably check them first.</li>
<li>You can call <code>model.eval()</code> and wrap the testing loop into <code>with torch.no_grad()</code> to calculate the validation/testing results.</li>
</ol>
Pytorch 1.2.0, pytorch 1.7.0, dilated conv

Tony_dong: <p>I used PyTorch 1.2.0 to train a semantic segmentation model, which used dilated Conv. Using pytorch1.2.0 to load the model for reasoning, the speed and effects are normal. But now using pytorch1.7.0 to load the model for reasoning, speed will be more than a hundred times slower than the former. What is the problem?</p>
Diagonal of Dot product of two large tensor

xdwang0726: <p>I am wondering is there a fast way to get the main diagonal of a dot product of two large tensor?</p>
<p>Example:<br>
a = tensor([[0.9651, 0.5328],<br>
[0.4087, 0.1779]])<br>
b = tensor([[0.9567, 0.1259],<br>
[0.7130, 0.4468]])</p>
<p>a * b  = tensor([[0.9233, 0.0671],<br>
[0.2914, 0.0795]])<br>
expected: torch.diagonal(a*b) = tensor([0.9233, 0.0795])</p>
<p>Thanks!</p>

KFrank: <p>Hi xdwang!</p>
<p>First, the example you give is not the “dot product” of two tensors,<br>
as you refer to it in the title of the your thread.  Rather, it is<br>
<em>element-wise</em> multiplication of two tensors.</p>
<p>If indeed what you want is the diagonal of the element-wise product<br>
of two tensors, it will be more efficient to extract the diagonals of<br>
the two tensors first and then multiply them element-wise:</p>
<pre><code class="lang-nohighlight">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.9.0'
&gt;&gt;&gt; a = torch.tensor ([[0.9651, 0.5328], [0.4087, 0.1779]])
&gt;&gt;&gt; b = torch.tensor ([[0.9567, 0.1259], [0.7130, 0.4468]])
&gt;&gt;&gt; torch.diag (a) * torch.diag (b)
tensor([0.9233, 0.0795])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>
How to prune a pretrained model?

tslermen: <p>Hey,</p>
<p>How can i prune a pretrained model? I am using the EfficientNetB0 as backbone and I was trying to get just the 4 first blocks of this model. The first blocks look like this</p>
<p><code>EfficientNet_SW(   (base_model): EfficientNet(     (_conv_stem): Conv2dStaticSamePadding(       2, 32, kernel_size=(3, 3), stride=(2, 2), bias=False       (static_padding): ZeroPad2d(padding=(0, 1, 0, 1), value=0.0)     )     (_bn0): BatchNorm2d(32, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)     (_blocks): ModuleList(       (0): MBConvBlock(         (_depthwise_conv): Conv2dStaticSamePadding(           32, 32, kernel_size=(3, 3), stride=[1, 1], groups=32, bias=False           (static_padding): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0)         )         (_bn1): BatchNorm2d(32, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)         (_se_reduce): Conv2dStaticSamePadding(           32, 8, kernel_size=(1, 1), stride=(1, 1)           (static_padding): Identity()         )         (_se_expand): Conv2dStaticSamePadding(           8, 32, kernel_size=(1, 1), stride=(1, 1)           (static_padding): Identity()         )         (_project_conv): Conv2dStaticSamePadding(           32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False           (static_padding): Identity()         )         (_bn2): BatchNorm2d(16, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)         (_swish): MemoryEfficientSwish()       ) </code></p>
<p>Repeating the MBConvBlock for a few times.</p>
How to cache dataloader result

yjguo: <p>I have a dataset which is not big, but the torch.utils.data.DataLoader performance is not good on my system, so, I plan to cache the result of the first dataloader, and then use the cached results.</p>
<p>For speed, I don’t want to do torch.tensor copy during the cache processing, I tried with queue/list, but looks that the tensor copy is always there. Could you help on this? How to cache the result with reference, instead of full data copy? thanks.</p>
<p>The code looks like</p>
<pre><code class="lang-auto">val_loader = torch.utils.data.DataLoader(
    val_dataset,
    batch_size=16, shuffle=False,
    num_workers=8, pin_memory=True)

cached = [] #queue.Queue()
for i, (_input, _target) in enumerate(val_loader):
    cached.append((_input, _target))  // the performance drops a lot when this line added
    //do something
</code></pre>

eqy: <p>Are you concerned about data loading speed due to the files being on disk rather than in-memory? I believe that if there is enough main memory to cache the files, the filesystem will do this by default so any additional savings from caching the loaded tensor may be minimal. Furthermore, it seems this would require turning off shuffling (which you have correctly done), which may hurt model accuracy.</p>

yjguo: <p>thanks <a class="mention" href="/u/eqy">@eqy</a></p>
<p>I want to save all the time from disk read to decoder to image preprocessing, so to cache the dataloader’s final result (after preprocessing).  It is for validation, so shuffle=false doesn’t matter.</p>
<p>I have some other findings.</p>
<ol>
<li>
<p>there’s no data copy here, I verified it with data_ptr().</p>
</li>
<li>
<p>the cached.append time is very small, but it adds much in the total time, don’t know why.</p>
</li>
</ol>
<pre><code class="lang-auto">start = time.time()
cache_time = 0
cached = [] #queue.Queue()
for i, (_input, _target) in enumerate(val_loader):
    cache_start = time.time()
    cached.append((_input, _target))
    cache_time += (time.time() - cache_start) * 1000

total_time = time.time() - start

print(cache_time)
print(total_time)
</code></pre>
<p>For dataset with 50000 images, the cache_time is ~0.3ms, but the total time increases from 15 seconds to 22 seconds after I add ‘cached.append((_input, _target))’ in the code.</p>
<ol start="3">
<li>If I change pin_memory from true to false, all the time are expected, the cache_time is still ~0.3ms, and the total time is still around 15 seconds.</li>
</ol>
<p>I see the time from 15s to 22s is not a big problem, but still want to understand why? thanks.</p>
Different machines with different results?

jhp: <p>HI Im currently do some research by using multi-gpu.<br>
My concern is, if I use different machines even though I set <code>torch.manual_seed(seed)</code>, will the results be different?</p>
<p>cf)all the other settings(i.e torch version, machine kinds etc…)</p>

eqy: <p>I think using <code>torch.manual_seed</code> is necessary to ensure that the results are the same, but not necessarily sufficient. For example, you might need to also run in deterministic mode (see this page for more details: <a href="https://pytorch.org/docs/stable/notes/randomness.html" class="inline-onebox" rel="noopener nofollow ugc">Reproducibility — PyTorch 1.9.1 documentation</a>).</p>
<p>Additionally, mixing other random number generators (e.g., numpy) might be an issue if they do not use the manual seed set with <code>torch.manual_seed</code>.</p>

jhp: <p>Thanks for ur reply.<br>
Actually it was sufficient for me to use only <code>torch.manual_seed</code> in my machine, but Im not sure if my model gets different results when I use other machines .</p>
Is PyTorch (and related CV libraries) optimized differently for Intel vs AMD CPUs?

Mohamad_Moustafa: <p>I am currently configuring a PC to use for deep learning and computer vision; it has a single Nvidia rtx 3090 where I will be doing all my training. I mostly use PyTorch, so I wanted to know if PyTorch (or any of the libraries commonly used with it for computer vision) would run better or is specifically optimized for either Intel or AMD CPUs.</p>
<p>Right now I am considering AMD Ryzen 9 5900X.</p>

eqy: <p>AFAIK PyTorch itself tries to strike a balance for performance rather than optimizing for any CPU in particular. I think it depends on what computation you expect to do on CPU and the relative performance difference due to Intel vs. AMD differences (e.g., MKLDNN has AVX-512 support and AVX-512 support was recently landed in upstream: <a href="https://github.com/pytorch/pytorch/pull/56992" rel="noopener nofollow ugc">Add AVX512 support in ATen &amp; remove AVX support by imaginary-person · Pull Request #56992 · pytorch/pytorch (github.com)</a>, is maximum SIMD width or higher core count more important for your workload?).</p>
How could I inspect the scale and zero_point value of "QFunctional" from a quantized torchScript Model?

xiuyangleiasp: <p>Hello, I try to inspect the Scale and zero_point value of “QFuncational” with a torchscript Model.<br>
But, it always got error: “‘RecursiveScriptModule’ object has no attribute ‘scale’” and “zero_point”.</p>
<p>for “Conv” it can directly got “scale” and “zero_point” value with conv.scale/conv.zero_point.</p>
<p>And, the “QFuncational” actually is “Add”</p>

HDCharles: <p>this works, can you give a repro?</p>
<pre><code class="lang-auto">model = nn.Sequential(torch.quantization.QuantStub(), torch.nn.Conv2d(2,2,2))
model.qconfig = torch.quantization.get_default_qconfig()
modelp = torch.quantization.prepare(model)
modelp(torch.randn(1,2,32,32))
modelq = torch.quantization.convert(modelp)
modelj = torch.jit.script(modelq)
print(modelq[1].scale)
print(modelj[1].scale)
</code></pre>
<p>also it looks like QFunctionals should still work in the same way (<a href="https://github.com/pytorch/pytorch/blob/22ec6250283568fc475d030dc0b15eda5d13b00b/torch/nn/quantized/modules/functional_modules.py#L153" class="inline-onebox" rel="noopener nofollow ugc">pytorch/functional_modules.py at 22ec6250283568fc475d030dc0b15eda5d13b00b · pytorch/pytorch · GitHub</a>)</p>
<p>so you should be able to inspect them directly.</p>
Natural language processing for japanese search engine

coderman: <p>I want to build a search engine for Japanese. Japanese is difficult because there are no spaces between words, verbs are conjugated to show tense, negation, and politeness. Japanese is also tricky because it uses multiple character systems: two phonetic systems (one for words from Japanese, hiragana, and one for foreign words, katakana) and a symbolic systems (borrowed from Chinese).</p>
<p>What would you need to do first before you could create a tf-idf index that is different<br>
from English?</p>
Classifynig dicom images for lung cancer

Ali_Buyukanbarlar: <p>Hi everyone,</p>
<p>I want to write a code to find lung cancer for me. To illustrate, if I have 100 MRI pictures, the code should seperates them as those which have the lung cancer and which have not. If there is a similir code or example for it, it would be helpful for me.</p>

eqy: <p>Since this appears to be a classification task, the MNIST or ImageNet examples:<br>
<a href="https://github.com/pytorch/examples/blob/master/mnist/main.py" rel="noopener nofollow ugc">examples/main.py at master · pytorch/examples (github.com)</a><br>
<a href="https://github.com/pytorch/examples/blob/master/imagenet/main.py" rel="noopener nofollow ugc">examples/main.py at master · pytorch/examples (github.com)</a></p>
<p>should be a good starting point.</p>
Gloo in Pytorch for GPU tensor collective communication

DanielWang: <p>For Gloo in Pytorch distributed, as shown in this document <a href="https://pytorch.org/docs/stable/distributed.html" class="inline-onebox" rel="noopener nofollow ugc">Distributed communication package - torch.distributed — PyTorch 1.9.1 documentation</a>, will the following code get performance benefits of using CUDA-aware MPI? (e.g., GPU-to-GPU transferring via PCIe while bypassing CPU)</p>
<pre><code class="lang-auto">group = dist.new_group([0, 1], backend="gloo")
dist.all_reduce(gpu_tensor_a, op=dist.ReduceOp.SUM, group=group)
</code></pre>
Read .pt files in the form of graphs or discrete numerical values

dhruvsheth-ai: <p>Hi, this question might have been repeated but I can’t find an exact way to go around doing this.<br>
I have loss functions and Mean squared error functions stored in the form of .pt files produced by an external script. An example of this .pt file is -</p><aside class="onebox githubblob" data-onebox-src="https://github.com/dhruvsheth-ai/PlantGAN/blob/main/pred/mse_avg.pt">
  <header class="source">

      <a href="https://github.com/dhruvsheth-ai/PlantGAN/blob/main/pred/mse_avg.pt" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/dhruvsheth-ai/PlantGAN/blob/main/pred/mse_avg.pt" target="_blank" rel="noopener nofollow ugc">dhruvsheth-ai/PlantGAN/blob/main/pred/mse_avg.pt</a></h4>


  This file is binary. <a href="https://github.com/dhruvsheth-ai/PlantGAN/blob/main/pred/mse_avg.pt" target="_blank" rel="noopener nofollow ugc">show original</a>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<aside class="onebox githubblob" data-onebox-src="https://github.com/dhruvsheth-ai/PlantGAN/blob/main/pred/psnr_avg.pt">
  <header class="source">

      <a href="https://github.com/dhruvsheth-ai/PlantGAN/blob/main/pred/psnr_avg.pt" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/dhruvsheth-ai/PlantGAN/blob/main/pred/psnr_avg.pt" target="_blank" rel="noopener nofollow ugc">dhruvsheth-ai/PlantGAN/blob/main/pred/psnr_avg.pt</a></h4>


  This file is binary. <a href="https://github.com/dhruvsheth-ai/PlantGAN/blob/main/pred/psnr_avg.pt" target="_blank" rel="noopener nofollow ugc">show original</a>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Would be really helpful if any code or instructions can be provided on how to load these files and read the numerical values. I was able to load and read the file, but the only output I receive is in the form of a tensor.<br>
What I expect is, few numerical values sufficient to plot a graph like -</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/1/e1e6152e5eb3396d16b0fb8a1d8480703bef3dfb.png" alt="image" data-base62-sha1="weonBY8mVYgTziVNvmFWgRuWeyn" width="440" height="314"></p>

ptrblck: <p>Could you explain how these files were created?<br>
If you’ve stored tensors in them, you should be able to load all tensors (containing all values) back.</p>
Quantized model parameter after PTQ, INT8?

Jia-Ming_Lin: <p>Hi,</p>
<p>I quantized a pre-trained FP32 model to INT8 by using PTQ in Pytorch, but when I print out the model parameters, they are not integer(INT8). Why? and how can I obtain the INT8 model?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/f/3f2c80d043a3ef4a34ea2d9aad1f5175a84e9997.png" data-download-href="https://discuss.pytorch.org/uploads/default/3f2c80d043a3ef4a34ea2d9aad1f5175a84e9997" title="Screen Shot 2021-10-10 at 3.59.23 AM"><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/f/3f2c80d043a3ef4a34ea2d9aad1f5175a84e9997.png" alt="Screen Shot 2021-10-10 at 3.59.23 AM" data-base62-sha1="90RqydF2VGg8uAiViMZtqxMjRTF" width="689" height="324" data-dominant-color="3A3B35"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2021-10-10 at 3.59.23 AM</span><span class="informations">1106×520 56.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>PS. My goal is to port the Pytorch model to FPGA, and run in INT8, is there any suggestion?</p>
<p>Thanks</p>

HDCharles: <p>qtensor.int_repr() gives the integer weights but I think there might be a misunderstanding.</p>
<p>in general quantized tensors contain 3 components: int weights, scale and zero_points.</p>
<p>if your original tensor is something like: 0, .1, .2, .31<br>
if you were to quantize it, you might get something with weight=[0,1,2,3] scale=.1, zero_point=0.</p>
<p>we aren’t actually quantizing the weights to integer values, for to be quantized tensor T we are trying to find s,z,T_int such that s<em>T_int+z ~ T so what you are seeing is the actual values of s</em>T_int+z (note: the actual quant/dequant equation depends)</p>
Utilizing autograd in C++ extensions for custom operator

fanl: <p>Hi, everyone<br>
Is there any way to use autograd mechanism in C++ extension?<br>
Specifically,  I am going to write a custom operator in C++, which might contain some off-the-shelf functions, for example, <code>at::softmax</code> . I do not want to manually re-implement the backward process of softmax.<br>
I describe my requirement in the following pseudo-code:</p>
<pre><code class="lang-auto">at::Tensor forward(at::Tensor input){
    auto a = custom_func(input);
    auto b = at::softmax(a); 
    save_for_backward_somehow(a, b);
    return b;
}

at::Tensor backward(at::Tensor out_grad){
    auto a, b = saved_tensor();
    b.backward(out_grad); // avoid re-implementing the backward of softmax
    auto grad_of_a = a.grad; // after calling backward on b, we can get the gradient of a somehow.
    return get_input_grad(grad_of_a);
}
</code></pre>
<p>I will appreciate any comments and suggestions.</p>

ptrblck: <p>Yes, <code>libtorch</code> supports Autograd and you might find some examples in <a href="https://github.com/pytorch/pytorch/blob/6eafe7f15e264e1f163ccf53b3ef9c5306335945/test/cpp/api/autograd.cpp">this test file</a>.</p>

Hamster: <p>You can use <code>torch::autograd::Function</code> exactly the same way you’d use <code>torch.autograd.Function</code> in Python. So you could first go to the Python documentation on <code>torch.autograd.Function</code> and then navigate in your C++ IDE to where <code>torch::autograd::Function</code> is defined and look at the example that is given in the comments. That’s how I got very quickly the hang of it.</p>
A possible bug in https://github.com/pytorch/pytorch/blob/master/caffe2/operators/roi_pool_op.cc

UlyssesStrong: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/a/9abfd73d922e922c868215b2f7d6157e10922173.png" data-download-href="https://discuss.pytorch.org/uploads/default/9abfd73d922e922c868215b2f7d6157e10922173" title="torch_a_bug"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/a/9abfd73d922e922c868215b2f7d6157e10922173_2_690x470.png" alt="torch_a_bug" data-base62-sha1="m4YxWkinnalh6SwKfEa9HygkZ4D" width="690" height="470" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/a/9abfd73d922e922c868215b2f7d6157e10922173_2_690x470.png, https://discuss.pytorch.org/uploads/default/optimized/3X/9/a/9abfd73d922e922c868215b2f7d6157e10922173_2_1035x705.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/9/a/9abfd73d922e922c868215b2f7d6157e10922173.png 2x" data-dominant-color="FAFAFA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">torch_a_bug</span><span class="informations">1076×733 57 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
As the screenshort, the position of the red frame should be -INFINITY. Secondly, if the output1 is -INFINITY, the output2, argmax, should be the first index in the rois, not be -1.</p>

ptrblck: <p>Could you create an issue on <a href="https://github.com/pytorch/pytorch/issues">GitHub</a> so that the code owners could have a look at it and discuss the issue, please?</p>
Calculating hessian matrix

Mari: <p>Is there anyone who can provide me an example how I am able to compute the hessian matrix of the loss(d ^2 loss/d^2 weights) of below network using torch.autograd.functional.hessian() ?</p>
<p>class Net(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Net, self).<strong>init</strong>()<br>
self.main = nn.Sequential(<br>
nn.Linear(input_n,h_n),<br>
Swish(),<br>
nn.Linear(h_n,h_n),<br>
Swish(),<br>
nn.Linear(h_n,1),<br>
)<br>
def forward(self,x):<br>
output = self.main(x)<br>
return  output<br>
for epoch in range(epochs):<br>
for batch_idx, (x,y) in enumerate(dataloader):<br>
loss = criterion(x,y)<br>
loss.backward()</p>
Manipulating batchnorm running_mean

avalon1511: <p>Hello,</p>
<p>I am trying to manipulate the running mean of the batchnorm layer in a NN as follows -</p>
<pre><code class="lang-auto">new_running_mean = running_mean * x + b
</code></pre>
<p><code>x and b</code> are both trainable parameters defined as -</p>
<pre><code class="lang-auto">nn.Parameter(x, requires_grad = True)
</code></pre>
<p>I understand that batchnorm running mean is non trainable hence the <code>requires_grad = True</code> flag is giving me an error - <code>the derivative for 'running_mean' is not implemented</code></p>
<p>However, I want<code> x and b</code> to be trainable parameters, setting <code>requires_grad = False</code> will directly contradict this.</p>
<p>How can I work around this issue?</p>
<p>Thanks</p>
CPU to GPU becomes slower after 1st iteration

henrysun: <p>I am dealing with a huge matrix, but my GPU memory is limited.<br>
So, I chunk the huge matrix on CPU and use for-loop to transfer smaller chunked one to GPU one-by-one. However, <strong>the CPU-to-GPU becomes slower after 1st iteration</strong>.</p>
<p>sample codes:</p>
<pre><code class="lang-auto">chunks = torch.chunk(huge_matrix, chunks=10, dim=0) # on CPU
for i in range(len(chunks):
    x = chunks[i].to(device) # device: cuda
    # 1st: 0.75s, 2nd: 3.49s, 3rd: 3.65s, ....
    ....
</code></pre>

henrysun: <p>Seems like if I changed as follows,</p>
<pre><code class="lang-auto">x = chunks[i].pin_memory().to(device, non_blocking=True) # device: cuda:0
</code></pre>
<p>if I need to assess these codes multiple time for one epoch, from 2nd time, the transfer time is greatly reduced, but <strong>why is it</strong>?</p>

ptrblck: <p>How are you profiling the code?<br>
Since CUDA operations are executed asynchronously, you would have to synchronize the code before starting and stopping the timers.<br>
Otherwise your profiles will be wrong and would accumulate the runtime of previous kernels into a blocking operation.</p>
How to evaluate with RunTImeError

dadaddapyer: <p>Some of the data in my dataset are too large to make one pass forward during evaluation.<br>
In that case, my model would throw a run time error.<br>
Is there any way to make the model ignore the error and move on to the next data point?<br>
Thanks!</p>

ptrblck: <p>You could try to catch the exception and try to continue the code.<br>
However, depending on the raised exception this might not be possible.</p>
Backwared through bernoullie sample

NirmalaNakka: <p>I  need some information and help  regarding how to do the Bernoullie distribution sampling using weights. can you please help me with this if you have a free time?.</p>

Sayed_Nadim: <p>Hi,<br>
What kind of issues are you facing?</p>

googlebot: <p>check if RelaxedBernoulli (supporting rsample) would do what you want. For more advanced use, modeling uncertainty, sample Bernoulli parameter from a trainable Beta distribution:</p>
<p>pseudo-code is something like:</p>
<p>P = Beta(softplus(raw_a), softplus(raw_b)) # raw_a = nn.Parameter(…)<br>
p = P.rsample()<br>
B = RelaxedBernoulli(probs=p)<br>
b = B.rsample()<br>
loglik = P.log_prob(p) + B.log_prob(b) <span class="hashtag">#for</span> proper probabilistic models</p>

NirmalaNakka: <p>I am facing probelm<br>
RuntimeError: Expected p_in &gt;= 0 &amp;&amp; p_in &lt;= 1 to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)</p>

<p>for the following problem<br>
class SigmoidForwardLinearBackward(torch.autograd.Function):<br>
<span class="mention">@staticmethod</span><br>
def forward(ctx, ins):<br>
# return torch.nn.functional.sigmoid(ins)</p>
<pre><code>     return torch.bernoulli(ins)
    

@staticmethod
def backward(ctx, grad_output):
    return grad_output
</code></pre>
<p>MyFunction = SigmoidForwardLinearBackward.apply</p>
<p>plt.figure()</p>
<h1>
<a name="sigmoids-1" class="anchor" href="#sigmoids-1"></a>sigmoids = []</h1>
<p>bernoulli = []<br>
backwards = []<br>
for x in torch.arange(-5, 5, 0.01):<br>
x.requires_grad = True<br>
outs = MyFunction(x)<br>
outs.backward()<br>
bernoulli.append(outs)<br>
backwards.append(x.grad)</p>

NirmalaNakka: <p>thank you so much for providing me information. can you please provide me bit more clear information please.</p>

googlebot: <ol>
<li>torch.bernoulli derivative is zero (due to discrete output)</li>
<li>your autograd.Function doesn’t make sense, because your “function”'s derivative doesn’t depend on its input. you don’t handle the problem of discreteness, and can as well supply any “external” binary tensor to the network with the same effect (no backprop beyond it).</li>
<li>RelaxedBernoulli is a workaround, that works by allowing continuous outputs (samples) during training. It is not an easy way, but you can read about it - <a href="https://arxiv.org/abs/1611.00712" class="inline-onebox" rel="noopener nofollow ugc">[1611.00712] The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables</a>
</li>
</ol>

googlebot: <p>another alternative, esp. if you can’t use continuous 0…1 outputs, is to use a “score function”, similar to how it is described <a href="https://pytorch.org/docs/master/distributions.html" rel="noopener nofollow ugc">in the docs</a></p>