akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How to static quantization? I have some questions after reading the tutorial

Silbernitrat: <p>I quantified the model based on the tutorial, but the model’s error is a bit large. Therefore, I try to quantify the data myself, then dequantize it, and finally calculate the convolution, and find that this error is smaller than the error of the tutorial method. what is the reason? Did I miss some steps when quantifying the model? The following  is a demo.</p>
<pre><code class="lang-python">import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.quantization import QuantStub, DeQuantStub

torch.backends.quantized.engine = 'qnnpack'

class Test(nn.Module):
    def __init__(self):
        super(Test, self).__init__()
        self.conv = nn.Conv2d(132, 121, 3, 1, bias=False)
        self.quan = QuantStub()
        self.dequan = DeQuantStub()
        nn.init.kaiming_normal_(self.conv.weight, mode='fan_out')

    def forward(self, x):
        out = self.quan(x)
        out = self.conv(out)
        out = self.dequan(out)
        return out

def test():
    model = Test()
    weight = model.conv.weight
    conv = nn.Conv2d(132, 121, 3, 1, bias=False)
    conv.weight = weight
    max, min = torch.max(weight.flatten()), torch.min(weight.flatten())
    scale = (max - min) / 256
    temp = torch.quantize_per_tensor(weight, scale=scale.item(), zero_point=0, dtype=torch.qint8)
    temp_de = temp.dequantize()
    example = torch.rand(1, 132, 64, 64)
    print(F.mse_loss(temp_de, weight))
    # tensor(1.9924e-07, grad_fn=&lt;MeanBackward0&gt;)
    print(F.mse_loss(F.conv2d(example, weight=temp_de, bias=None, stride=1), conv(example)))
    # tensor(8.7799e-05, grad_fn= &lt; MeanBackward0 &gt;)
    torch.backends.quantized.engine = 'qnnpack'
    model.qconfig = torch.quantization.get_default_qconfig('qnnpack')
    torch.quantization.prepare(model, inplace=True)
    torch.quantization.convert(model, inplace=True)
    print(F.mse_loss(conv(example), model(example)))
    # tensor(0.4824, grad_fn= &lt; MseLossBackward &gt;)


if __name__ == "__main__":
    test()
    print(torch.__version__)
    # 1.5.0
</code></pre>

raghuramank100: <p>In the first comparison, you are not quantizing the activations (i.e the example tensor). When you call the torch.quantization.prepare and convert, the activations are quantized too. In this case, since you are skipping calibration (i.e you are calling convert after prepare), the activations are quantized with a default scale of 1.0, leading to a large quantization error. Please repeat this experiment with calibration and you should see lower error</p>

Silbernitrat: <p>Thanks. I konw that I need  to run the model before convernting it.</p>
Using pytorch reduction in custom loss

loskutak: <p>Hi, I am trying to write custom loss function by creating a nn.module subclass with forward method.<br>
Is it possible to use pytorch reduction mode handling in my loss?<br>
Sure I can do:</p>
<pre><code class="lang-auto">if reduction == 'mean':
  return torch.mean(my_loss_value)
elif ...
</code></pre>
<p>But I would rather just call something like:</p>
<pre><code class="lang-auto">return torch_reduction_handling_function(my_loss_value, reduction=reduction)
</code></pre>
<p>However, I don’t know where to find this reduction handling function and if it is exposed to me at all.</p>
<p>Thanks for your help!</p>

ptrblck: <p>If I understand the use case correctly, you are looking for a generic “reduction” method, which would take the desired reduction type as an argument?<br>
If so, then unfortunately PyTorch doesn’t provide this generic method.</p>
<p>You could pass a <code>reduction</code> argument to the functional calls of some criteria, e.g. <code>F.mse_loss(..., reduction=reduction)</code>, but this would of course only work for this particular criterion.</p>

loskutak: <p>ok, then I will have to re-implement. Thanks!</p>
Best way to import Caffe2 Modell to Pytorch

LeroyF: <p>Hello,</p>
<p>is there a good way to import a pre-trained Caffe2 model into Pytorch?</p>
<p>My research only gives me articles that are very old.</p>
<p>Many thanks and best regards</p>

ptrblck: <p>How old are these publications and are you sure these model are implemented in Caffe2, not Caffe?</p>

LeroyF: <p>The models were created in 2018.</p>
<p>They was not trained by myselve, but my client made sure that they were trained in Caffe2.</p>

ptrblck: <p>OK, thanks for the information.<br>
I’m not sure, if there is an automatic way of transforming a C2 model to PyTorch, so I would assume you would have to rewrite the model in PyTorch and load the C2 parameters by creating a <code>state_dict</code> and mapping the names to the expected ones in PyTorch.</p>

LeroyF: <p>Okay, great.<br>
Big thanks for your help!</p>
Detecting covered eyes

Sjay: <p>Hi, here to know about an approach to a problem of detecting covered eyes, the eyes could be covered with anything, cloth, glasses etc. Basically what should be the approach in finding covered human body/face parts?</p>

ptrblck: <p>You could treat it as an image binary classification task.<br>
Once you get the data for positive and negative classes, you could start with fine tuning a model as explained <a href="https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html">here</a>.</p>
<p>Or would you like to “detect” the eyes with bounding boxes?<br>
In that case, you would need to annotate your dataset with these boxes and could fine tune a detection model as explained <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html">here</a>.</p>
Understanding CUDAStream Usage in pytorch

flint-stone: <p>Hi – I’m trying to visualize how cuda stream implemented by putorch ( CUDAStream.cpp ) impacts Cuda stream usage for concurrent model training tasks. For two models created by one process (initGlobalStreamState in CUDAStream.cpp is called only once) both running on gpu:0, my nvprof shows all kernels are issued on the default stream on gpu:0. However, if I create these models by separate processes each process has its own cuda context and kernels are issued to different streams. Just wondering if sharing cuda context would change the way cuda kernels (from two different models, running simultaneously) are dispatched on to GPUs.</p>
<p>Thanks!</p>