Why does an unsigned torch.quint8 tensor have a sign?

username: Andrew_Holmes
cooked: <p>I was implementing quantization and PyTorch and I noticed something that seemed off. Why does applying quantization on a tensor with the dtype torch.quint8 result in a quantized tensor that has a sign.</p>
<p>To reproduce:</p>
<pre data-code-wrap="python3"><code class="lang-plaintext">import torch

x = torch.quantize_per_tensor(torch.tensor(
    [-1.0, 0.0, 1.0, 2.0]), 0.1, 10, torch.quint8)
print(x)
</code></pre>
<p>Output:</p>
<pre><code class="lang-bash">tensor([-1.,  0.,  1.,  2.], size=(4,), dtype=torch.quint8,
       quantization_scheme=torch.per_tensor_affine, scale=0.1, zero_point=10)
</code></pre>

username: jcaip
cooked: <p>Hi <a class="mention" href="/u/andrew_holmes">@Andrew_Holmes</a> this is because the data you see represents the original (non-quantized) values.</p>

username: Andrew_Holmes
cooked: <p><a class="mention" href="/u/jcaip">@jcaip</a> Interesting. I’m aware that much of the quantization process is handled in C++, but isn’t there a Python representation for a quantized tensor? My aim is to construct a small framework utilizing some abstractions provided by PyTorch. In doing so, I’ve been finding it hard to understand how PyTorch performs some quantization operations. It’s been kinda challenging since the details of documentation isn’t the greatest.</p>
<p>Would you have any references on how the quantization process works on torch’s side? Specifically, I’m curious about how <code>torch.quantize_per_tensor()</code> applies the <code>scale</code> and <code>zero_point</code>. Is the formula <code>x * s + z</code>? Or is it <code>x / s - z</code>, etc.? I stumbled upon a brief <a href="https://pytorch.org/blog/quantization-in-practice/" rel="noopener nofollow ugc">article</a> that seemingly outlines PyTorch’s approach to tensor quantization. Yet, without a way to view the resultant tensor, how can be sure the quantization parameters I’m passing are correct for my purposes?</p>

username: jcaip
cooked: <p>For quantize_per_tensor, it’s the first formula (x * s + z)</p>
<p>Sorry I think I misunderstood your initial question. if you just want to see the quantized values, you should be able to call <code>x.int_repr()</code><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/b/8b164b036531fb60352ed21dfaecaf4eb469bef5.png" data-download-href="https://discuss.pytorch.org/uploads/default/8b164b036531fb60352ed21dfaecaf4eb469bef5" title="Screenshot 2023-10-26 at 2.27.38 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8b164b036531fb60352ed21dfaecaf4eb469bef5_2_690x95.png" alt="Screenshot 2023-10-26 at 2.27.38 PM" data-base62-sha1="jQq87rviDR3Cr1GJ1T7tjSsbAep" width="690" height="95" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8b164b036531fb60352ed21dfaecaf4eb469bef5_2_690x95.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8b164b036531fb60352ed21dfaecaf4eb469bef5_2_1035x142.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8b164b036531fb60352ed21dfaecaf4eb469bef5_2_1380x190.png 2x" data-dominant-color="F3F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-10-26 at 2.27.38 PM</span><span class="informations">1420×196 30 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Andrew_Holmes
cooked: <p><a class="mention" href="/u/jcaip">@jcaip</a> This helps a ton! Thanks a lot.</p>

username: Andrew_Holmes
cooked: <p>Hello, I think it’s actually applying the following: X/s + z. Here is why:</p>
<pre><code class="lang-auto">import torch
scale = 0.5
zero = 1

tensor = torch.tensor([[4, 6, 8]]).float()
q_tensor = torch.quantize_per_tensor(
    tensor, scale, zero, dtype=torch.qint8)
print(q_tensor.int_repr())
</code></pre>
<p>Output:<br>
<code>tensor([[ 9, 13, 17]], dtype=torch.int8)</code></p>
<p>If it was multiplicative it would be: <code>tensor([[ 3, 4, 5]], dtype=torch.int8)</code></p>

username: HDCharles
cooked: <p>yep its in our docs in a few places e.g. <a href="https://pytorch.org/docs/stable/generated/torch.fake_quantize_per_tensor_affine.html#torch.fake_quantize_per_tensor_affine" class="inline-onebox" rel="noopener nofollow ugc">torch.fake_quantize_per_tensor_affine — PyTorch 2.1 documentation</a></p>