About Quantization schemes

username: Himajyothi_Rajamahen
cooked: <p>Hi,</p>
<p>I’m new to this topic.Plz provide the clear insight on the following questions.</p>
<p>What is the difference b/w symmetric and asymmetric quantization?<br>
How to choose the suitable scheme for our model? Does that depend on the weights or on the quantization dtype?</p>
<p>Thanks</p>

username: ZimoNitrome
cooked: <p>Tensorflow has a good section on this: <a href="https://www.tensorflow.org/lite/performance/quantization_spec#symmetric_vs_asymmetric" class="inline-onebox" rel="noopener nofollow ugc">Spesifikasi kuantisasi TensorFlow Lite 8-bit</a></p>

username: HDCharles
cooked: <p>This whitepaper was made by one of the pytorch quantization team members and informs a lot of the implementation.</p>
<aside class="onebox pdf" data-onebox-src="https://arxiv.org/pdf/1806.08342.pdf">
  <header class="source">

      <a href="https://arxiv.org/pdf/1806.08342.pdf" target="_blank" rel="noopener nofollow ugc">arxiv.org</a>
  </header>

  <article class="onebox-body">
    <a href="https://arxiv.org/pdf/1806.08342.pdf" target="_blank" rel="noopener nofollow ugc"><span class="pdf-onebox-logo"></span></a>

<h3><a href="https://arxiv.org/pdf/1806.08342.pdf" target="_blank" rel="noopener nofollow ugc">1806.08342.pdf</a></h3>

  <p class="filesize">858.24 KB</p>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>it shows how symmetric quantization is essentially just when the zero point is set to 0. Note the signed vs unsigned implementation depends on the dtype i.e. quint8 vs qint8.</p>
<p>The best information we have in the documentation is:</p>
<p><a href="https://pytorch.org/docs/stable/torch.quantization.html#torch-quantization" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/torch.quantization.html#torch-quantization</a></p>
<p>which is not great, I created an issue for this here:</p>
<aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/68540">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/68540" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/68540" target="_blank" rel="noopener nofollow ugc">Improve documentation about meaning of different qschemes</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2021-11-17" data-time="21:13:50" data-timezone="UTC">09:13PM - 17 Nov 21 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/HDCharles" target="_blank" rel="noopener nofollow ugc">
          <img alt="HDCharles" src="https://avatars.githubusercontent.com/u/39544797?v=4" class="onebox-avatar-inline" width="20" height="20">
          HDCharles
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          oncall: quantization
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">What exactly is symmetric and affine quantization?

The best doucmentation is <span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">kind of a side point here: https://pytorch.org/docs/stable/torch.quantization.html?highlight=torch%20minmaxobserver#torch.quantization.MinMaxObserver

there should be a clearer explanation for exactly what symmetric and affine quantization means rather than having to go backwards from the calculation of the qparams

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Also if you want a code definition, note that symmetric is generally handled as a special case of affine quantization and all that happens is the way qparams are calculated are different. Here is where that happens:</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/pytorch/blob/b0bdf588ea575928a94264c30999385d5ff2bc32/torch/ao/quantization/observer.py#L283">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/blob/b0bdf588ea575928a94264c30999385d5ff2bc32/torch/ao/quantization/observer.py#L283" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/b0bdf588ea575928a94264c30999385d5ff2bc32/torch/ao/quantization/observer.py#L283" target="_blank" rel="noopener nofollow ugc">pytorch/pytorch/blob/b0bdf588ea575928a94264c30999385d5ff2bc32/torch/ao/quantization/observer.py#L283</a></h4>



  <pre class="onebox">    <code class="lang-py">
      <ol class="start lines" start="273" style="counter-reset: li-counter 272 ;">
          <li>    return torch.tensor([1.0], device=min_val.device.type), torch.tensor([0], device=min_val.device.type)</li>
          <li>
          </li>
<li>quant_min, quant_max = self.quant_min, self.quant_max</li>
          <li>min_val_neg = torch.min(min_val, torch.zeros_like(min_val))</li>
          <li>max_val_pos = torch.max(max_val, torch.zeros_like(max_val))</li>
          <li>
          </li>
<li>device = min_val_neg.device</li>
          <li>scale = torch.ones(min_val_neg.size(), dtype=torch.float32, device=device)</li>
          <li>zero_point = torch.zeros(min_val_neg.size(), dtype=torch.int64, device=device)</li>
          <li>
          </li>
<li class="selected">if (</li>
          <li>    self.qscheme == torch.per_tensor_symmetric</li>
          <li>    or self.qscheme == torch.per_channel_symmetric</li>
          <li>):</li>
          <li>    max_val_pos = torch.max(-min_val_neg, max_val_pos)</li>
          <li>    scale = max_val_pos / (float(quant_max - quant_min) / 2)</li>
          <li>    scale = torch.max(scale, self.eps)</li>
          <li>    if self.dtype == torch.quint8:</li>
          <li>        if self.has_customized_qrange:</li>
          <li>            # When customized quantization range is used, down-rounded midpoint of the range is chosen.</li>
          <li>            zero_point = zero_point.new_full(</li>
      </ol>
    </code>
  </pre>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>