How to use cpu_kernel_vec

zetyquickly: <p>Hello,</p>
<p>I am trying to write custom quantized operator. This is my first dive into the cpp code base and it seems difficult for me to understand how <code>cpu_kernel_vec</code> from <code>Loops.h</code> works.</p>
<p>Would you mind to help with this example?</p>
<pre><code class="lang-auto">AT_DISPATCH_QINT_TYPES(out.scalar_type(), "qmul", [&amp;]() {
    using Vec = Vec256&lt;scalar_t&gt;;
    cpu_kernel_vec(
        iter,
        [&amp;](scalar_t a, scalar_t b) -&gt; scalar_t {
          int32_t a_sub_z = static_cast&lt;int32_t&gt;(a.val_) -
              static_cast&lt;int32_t&gt;(self_zero_point);
          int32_t b_sub_z = static_cast&lt;int32_t&gt;(b.val_) -
              static_cast&lt;int32_t&gt;(other_zero_point);
          int32_t c = a_sub_z * b_sub_z;
          scalar_t res = at::native::requantize_from_int&lt;scalar_t&gt;(
              multiplier, zero_point, c);
          if (ReLUFused) {
            res.val_ = std::max&lt;scalar_t::underlying&gt;(res.val_, zero_point);
          }
          return res;
        },
        [&amp;](Vec a, Vec b) -&gt; Vec {
          Vec::int_vec_return_type a_sub_zp =
              a.widening_subtract(Vec(static_cast&lt;scalar_t&gt;(self_zero_point)));
          Vec::int_vec_return_type b_sub_zp =
              b.widening_subtract(Vec(static_cast&lt;scalar_t&gt;(other_zero_point)));
          Vec::int_vec_return_type c;
          for (int i = 0; i &lt; Vec::int_num_vecs(); ++i) {
            c[i] = a_sub_zp[i] * b_sub_zp[i];
          }
          Vec rv = Vec::requantize_from_int(c, multiplier, zero_point);
          if (ReLUFused) {
            rv = rv.maximum(Vec(static_cast&lt;scalar_t&gt;(zero_point)));
          }
          return rv;
        });
  });
}
</code></pre>
<p>So here I can see that <code>cpu_kernel_vec</code> takes</p>
<ol>
<li>
<code>iter</code> that data to process,</li>
<li>function that processes scalar values,</li>
<li>function that processes vectors.</li>
</ol>
<p>I have hard times to understand when the first is called. Does it overrides <code>Vec</code> multiplication operator? If so how does it know which operator to override.<br>
Confused <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>

zetyquickly: <p><a class="mention" href="/u/tom">@tom</a><br>
Hello Thomas,<br>
maybe you can guide me?</p>

googlebot: <p>Function <span class="hashtag">#2</span> processes AVX2 chunks (e.g. blocks of 8 qint32 values). Function <span class="hashtag">#1</span> is still needed to handle incomplete blocks or non-contiguous tensors - loops.h has the logic to call it as needed.</p>

zetyquickly: <p>Thank you!</p>
<p>So do we just need to specify in <em>function <span class="hashtag">#1</span></em> what should be done with scalars, and in <em>function <span class="hashtag">#2</span></em> what should be done with vectors? And do not care about in which conditions they’ll be used?</p>

googlebot: <p>Yeah, that would be one of approaches. You don’t have to use cpu_kernel_vec - there is also cpu_kernel, or use can dispatch to an arbitrary c++ template function (e.g. cuda operators look a bit differently, though built-in ones often use similar gpu_kernel function)</p>