FwFM Quantization

username: pintonos
cooked: <p>I want to perform dynamic quantization on a FwFM model (<a href="https://arxiv.org/pdf/1806.03514.pdf" rel="nofollow noopener">paper</a>). Unfortunately, i did not manage to make it work!</p>
<p>I get following <strong>error</strong>:</p>
<pre><code class="lang-auto">AttributeError: 'function' object has no attribute 't'
</code></pre>
<p>corresponding to the line in my <strong>forward</strong> function:</p>
<pre><code class="lang-auto">outer_fwfm = torch.einsum('klij,kl-&gt;klij', outer_fm,
                                              (self.field_cov.weight.t() + self.field_cov.weight) * 0.5)
</code></pre>
<p>Corresponding model layer <strong>before</strong> quantization:</p>
<pre><code class="lang-auto">(field_cov): Linear(in_features=39, out_features=39, bias=False)
</code></pre>
<p><strong>After</strong> quantization:</p>
<pre><code class="lang-auto">(field_cov): DynamicQuantizedLinear(in_features=39, out_features=39, dtype=torch.qint8, qscheme=torch.per_tensor_affine)
</code></pre>
<p>Quantization line:</p>
<pre><code class="lang-auto">quantized_model = torch.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8)
</code></pre>
<p>Am I missing something out here? Thanks for your help!<br>
<a href="https://github.com/pintonos/xsDeepFwFM" rel="nofollow noopener">GitHub Code</a></p>

username: ptrblck
cooked: <p>It seems that <code>DynamicQuantizedLinear</code> replaces the <code>weight</code> attribute with a method:</p>
<pre><code class="lang-python">lin = torch.nn.quantized.dynamic.Linear(1, 1)
print(lin.weight())
</code></pre>
<p>So you might need to call <code>self.field_cov.weight().t() + self.field_cov.weight()</code>.</p>
<p>Note that, while this might work functionality-wise, I’m not familiar enough with your use case or the dynamic quantization to claim it’s the right approach to use when quantization is applied.</p>

username: pintonos
cooked: <p>Thanks!</p>
<p>Code now looks like this:</p>
<pre><code class="lang-auto"> if self.dynamic_quantization or self.static_quantization or self.quantization_aware:
                        q_func = QFunctional()
                        q_add = q_func.add(self.field_cov.weight().t(), self.field_cov.weight())
                        q_add_mul = q_func.mul_scalar(q_add, 0.5)
                        outer_fwfm = torch.einsum('klij,kl-&gt;klij', outer_fm, q_add_mul)
</code></pre>
<p><strong>Error</strong> Traceback:</p>
<pre><code class="lang-auto">...
return _VF.einsum(equation, operands)
RuntimeError: Could not run 'aten::mul.Tensor' with arguments from the 'QuantizedCPU' backend. 'aten::mul.Tensor' is only available for these backends: [CPU, CUDA, MkldnnCPU, SparseCPU, SparseCUDA, Named, Autograd, Profiler, Tracer, Batched].
</code></pre>
<p>Can <code>torch.einsum(...)</code> be quantized? Would there be a workaround since it consists of mulitplication and addition?</p>

username: ptrblck
cooked: <p>Unfortunately, I’m not experienced enough using the quantization package, so we would need to wait for an expert. <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>

username: Vasiliy_Kuznetsov
cooked: <aside class="quote no-group" data-username="pintonos" data-post="3" data-topic="94144">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pintonos/48/28210_2.png" class="avatar"> pintonos:</div>
<blockquote>
<p><code>einsum</code></p>
</blockquote>
</aside>
<p>hi <a class="mention" href="/u/pintonos">@pintonos</a>, currently we don’t have a quantized kernel for <code>einsum</code>, we would be happy to review a PR if someone is interested in implementing.  In the meanwhile, a workaround could be to dequantize → floating point einsum → quantize.</p>