Empty_strided not supported on quantized tensors

username: lucala
cooked: <p>I’m trying to get the following code to work:</p>
<pre><code class="lang-auto">from torch.nn.modules import Module
class Sine(Module):
    def forward(self, z):
        return torch.sin(100 * z)
        
act_fn = Sine()
m = nn.Sequential(
    nn.Linear(1, 256, bias=True),
    act_fn,
    nn.Linear(256, 256, bias=True),
    act_fn,
    nn.Linear(256, 1, bias=True),
)

m = nn.Sequential(torch.ao.quantization.QuantStub(), 
                  *m, 
                  torch.ao.quantization.DeQuantStub())

m.train()
m.qconfig = torch.ao.quantization.get_default_qconfig('x86')
torch.ao.quantization.prepare_qat(m, inplace=True)
m.cuda()
n_epochs = 500
optim = torch.optim.AdamW(lr=1e-4, params=m.parameters(), weight_decay=1e-5)
for epoch in tqdm(range(n_epochs)):
    x = timestamps
    out = m(x)
    loss = F.mse_loss(out, ground_truth)
    optim.zero_grad()
    loss.backward()
    optim.step()

m.eval()
m.cpu()
torch.ao.quantization.convert(m, inplace=True)
res = m(timestamps.cpu())
</code></pre>
<p>The code runs until <code>res = m(timestamps.cpu())</code>. There, I get the error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "qat.py", line 132, in &lt;module&gt;
    res = m(model_input.cpu())
  File "anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/container.py", line 217, in forward
    input = module(input)
  File "anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "qat.py", line 94, in forward
    return torch.sin(100 * z)
RuntimeError: empty_strided not supported on quantized tensors yet see https://github.com/pytorch/pytorch/issues/74540
</code></pre>
<p>How do I solve this? I’d really like to test quantization aware training on this net…</p>

username: lucala
cooked: <p>So it seems there are two issues: multiplying <code>z</code> by 100 throws the empty_strided not supported error.</p>
<p>If I remove this and only use <code>torch.sin(z)</code> I get the error that torch.sin is not implemented in QuantizedCPU backend.</p>

username: jcaip
cooked: <p>Hi <a class="mention" href="/u/lucala">@lucala</a>,</p>
<p>I don’t think <code>torch.sin</code> is implemented for QuantizedTensor, so that is expected. If the model works on <code>cuda</code> maybe try moving the timestamps to <code>cuda</code> and then moving the result to <code>cpu</code>.</p>
<p>I am not sure exactly what is causing the empty_strided not supported error.</p>

username: HDCharles
cooked: <p>Yeah the issue is that torch.sin isn’t implement for QuantizedTensors, if you replace the sine with ReLU or something that is supported, the code works without issue.</p>
<p>Similarly multiplying quantized tensors with arbitrary scalars isn’t supported.</p>

username: Miguel_Campos
cooked: <p>Hello there.</p>
<p>I am dealing with the same issue with the operation <code>r = 1.2 * torch.atan(u)</code><br>
And this is the error:</p>
<p><code>RuntimeError: empty_strided not supported on quantized tensors yet see https://github.com/pytorch/pytorch/issues/74540</code></p>
<p>I am stuck on this. Could someone share any guidelines to multiply these values?</p>
<p>FYI: <code>**u**</code> is a tensor</p>

username: jcaip
cooked: <p>Hi <a class="mention" href="/u/miguel_campos">@Miguel_Campos</a>,</p>
<p>As we mentioned above, torch.sin and torch.atan are not implemented yet for QuantizedTensors. You can convert the quantized representation to it’s float form using a DeQuantStub and then do your atan and then call into QuantStub to requantize the tensor.</p>

username: Miguel_Campos
cooked: <p>Thanks, I will give it a try =)</p>