akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Test accuracy decreases with epochs for Contrastive learning

NitishJaiswal: <p>I’m using a model based on CNN backbone and Transformer as long term feature aggregator for video kind of dataset. The model performs well on same domain and I want to boost it’s performance using self-supervised learning approach utilising a huge dataset that I have.<br>
The accuracy on the smaller dataset is about 90%. When I try to pretrain this model on the larger dataset in a self-supervised fashion (using contrastive loss), initially the accuracy (on direct testing on the smaller dataset) is around 40% which is quite acceptable because of the domain gap in the two datasets, but instead of increasing, it decreases over epochs quite steadily until it reaches somewhere around 35%. I need some suggestions about how can I address this problem and possibly see a usual increasing graph for the accuracy.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/8/9804fc6329885eff88d9256203d4963300d42003.png" data-download-href="https://discuss.pytorch.org/uploads/default/9804fc6329885eff88d9256203d4963300d42003" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/8/9804fc6329885eff88d9256203d4963300d42003.png" alt="image" data-base62-sha1="lGPfYGTTpfdFdrsVSR9XtIxCJWz" width="690" height="499" data-dominant-color="3A3A3A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">694×502 24 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ANUMULA_CHAITANYA_SA: <p>please let me know if you found a solution, I am facing the same issue.</p>
About deprecated torch.set_default_tensor_type API

tye1: <p>Regarding to torch.set_default_tensor_type API, I know this API has been deprecated since PyTorch 2.1 according to <a href="https://pytorch.org/docs/stable/generated/torch.set_default_tensor_type.html" class="inline-onebox" rel="noopener nofollow ugc">torch.set_default_tensor_type — PyTorch 2.3 documentation</a>, however it is still available in latest PyTorch 2.3 drop and can be used. May I know if there is a plan when this deprecated API will be totally removed from any version of PyTorch release?</p>

ptrblck: <p>The common guideline is to remove deprecated functions after ~2 minor releases. However, <code>Variable</code>s are deprecated since PyTorch <code>0.4</code> and are still around in <code>2.4.0.dev20240506</code>:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
a = torch.autograd.Variable(torch.randn(1), requires_grad=True)
b = a * 2
b.backward()
print(a.grad)
# tensor([2.])
</code></pre>
<p>With that being said, you should plan on updating your code after 2 minor releases.</p>

tye1: <p>Thanks for the comments.<br>
Is there any particular reason that this API has been deprecated in 2.1 but still kept in 2.3 and seems no solid plan when it will be totally removed? Like for instance, is the perf impact discussed in <a href="https://github.com/pytorch/pytorch/issues/92701" rel="noopener nofollow ugc">set_default_device/torch.device has performance impact for non-factory functions · Issue #92701 · pytorch/pytorch (github.com)</a> be one of the possible reason?</p>
Problem: Pipeline Parallelism with distributed-rpc on Jetson Nano devices

Nishant_Madan: <p>Hi, i’m a student and currently working on pipelining models on edge devices.</p>
<p>My current setup is: 1 Master linux machine, 2 Jetson Nano as workers.<br>
I used the example on:<br>
<a href="https://pytorch.org/tutorials/intermediate/dist_pipeline_parallel_tutorial.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/intermediate/dist_pipeline_parallel_tutorial.html</a></p>
<p>I also trying to investigate the impact of split size (for parallel execution) on the performance.</p>
<p>I have couple of queries as I have problem connecting to the edge devices using their IP addresses. And later I also want to do the performance analysis for each split using Pytorch.</p>
AVX512 support - Illegal instruction (core dumped)

4TINI: <p>I’m building PyTorch from Source inside a Dockerfile on a PC with an <code>Intel® Xeon(R) W-2245</code> CPU. Everything works fine, I can import PyTorch smoothly. When running the same Docker image on a PC with a <code>13th Gen Intel® Core™ i7-1360P CPU</code> I get</p>
<pre><code class="lang-auto">Illegal instruction (core dumped)
</code></pre>
<p>when importing torch.</p>
<p>Here an extract of the Dockerfile</p>
<pre><code class="lang-auto"># Build PyTorch wheel with extra environmental variables for custom feature switch
RUN git clone --branch v2.1.0 --depth=1 --recursive https://github.com/pytorch/pytorch /tmp/pytorch

WORKDIR /tmp/pytorch
RUN export USE_NCCL=OFF &amp;&amp; \
    export USE_CUDA=ON &amp;&amp; \
    export USE_CUDNN=ON &amp;&amp; \
    export USE_MKLDNN=ON &amp;&amp; \
    export ATEN_CPU_CAPABILITY=avx2 &amp;&amp; \
    export CUDA_NVCC_EXECUTABLE=$(which nvcc) &amp;&amp; \
    export CUDA_HOME="/usr/local/cuda" &amp;&amp; \
    export USE_QNNPACK=OFF &amp;&amp; \
    export USE_PYTORCH_QNNPACK=ON &amp;&amp; \
    export USE_NATIVE_ARCH=ON &amp;&amp; \
    export USE_DISTRIBUTED=ON &amp;&amp; \
    export USE_TENSORRT=OFF &amp;&amp; \
    pip3 install --no-cache-dir -r requirements.txt &amp;&amp; \
    pip3 install --no-cache-dir scikit-build ninja &amp;&amp; \
    python3 setup.py bdist_wheel &amp;&amp; \
    cp dist/*.whl /opt &amp;&amp; \
    rm -rf /tmp/pytorch

RUN pip3 install --no-cache-dir --verbose /opt/torch*.whl
</code></pre>
GPU utilization always at 40%

GonzalesPy: <p>Whatever I do the GPU utilization is always at 40%, when training and when not training.</p>
<p>GPU: NVIDIA GeForceRTX 4060<br>
PyTorch version: Version: 2.3.0+cu121</p>
<p>This should boost the GPU utilisation to about 90% but it doesnt, what could be the reason?:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from torch.utils.data import DataLoader

import torchvision.models as models
import torchvision.datasets as datasets
import torchvision.transforms as transforms


model = models.resnet50()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)

dataset = datasets.FakeData(
    size=1000,
    transform=transforms.ToTensor())
loader = DataLoader(
    dataset,
    num_workers=1,
    pin_memory=True
)

model.to('cuda')

if __name__ == '__main__':
    for data, target in loader:
        data = data.to('cuda', non_blocking=True)
        target = target.to('cuda', non_blocking=True)
        optimizer.zero_grad()
        output = model(data)
        loss = criterion(output, target)
        loss.backward()
        optimizer.step()
    
    print('Done')
</code></pre>

JosueCom: <p>That seems okay for the size of the batch/data and model you are using. Remember some stuff happens on the CPU so the GPU isn’t always betting utilized. For example, the loader operates in the CPU</p>
VSCode Jupyter Kernel crashes when running transforms.toTensor()

jharna: <p>I am trying to prepare the dataloader for fine tuning a pretrained classification model. The path to training images are added to a variable called ‘data_paths’. I am not able to resolve a “Kernel crashed” error in my jupyter notebook. I believe its happening when I call the ‘transforms.ToTensor’ method on the images. Below is a sample code which throws the error for me. It works if I comment out the ToTensor transformation</p>
<p>'trans = transforms.Compose([<br>
transforms.Resize((224, 224)),<br>
transforms.ToTensor()<br>
])</p>
<p>img = Image.open(data_paths[5])<br>
demo_img = trans(img)’</p>
<p>I’m looking for some suggestions on fixing the issue. I am using Python 3.10.12 and I have installed the latest pytorch and torchvision, but the issue hasn’t resolved</p>
A different usage of torch.nn.Linear()

floatingpale: <p>I initialize a module M=nn.Linear(a,b,bias=True).<br>
But I don’t use it as a fully connected layer to accept an input and project it into the output.<br>
I view the module M as a standalone entity with parameters M.weight and M.bias, which is like a “weight factory”.<br>
Then I use M.weight to do operations, which is just like I instead initialize with nn.Parameter().<br>
What is the difference between nn.Linear() and nn.Parameter() where we both treat them as weight factories.</p>

ptrblck: <aside class="quote no-group" data-username="floatingpale" data-post="1" data-topic="203602">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/floatingpale/48/70355_2.png" class="avatar"> floatingpale:</div>
<blockquote>
<p>What is the difference between nn.Linear() and nn.Parameter() where we both treat them as weight factories.</p>
</blockquote>
</aside>
<p>There is no difference since internally <code>nn.Linear</code> will also use its parameters in the same way by calling into the functional API via: <code>F.linear(input, self.weight, self.bias)</code> as seen <a href="https://github.com/pytorch/pytorch/blob/c40408850a9e6d529aa2437cd2d6fbb54aa9afa1/torch/nn/modules/linear.py#L116">here</a>. Creating the module would allow you to directly call the forward pass but won’t change any other behavior.</p>

floatingpale: <p>Thank you very much! And I want to ask further that if I only use the weight in nn.Linear() but ignore the bias(set bias=True), how the update will be influenced? Since in my experiment, the performance of initialization with nn.Linear() and nn.Parameter() are different. Thank you again!</p>

ptrblck: <aside class="quote no-group" data-username="floatingpale" data-post="3" data-topic="203602">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/floatingpale/48/70355_2.png" class="avatar"> floatingpale:</div>
<blockquote>
<p>And I want to ask further that if I only use the weight in nn.Linear() but ignore the bias(set bias=True), how the update will be influenced?</p>
</blockquote>
</aside>
<p>Only the <code>weight</code> parameter will receive a gradient in its <code>.grad</code> attribute and will thus be updated. The <code>bias.grad</code> will be <code>None</code> (assuming the gradient was never computed before) and this parameter will thus not be updated (unless the optimizer has updated this parameter before and is using running internal stats).</p>

floatingpale: <p>Thank you sir! It really helps me a lot!</p>
Aten::copy_ not safety when copy tensor from cpu to device

ryan_cai: <p>I have recently been reading the implementation of the PyTorch <code>copy_</code> operator. The link is: <a href="https://github.com/pytorch/pytorch/blob/v2.1.0/aten/src/ATen/native/cuda/Copy.cu" rel="noopener nofollow ugc">PyTorch GitHub - Copy.cu</a>. My understanding is as follows:</p>
<ol>
<li>When copying a CPU tensor to a device, it seems that the CPU tensor may be released prematurely, which could potentially cause the <code>copy_</code> operator to execute incorrectly.</li>
<li>When the CPU tensor is in pinned memory, the code at <a href="https://github.com/pytorch/pytorch/blob/v2.1.0/aten/src/ATen/native/cuda/Copy.cu#L256C5-L256C37" rel="noopener nofollow ugc">PyTorch GitHub - Copy.cu#L256C5-L256C37</a> will take effect and ensure that the CPU tensor is released only after it has been used, thus ensuring the correctness of the <code>copy_</code> operator.</li>
</ol>
<p>My question is: Is there really a bug with copying a CPU tensor to a device?</p>
<p>Here is my test code.</p>
<pre><code class="lang-auto">import torch

def copy_tensor(device_tensor):
    cpu_tensor = torch.empty(10000, 10000, dtype=torch.float32, pin_memory=False)
    device_tensor.copy_(cpu_tensor, non_blocking=True)


def main():
    device_tensor = torch.empty(10000, 10000, dtype=torch.float32, device='cuda')
    copy_tensor(device_tensor)


if __name__ == "__main__":
    main()
</code></pre>

ptrblck: <p>Do you see any issues or is this a theoretical concern?</p>

ryan_cai: <p>It is a theoretical concern. I tried to write some code to reproduce the issue, but I was not successful.</p>
<p>The code is as follows:</p>
<pre><code class="lang-auto">import torch

def copy_tensor(device_tensor, val):
    cpu_tensor = torch.empty(10000, 10000, dtype=torch.float32, pin_memory=False).fill_(val)
    device_tensor.copy_(cpu_tensor, non_blocking=True)


def main():
    tensor_list = []
    for i in range(50):
        device_tensor = torch.empty(10000, 10000, dtype=torch.float32, device='cuda')
        copy_tensor(device_tensor, float(i))
        tensor_list.append(device_tensor)
    
    for i in range(50):
        assert torch.all(tensor_list[i] == float(i)), f"i = {i}"


if __name__ == "__main__":
    main()
</code></pre>

ryan_cai: <p>Could the expert who is familiar with the PyTorch copy_ operator help me take a look at this issue?</p>

ryan_cai: <p>There is some discussion at <a href="https://github.com/pytorch/pytorch/issues/127612" class="inline-onebox" rel="noopener nofollow ugc">[Bug] Data on CPUs Are Not Synchronized Before Subsequent Operations · Issue #127612 · pytorch/pytorch · GitHub</a> which solved my doubts.</p>
Gradient accumulation loss compute

602389789: <p>Hello, everyone!<br>
Suppose we have data [b,s,dim], I recently noticed that CrossEntropyLoss is (1) computed the average on all tokens (b * s) in a batch instead of (2) computing on each sentence and then compute the average.<br>
Here is the code to compute loss for hugging_face transformers BertLMHeadModel</p>
<pre><code class="lang-auto">        sequence_output = outputs[0]
        prediction_scores = self.cls(sequence_output)

        lm_loss = None
        if labels is not None:
            # we are doing next-token prediction; shift prediction scores and input ids by one
            shifted_prediction_scores = prediction_scores[:, :-1, :].contiguous()
            labels = labels[:, 1:].contiguous()
            loss_fct = CrossEntropyLoss()
            lm_loss = loss_fct(shifted_prediction_scores.view(-1, self.config.vocab_size), labels.view(-1))
</code></pre>
<p>I know that  (1) and (2) have no difference in this situation. But when we apply gradient accumulation, I think the situation is different.</p>
<p>Suppose I have a batch_size 4, and the lengths of 4 sentences are 100,200,300,400.<br>
With batch_size 4, the loss is computed on the average of total 1000 tokens.</p>
<p>But with batch_size = 1 and gradient accumulation = 4, I think the loss is different.  We first compute the loss on each sentence separately and then compute the average, which means for the sentence of 100 tokens, we compute the loss average of 100 tokens and then divide by 4 and add it to total loss the same for others 3 sentences, and I think the loss computed this way is different from that computed with batch_size =4.</p>
<p>Did I misunderstand something?</p>

ptrblck: <p>If you want to reduce the loss in a custom way, use <code>reduction="none"</code> while initializing the criterion, and apply your custom reduction afterwards.</p>

602389789: <p>I noticed that most of the gradient accumulation implementation did not take this into consideration,</p>
<p>for example , the huggingface transformers trainer</p>
<pre><code>    if self.args.gradient_accumulation_steps &gt; 1:
        loss = loss / self.args.gradient_accumulation_steps
</code></pre>
<p>where loss is a normal CE loss from micro batch.</p>
<p>Dose this means these different behavior on computing loss will not influence the performance?</p>
An error with torch.distributed: RANK expected, but not set

MartinZhang: <p>Hi,</p>
<p>Today I try the DDP to forward my module, But I found this error.</p>
<p>When I except this code:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.distributed.is_nccl_avilable()
True
&gt;&gt;&gt; torch.distributed.init_process_group(backend='nccl')
</code></pre>
<p>there is an error like this:</p>
<pre><code class="lang-auto">ValueError: Error initializing torch.distributed using env:// rendezvous: environment variable RANK expected, but not set
</code></pre>
<p>How to deal it.</p>

MartinZhang: <p>I found a good way to deal this error:</p>
<p>as follows:</p>
<pre><code class="lang-auto">python -m torch.distributed.launch mytestfile.py
</code></pre>
<p>but why?</p>

guotong1988: <p>how to put ‘<code>torch.distributed.launch</code>’ into py file?</p>
Conditional indexing in Pytorch C++

pbaggens: <p>I have tried unsuccessfully for several days to implement a simple indexing task for<br>
Pytorch in C++.  Here is a trivial example.  In MATLAB, I use:</p>
<p>x=randn(5,5); % create 5x5 random matrix<br>
z=exp(x);             % create other matrices of same type and shape as x<br>
ii=find(x&gt;10);  % indexes to valuesgreater than 10<br>
z(ii) = 1./x(ii);     % get and set values at specified indexes,<br>
x(ii) = z(ii);         % while re-using indexes “ii” to save time.</p>
<p>In Python:</p>
<p>z=np.exp(x)<br>
ii = np.where(x &gt; 10)[0]<br>
z[ii] = 1/x[ii]<br>
x[ii] = z[ii]</p>
<p>Here is my C++ implementation that does not work:</p>
<pre><code class="lang-auto">torch::Tensor test_fn(torch::Tensor x) {
    using namespace torch::indexing;
    auto z =  x.exp();
    auto mask = x&gt;10;
    // This works, but does not allow me to re-use the index:
    //     z.index_put_({mask}, 1/x.index({mask}));

    // auto ii = mask.nonzero();  // tried this
    // ii = at::nonzero(mask);   // tried this too
    auto ii = torch::nonzero(mask).squeeze();
    // z.index_put_({ii}, 1/x.index({ii}));  // tried this
    z.index_put_({ii}, 1/x.index_select(0,torch::tensor(ii)));
    return z;
}
````Preformatted text`


Can someone help me with this? Note that it is important that I have
the required indexes, so I can re-use them. The number of elements satisfying
the condition are relatively few, and the operations required on these few indexes
is large (not like in my trivial example). So it is important to do the operations
only on those indexes and re-use the indexes.</code></pre>

ptrblck: <p>Could you describe the issue in more detail and what exactly is failing?<br>
Based on your comments it seems:</p>
<pre data-code-wrap="python"><code class="lang-python">// This works, but does not allow me to re-use the index:
</code></pre>
<p>you are unable to re-use the indices? Do you see an error?</p>

pbaggens: <p>Wheh I use the line:<br>
z.index_put_({mask}, 1/x.index({mask}));<br>
I get the desired result, but I do not have a set of indexes, analogous to “ii” in the MATLAB code I provided.</p>

ptrblck: <p>Would <code>torch::where</code> work as seen in the corresponding numpy example?</p>

pbaggens: <p>Torch where and numpy where are fundamentally different. torch::where is like an if/else, but numpy.where returns indexes to the elements of a boolean vector that are true.  Although I have working code using  index_put_ , I still would like to know how to implement MATLAB “find”, or else Python “where” in torch or C++, whereby I gt a list of the elements that are nonzero or true.</p>
Torch doesnt recognize gpu

ofek_Shafir: <p>i need torch to use my gpu but no matter what i do it doesnt recognize it with torch.cuda.is_available() i get false<br>
I tried reinstalling pytorch with:<br>
conda uninstall pytorch torchvision torchaudio cudatoolkit<br>
conda install pytorch torchvision torchaudio cudatoolkit=11.2 -c pytorch -c conda-forge<br>
I don’t know what to do anymore</p>

jiversivers: <p>May be a dumb question, but are you sure you have a GPU that is supported?</p>

ptrblck: <aside class="quote no-group" data-username="ofek_Shafir" data-post="1" data-topic="203978">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ofek_shafir/48/70504_2.png" class="avatar"> ofek_Shafir:</div>
<blockquote>
<p>conda install pytorch torchvision torchaudio cudatoolkit=11.2 -c pytorch -c conda-forge</p>
</blockquote>
</aside>
<p>Where did you get this command from?<br>
I don’t think the PyTorch binaries were build with 11.2, so you should stick to the supported install commands from our <a href="https://pytorch.org/get-started/locally/">website</a>.</p>
Concatenate unequal size tensors, do operation and then split them back

sprakashdash: <p>Lets say I have 3 tensors with shape <code>[B, S, d_0], [B, S, d_1], [B, S, d_2]</code>.<br>
I concat them and the resulting size is <code>[B, S, d_0+d_1+d_2]</code><br>
Now I need to perform <code>torch.matmul</code> with the transposed version of itself resulting in <code>[B, d_0+d_1+d_2, d_0+d_1+d_2]</code> size tensor.<br>
Then I need to get back only the blocks <code>[B, d_0, d_0], [B, d_1, d_1], [B, d_2, d_2]</code></p>
<p>Is there an efficient way to do these operations?</p>

ptrblck: <p>You could write these operations directly as you’ve described them and let <code>torch.compile</code> try to optimize them. With that being said, Your code would use a few metadata manupilations and should launch two kernels (<code>torch.cat</code> and the <code>matmul</code>), so unsure what <code>torch.compile</code> can optimize.</p>
What is torch.cuda.empty_cache do and where should i add it?

chao_xu: <p>hi guys, i am new to pytorch and i was wondering what will torch.cuda.empty_cache() do? and where should i add it?<br>
for example,</p>
<pre><code class="lang-auto">for epoch in range(epochs):
   train(......)
   torch.cuda.empty_cache()
   val(....)
</code></pre>
<p>or</p>
<pre><code class="lang-auto">for epoch in range(epochs):
   train(......)
   val(....)
   torch.cuda.empty_cache()
</code></pre>

albanD: <p>Hi,</p>
<p>This is a low level function to finely control some of the internals.<br>
You don’t need to use it as it will slow down your code for no gain !</p>

Albertma: <p>really?<br>
many students think it is a method to speed up training??</p>

ptrblck: <p>It’s not, as it will synchronize your code and free all cached memory. Since no memory is in the cache, the next allocations will again synchronize your code during the <code>cudaMalloc</code> calls and thus cause slowdowns in your code.</p>
Pytorch Geometric, sparse matrix and dense vector multiplication

alireza378: <p>I want to multiply laplacian matrix of graph by a vector which is a graph signal, suppose I have a graph with N nodes and E edges, and I define a signal (vector) on graph nodes.<br>
I want to multiply Lx where L is graph laplacian matrix and x is graph signal.<br>
Given that graph is large, it is expensive to multiply dense L by x. so I am curious whether there is a way I can multiply sparse L by graph signal x?</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/alireza378">@alireza378</a>,</p>
<p>Can you share a minimal reproducible example?</p>
<p>Also, if you’re trying to calculate a matrix-vector product, depending on how your matrix is computed you might be able to perform this matrix-vector product via a Jacobian-vector product (via <code>torch.func.jvp</code>) without ever fully materializing the Laplacian matrix.</p>
Qlinear (ONEDNN): data type of input should be QUint8

fbernhar: <p>Hello,</p>
<p>why do i have to choose the activation quantization of QAT to be quint8 ? Is there a way to get passed that ?</p>
<p>I used the qconfig of one of the people here from the forum:<br>
activation_bitwidth = 8 <span class="hashtag-raw">#whatever</span> bit you want<br>
bitwidth = 8 <span class="hashtag-raw">#whatever</span> bit you want</p>
<p>fq_activation = torch.quantization.FakeQuantize.with_args(observer=torch.quantization.MinMaxObserver.with_args(<br>
quant_min=-(2 ** bitwidth) // 2,<br>
quant_max=(2 ** bitwidth) // 2 - 1,<br>
dtype=torch.qint8,<br>
qscheme=torch.per_tensor_symmetric,<br>
reduce_range=False,))</p>
<p>fq_weights = torch.quantization.FakeQuantize.with_args(<br>
observer = torch.quantization.MinMaxObserver.with_args(<br>
quant_min=-(2 ** bitwidth) // 2,<br>
quant_max=(2 ** bitwidth) // 2 - 1,<br>
dtype=torch.qint8,<br>
qscheme=torch.per_tensor_symmetric,<br>
reduce_range=False,))</p>
<p>intB_qat_qconfig = torch.quantization.QConfig(activation= fq_activation,weight = fq_weights)</p>

jerryzh168: <p>this is a limitation of the backend (quantized engine) you are lowering to I think, do you want qint8 instead? you can try using XNNPACK instead. if you set <code>torch.backends.quantized.engine = "qnnpack"</code> you should be able to run the model that’s quantized with qint8</p>

MJITG_LV: <pre><code class="lang-auto">RuntimeError: QNNPACK only supports conv2d_unpack right now.
</code></pre>
<p>Is there any other way to deal with this problem? Since I have layers such as Conv3D and ConvBNReLU in my model…</p>