akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Can torchvision transforms/augmentations be applied after normalization?

Sam_Lerman: <p>Or must <code>Normalize(mean, std)</code> always be applied last?</p>

ptrblck: <p><code>torchvision.transform</code> accept tensors as well so you could apply some of them after normaliying the tensor.<br>
However, I would use transformations designed to change the actual pixel values of images (e.g. random color transforms) before convertig the image to a tensor.</p>
Quantization backend(fbgemm) is installed automatically during pip install?

jhw4824: <p>Hello.</p>
<p>I’ve been trying to do quantization using torch quant lib.<br>
I read this contents that we should set a proper backend qconfig(e.g. fbgemm for x86).<br>
In the official tutorial, there isn’t any guide about installing of the backend.<br>
I thought the backend(fbgemm) is installed automatically during pip install (torch or something else), is this right?<br>
Or, I should install the fbgemm on my linux server manually like this?(<a href="https://github.com/pytorch/FBGEMM" rel="noopener nofollow ugc">https://github.com/pytorch/FBGEMM</a>)</p>
<p>Thanks in advance for your help.</p>

andrewor: <p>Hi Je,</p>
<p>FBGEMM is a third party library of pytorch and should already be automatically installed when you install pytorch, so you don’t have to manually install it separately. You should be able to use pytorch quantization out of the box.</p>
<p>Best,<br>
-Andrew</p>
Pytorch example gives error. (Sub-class of torch.autograd.Function to ONNX)

Hatem: <p>I am trying to convert a custom function to ONNX that is a subclass of torch.autograd.Function. When I try to run the example provided here (<a href="https://pytorch.org/docs/stable/onnx.html#static-symbolic-method" class="inline-onebox" rel="noopener nofollow ugc">torch.onnx — PyTorch 1.11.0 documentation</a>), I get this error</p>
<pre><code class="lang-auto">IndexError: Invalid attribute specifier 'min' names  must be suffixed with type, e.g. 'dim_i' or 'dims_i'

</code></pre>
<p>Any ideas how to make it work?<br>
Thank you.</p>
No loss output when using model.eval() for validation

benduvi20: <p>Hi, I am currently using a Faster R-CNN pretrained model to detect bounding boxes on chest x-rays, I understand that for validation and testing i need to put my model in the model.eval() mode first in order for the dropout and bn layers to behave appropriately. However, when using model.eval() the model does not output the loss only predictions. Below is the code for my testing:</p>
<pre><code class="lang-auto">model = _get_detection_model(num_classes)
model.load_state_dict(torch.load(base_path + "model.pt", map_location=torch.device('cpu')))
model.eval() # or model.train()

with torch.no_grad():
    for batch_i, (batch_images, batch_targets) in enumerate(data_loader_test):

        image = list(img.to(device) for img in batch_images)
        target = [{k: v.to(device) for k, v in t.items()} for t in batch_targets]

        model_out = model(image, target)
        print(model_out)
</code></pre>
<p>With model.train() I get:</p>
<pre><code class="lang-auto">{'loss_classifier': tensor(0.0107), 'loss_box_reg': tensor(0.0981), 'loss_objectness': tensor(0.0008), 'loss_rpn_box_reg': tensor(0.0045)}
</code></pre>
<p>With mode.eval() I get:</p>
<pre><code class="lang-auto">[{'boxes': tensor([[ 58.0476,   0.0000, 245.1829, 256.0000],
                   [ 10.4306,   0.0000, 168.1798, 256.0000]]), 'labels': tensor([1, 1]), 'scores': tensor([0.9138, 0.9129])}]
</code></pre>
<p>How can I get the model to output both the loss and the predictions?</p>
Seq2Seq - same generation all batches

AfonsoSalgadoSousa: <p>Hi. I have an LSTM-based sequence-to-sequence model for paraphrase generation that seems to train (the loss goes down), but the generations are not good. For each epoch, the generations are all the same. Even though these seem to improve throughout the epochs, they plateau in some random arbitrary sentence and this sentence is the prediction for all entries of the batch.<br>
My generation function is as follows:</p>
<pre><code class="lang-auto">def greedy_search(self, decoder_input, state=None):
        batch_size = len(decoder_input)

        seqs = torch.zeros((batch_size, self.max_sequence_length))
        for t in range(self.max_sequence_length):
            words, _, state = self.decode_step(decoder_input, state, k=1,)
            seqs[:, t] = words[:, 0]
            decoder_input = words.tolist()

        return seqs.int()
</code></pre>
<p>My decode step is as follows:</p>
<pre><code class="lang-auto">def _decode_step(
        self,
        input_list,
        state_list,
        k=1,
    ):

        device = next(self.decoder.parameters()).device
        inputs = torch.stack([torch.tensor(inp).to(device) for inp in input_list])

        state = State().from_list(state_list)
        logits, new_state = self.forward_decoder(inputs, state=state)

        logits = logits.select(1, -1).contiguous()
        logprobs = F.log_softmax(logits, dim=-1)
        logprobs, words = logprobs.topk(k, dim=-1)
        new_state_list = [new_state[i] for i in range(len(input_list))]
        return words, logprobs, new_state_list
</code></pre>
<p>Do you have any suggestions on what may be causing this issue? Thanks in advance for the help.</p>
Dice loss for multiclass text classification 1D

sherouk_elsayed: <p>I use bert model for multi level text classification (6 classes) batch_size=256<br>
pred output for single post=[0.6 0.4 0.8 0.2 0.3 0.1] dim for batch=(256,6)<br>
true output =2 for single post    dim for batch=(256)<br>
I want to use dice_loss so I found this code<br>
from mxnet import nd, np</p>
<p>import numpy as np</p>
<p>smooth = 10</p>
<p>def dice_loss(y_pred, y_true):</p>
<pre><code>product = np.multiply(y_pred, y_true)

intersection = np.sum(product)

coefficient = (2.*intersection +smooth) / (np.sum(y_pred)+np.sum(y_true) +smooth)

loss = 1. - coefficient

# or "-coefficient"

return (torch.tensor(loss, requires_grad=True))
</code></pre>
<p>but it seems for binary classification not multi one<br>
so how to modify it to work for multi classification?</p>

mMagmer: <aside class="onebox githubblob" data-onebox-src="https://github.com/qubvel/segmentation_models.pytorch/blob/master/segmentation_models_pytorch/losses/dice.py">
  <header class="source">

      <a href="https://github.com/qubvel/segmentation_models.pytorch/blob/master/segmentation_models_pytorch/losses/dice.py" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/qubvel/segmentation_models.pytorch/blob/master/segmentation_models_pytorch/losses/dice.py" target="_blank" rel="noopener nofollow ugc">qubvel/segmentation_models.pytorch/blob/master/segmentation_models_pytorch/losses/dice.py</a></h4>


      <pre><code class="lang-py">from typing import Optional, List

import torch
import torch.nn.functional as F
from torch.nn.modules.loss import _Loss
from ._functional import soft_dice_score, to_tensor
from .constants import BINARY_MODE, MULTICLASS_MODE, MULTILABEL_MODE

__all__ = ["DiceLoss"]


class DiceLoss(_Loss):
    def __init__(
        self,
        mode: str,
        classes: Optional[List[int]] = None,
        log_loss: bool = False,
        from_logits: bool = True,
        smooth: float = 0.0,
        ignore_index: Optional[int] = None,
</code></pre>



  This file has been truncated. <a href="https://github.com/qubvel/segmentation_models.pytorch/blob/master/segmentation_models_pytorch/losses/dice.py" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
this implementation is for image segmentation with 2 spatial dim.<br>
but you should be able to use it by reshaping output  of your model and traget to have size NxCxHxW and NxHxW where is <code>W =1</code>  .</p>
How to sample k times by gumbel softmax

Anikily: <p>Hello,</p>
<p>I am trying to sample k elements from a categorical distribution in a differential way, and i notice that F.gumbel_softmax(logit, tau=1, hard=True) can return a one-hot tensor, but how can i sample t times using the gumbel sofmax, like topk function in pytorch.<br>
Thanks!</p>

mMagmer: <p>i don’t know but maybe you can interpolate logit with nearest neighbor mode and then give it to <code>gumbel_softmax</code>.<br>
this way you should not have any gradient issue.</p>
RuntimeError: "argmax_cuda" not implemented for 'Bool'

zimmer550: <p>I am getting this error when I run the following code:</p>
<pre><code class="lang-auto">score_flattened = score_.view(B * NP * SQ, B2 * NS * SQ)
target_flattened = target_.view(B * NP * SQ, B2 * NS * SQ)
target_flattened = target_flattened.argmax(dim=1)
</code></pre>
<p>This error only happens on Google Colab but not on my own laptop (which has windows 10 on it)</p>

ptrblck: <p>Which version are you using locally and on Google Colab?<br>
Could you update the Colab version, if it’s not the latest release?</p>

zimmer550: <p>Updated the colab version but I still get the same issue. Nevermind, I fixed the issue but just recasting my tensor as a float (before it was uint8)</p>

nlgranger: <p>I hit the same issue for the booleans, but not for bytes/uint8.<br>
This is my environment:</p>
<pre><code class="lang-auto">Collecting environment information...
PyTorch version: 1.3.1
Is debug build: No
CUDA used to build PyTorch: 10.0.130

OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
CMake version: Could not collect

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: GeForce GTX TITAN X
Nvidia driver version: 418.40.04
cuDNN version: Could not collect

Versions of relevant libraries:
[pip] numpy==1.17.3
[pip] torch==1.3.1
[pip] torchvision==0.4.2
[conda] blas                      1.0                         mkl  
[conda] mkl                       2019.4                      243  
[conda] mkl-service               2.3.0            py37he904b0f_0  
[conda] mkl_fft                   1.0.15           py37ha843d7b_0  
[conda] mkl_random                1.1.0            py37hd6b4f25_0  
[conda] pytorch                   1.3.1           py3.7_cuda10.0.130_cudnn7.6.3_0    pytorch
[conda] torchvision               0.4.2                py37_cu100    pytorch
</code></pre>
<p>I don’t have the issue with the pytorch 1.2/cuda 10.0 combo.</p>

zimmer550: <p>Recast your tensor to float:</p>
<p><code>my_tensor = my_tensor.double()</code></p>

Keeva: <p>I found this useful: <a href="https://stackoverflow.com/a/71449274/16733101" rel="noopener nofollow ugc">RuntimeError: “argmax_cuda” not implemented for ‘Bool’</a></p>
How to print value within __device__ function?

Ziyu_Huang: <p>In <strong>device</strong> or <strong>global</strong> function of GPU, it is hard to print out, we can use printf(%d",a); but this can not print out torch.tensor, because they are not one single value. So now I am using this code to print out:<br>
for (int i=0;i&lt;new_cell.size(0);i++ )<br>
for (int j=0;j&lt;new_cell.size(1);j++)<br>
printf("%f\n", new_cell[i][j]);<br>
Seems crazy… Any better suggestions?</p>
<p>Also I noticed a cuda debug tool Nsight, <a href="https://developer.nvidia.com/nsight-tools-visual-studio-integration" class="inline-onebox" rel="noopener nofollow ugc">NVIDIA Nsight Integration | NVIDIA Developer</a>    But not sure how to combine with .cu file with pytorch…Because here we are actually calling py file to run compiled cu file…?</p>
How to determine the largest batch size of a given model saturating the GPU?

zizhao.mo: <p>Hi,</p>
<p>I find that sometimes increasing the batch size will not increment the batch processing time significantly. For example, in this figure, the batch sizes smaller than 600 don’t impact the batch processing time somehow in V100.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/6/e/6ece8f18541af2785070982d0bf96169f28b3a7b.png" alt="image" data-base62-sha1="fOf4tSVC7dY1yZPZfHCuCqkd25B" width="245" height="208"></p>
<p>My question is, is it possible for me to calculate the maximum batch size that doesn’t change the batch processing time, given the GPU information and the model information trained in Pytorch? Which metric of GPU significantly impacts this saturation?</p>
<p>Thanks in advance.</p>

suraj.pt: <p>One thumb rule is to maximize your GPU’s memory usage. Run a simple binary search to identify this number. The GPU memory is occupied by not just the input data but also network weights (and activations if not using inside <code>torch.no_grad</code> or <code>torch.inference_mode</code>).</p>
<p>The two metrics that are commonly used during inference are <strong>throughput</strong> which denotes how many samples are inferenced in a unit time (you want this to be high), and <strong>latency</strong> the time taken to process a single sample (batch_sz = 1).</p>
<p>Does running a grid search of batch sizes and identifying the max batch size give you consistent results? Something like</p>
<pre><code class="lang-auto">def proc_time(b_sz, model, n_iter=10):
    x = torch.rand(b_sz, 3, 224, 224).cuda()
    torch.cuda.synchronize()
    start = time.time()
    for _ in range(n_iter):
        model(x)
    torch.cuda.synchronize()
    end = time.time() - start
    throughput = b_sz * n_iter / end
    print(f"Batch: {b_sz} \t {throughput} samples/sec")
    return (b_sz, throughput, )

</code></pre>
<p>Plotting these might give you a hump, take the maxima for most efficiency!</p>

zizhao.mo: <p>Thank you, Suraj,</p>
<p>Binary search does help me decrease the searching time. But I am still wondering if there is a graceful way to deterministically find such an inflection point for a given model type and GPU type?</p>
<p>By the way, I find that I forgot to post a figure. I have attached a figure illustrating such an inflection point now.</p>

ptrblck: <p>The proper way would be to profile the actual workload on your device.<br>
E.g. the lower bound visible in the attached figure could be the constant kernel launch overhead.<br>
Once you scale up the workload you might be seeing e.g. <a href="https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html#dim-quantization">quantization effects</a>.</p>

zizhao.mo: <p>Thank you. Profiling seems to be a more proper way in contrast to doing mathematics from a given model type and the hardware information.</p>
TypeError: object of type 'int' has no len() return len(self._index_sampler)

twisti14: <p>When running pytorch,I found this error</p>

ptrblck: <p>Based on the error message <code>self._index_sampler</code> is an <code>int</code> and thus the <code>len()</code> call fails. In case you get stuck, please post a minimal, executable code snippet to reproduce the issue.</p>
Accuracy in fisher discriminan analysis

S_M: <p>Hello all,<br>
I have a resnet network with a fisher discriminant analysis loss to update my weights. The network use for feature learning like as triplet loss. I have two questions</p>
<ol>
<li>How can I compute accracy here? As the last layer may represent features which learners with fisher.</li>
<li>The loss become to negative during training. Is it true here?</li>
</ol>
<p>Thanks in advance</p>
Can Tensors on different devices add together?

Danil_Hu: <p>Hi, everyone!</p>
<p>I found a curious stuff recently. As far as I know, when you want to do some operations on two tensors, you should make sure that they are on the same device. But when I write my code like this, it runs unexpectly well</p>
<pre><code class="lang-auto">import torch
a = torch.tensor(1, device='cuda')
print(a.device)
b = torch.tensor(2, device='cpu')
print(b.device)
torch(a+b)


cuda:0
cpu
tensor(3, device='cuda:0')
</code></pre>
<p>But it can’t work in my code like this:</p>
<pre><code class="lang-auto">pts_1_tile = torch.tensor([[0], [0]], dtype=torch.float32)
torch.add(pred_4pt_shift, pts_1_tile)
</code></pre>
<p><a href="https://i.stack.imgur.com/Pns4x.png" rel="noopener nofollow ugc"><img src="https://i.stack.imgur.com/Pns4x.png" alt="enter image description here" width="690" height="58"></a></p>
<p>Here <code>pred_4pt_shift</code> is an intermediate result from a sub-Net, and it is a tensor on GPU. My question is that why the first code can work but the second one reports this different device error?</p>

ptrblck: <p>In your use case <code>b</code> would be interpreted as a scalar value and should be forwarded as an argument to the CUDA kernel directly if I’m not mistaken.<br>
E.g. adding a dimension to <code>b</code> will break the code as seen here:</p>
<pre><code class="lang-python"># works
a = torch.tensor(1, device='cuda')
print(a.device)
b = torch.tensor(2, device='cpu')
print(b.device)
print(a+b)
print(torch.add(a, b))

# works
a = torch.tensor([1], device='cuda')
print(a.device)
b = torch.tensor(2, device='cpu')
print(b.device)
print(a+b)
print(torch.add(a, b))

# fails
a = torch.tensor(1, device='cuda')
print(a.device)
b = torch.tensor([2], device='cpu')
print(b.device)
print(a+b)
print(torch.add(a, b))
</code></pre>

Danil_Hu: <p>Thank you!<br>
So the right thing to do is that everytime a tensor is defined, I need to make sure that the tensor is on the same device as my model Right?</p>

ptrblck: <p>Yes, I would recommend to explicitly move the data to the desired device in order to avoid any surprising issues.<br>
However, as already described, <code>scalar</code> values are fine to use with tensors on e.g. a GPU:</p>
<pre><code class="lang-python">a = torch.randn(10, device='cuda')
print(a+2)
</code></pre>

Danil_Hu: <p>Thanks a lot! Have a nice day <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>
How to normalize image data from -1 to 1

Adam007: <p>Hi I have image with values from 0 to 255. What is best way to normalize this image to -1,1 ?</p>
<p>Thank you</p>

ptrblck: <p>This should work:</p>
<pre><code class="lang-python">x = torch.randint(0, 256, (100,)).float()
print(x.min(), x.max())
# &gt; tensor(0.) tensor(255.)
y = 2 * x / 255. - 1
print(y.min(), y.max())
# &gt; tensor(-1.) tensor(1.)
</code></pre>

Adam007: <p>Hi one more question, how can I get this normalization back now ? If I have image normalized between -1 an 1 and now I want back to 0 -255, iam trying to revert this equation but not working</p>

ptrblck: <p>You can just undo the normalization:</p>
<pre><code class="lang-python">x = torch.randint(0, 256, (100,)).float()
print(x.min(), x.max())
# &gt; tensor(0.) tensor(255.)
y = 2 * x / 255. - 1
print(y.min(), y.max())
# &gt; tensor(-1.) tensor(1.)

# undo normalization
z = (y + 1) / 2 * 255.
print(z.min(), z.max())
# &gt; tensor(0.) tensor(255.)
</code></pre>
Failed to build PyTorch from source

jiapei100: <p>I actually manage to have <strong>libtorch</strong> <strong>built and installed</strong> via <strong>CMake</strong> , our-of-source build under the folder named <strong>build</strong>.</p>
<p>I wonder while I build <strong>Pytorch</strong> using <code>python setup.py build -cmake</code>, is there a way to just use the configuration under the folder <strong>build</strong> already existing there ?</p>
<p>A similar topic is posted at <a href="https://discuss.pytorch.org/t/failed-to-build-pytorch-audio-from-source/144267">https://discuss.pytorch.org/t/failed-to-build-pytorch-audio-from-source/144267</a> . Thank you …</p>
<p>Cheers<br>
Pei</p>
RuntimeError: [enforce fail at CPUAllocator.cpp:68]

Samah_Abu_saleem: <p>I am new in Pytorch. I got this error when I was trying to train the GCN model on my custom dataset.</p>
<p>The error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_61/1753868253.py in &lt;module&gt;
     30 # Run for 200 epochs (range is exclusive in the upper bound)
     31 for epoch in range(1, 201):
---&gt; 32     train()
     33     train_acc = test(train_loader)
     34     test_acc = test(test_loader)

/tmp/ipykernel_61/1753868253.py in train()
     10     for data in train_loader:
     11       print('x_shape:',data.x.shape)
---&gt; 12       out = model(data.x.float(), data.edge_index, data.batch)
     13       loss = criterion(out, data.num_classes)
     14       loss.backward()

RuntimeError: [enforce fail at CPUAllocator.cpp:68] . DefaultCPUAllocator: can't allocate memory: you tried to allocate 8609587200 bytes. Error code 12 (Cannot allocate memory)
</code></pre>
<p>My code:</p>
<pre><code class="lang-auto">train_loader = DataLoader(trainset, batch_size=64, shuffle=True)
test_loader = DataLoader(testset, batch_size=64, shuffle=False)

# Define our GCN class as a pytorch Module
class GCN(torch.nn.Module):
    def __init__(self, hidden_channels):
        super(GCN, self).__init__()
        # We inherit from pytorch geometric's GCN class, and we initialize three layers
        self.conv1 = GCNConv(data.num_features, hidden_channels)
        self.conv2 = GCNConv(hidden_channels, hidden_channels)
        self.conv3 = GCNConv(hidden_channels, hidden_channels)
        # Our final linear layer will define our output
        self.lin = Linear(hidden_channels, 8)
        
    def forward(self, x, edge_index, batch):
      # 1. Obtain node embeddings 
      x = self.conv1(x, edge_index)
      x = x.relu()
      x = self.conv2(x, edge_index)
      x = x.relu()
      x = self.conv3(x, edge_index)   
 
      # 2. Readout layer
      x = global_mean_pool(x, batch)  # [batch_size, hidden_channels]
 
      # 3. Apply a final classifier
      x = F.dropout(x, p=0.5, training=self.training)
      x = self.lin(x)
      return x
    
model = GCN(hidden_channels=64)

</code></pre>

InnovArul: <aside class="quote no-group" data-username="Samah_Abu_saleem" data-post="1" data-topic="142666">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/samah_abu_saleem/48/43535_2.png" class="avatar"> Samah_Abu_saleem:</div>
<blockquote>
<p><code>DefaultCPUAllocator: can't allocate memory: you tried to allocate 8609587200 bytes.</code></p>
</blockquote>
</aside>
<p>The error message seems clear?<br>
That your model is needing more memory than the GPU memory size?</p>

lesnikow: <p><code>DefaultCPUAllocator</code> is not the same as e.g. <code>DefaultCUDAAllocator</code>.</p>
Help with custom dataloaders

zic: <p>Hello everyone, I need help</p>
<p>I have two datasets of images - indoors and outdoors, they don’t have the same number of examples.</p>
<p>Each dataset has images that contain a certain number of classes (minimum 1 maximum 4), these classes can appear in both datasets, and each class has 4 categories - red, blue, green, white. Example:<br>
Indoor - cats, dogs, horses<br>
Outdoor - dogs, humans</p>
<p>I am trying to train a model, where I tell it, “here is an image that contains a cat, tell me it’s color” regardless of where it was taken (Indoors, outdoors, In a car, on the moon)</p>
<p>To do that, I need to present my model examples so that every batch has only one category (cat, dog, horse or human), but I want to sample from all datasets (two in this case) that contains these objects and mix them. How can I do this?</p>
<p>It has to take into account that the number of examples in each dataset is different, and that some categories appear in one dataset where others can appear in more than one. and each batch must contain only one category.</p>
<p>I would appreciate any help, I have been trying to solve this for a few days now.</p>

ejguan: <p>Then, you need to create a dictionary to map each category to an array of image paths</p>

zic: <p>Thank you for your suggestion,<br>
Here is what I did (in case anyone tries to do this):<br>
(based on this <a href="https://stackoverflow.com/questions/71422639/how-to-load-data-from-multiply-datasets-in-pytorch" class="inline-onebox" rel="noopener nofollow ugc">How to load data from multiply datasets in pytorch - Stack Overflow</a>)</p>
<ol>
<li>concatenate the datasets (after translation)<br>
2.generate the indices corresponding to each dataset (1-15 dataset 1, 16-31 dataset 2, 32-40 dataset3 etc)</li>
<li>assign indices to each class (class1: 1-15, 32-40,  class2: 1-15, 16-32 etc) while making sure that the number of items in each class is divisible by the batch size</li>
<li>create shuffle the indices of each class using subsetrandomsampler and create for each a batchsampler</li>
<li>generate one array of all the indices of all classes by randomly sampling each batchsampler until they are depleted while recording the class order [class1, class3, class1, class2, class5]  inds[[],[],[],[],[]]</li>
<li>create a custom single sampler with these indices</li>
<li>create a custom batch loader using the sample<br>
while iterating over the loader, each batch come with its corresponding class</li>
</ol>
Building PyTorch for the Raspberry Pi (32 bits)

valmir.pretto: <p>Hello!</p>
<p>I am trying to build PyTorch 1.10.2 as a wheel for the ARM 32 bits architecture. More specifically, I want to be able to run PyTorch in a Raspberry Pi.</p>
<p>I am able to run the build process up until the 60% mark, where it fails with the following message:</p>
<pre><code class="lang-auto">/workdir/pytorch/third_party/XNNPACK/src/qs8-gemm/gen/1x8c4-minmax-fp32-neondot.c: In function ‘xnn_qs8_gemm_minmax_fp32_ukernel_1x8c4__neondot’:
/workdir/pytorch/third_party/XNNPACK/src/qs8-gemm/gen/1x8c4-minmax-fp32-neondot.c:94:18: warning: implicit declaration of function ‘vcvtnq_s32_f32’; did you mean ‘vcvtq_s32_f32’? [-Wimplicit-function-declaration]
     vacc0x0123 = vcvtnq_s32_f32(vproduct0x0123);
                  ^~~~~~~~~~~~~~
                  vcvtq_s32_f32
/workdir/pytorch/third_party/XNNPACK/src/qs8-gemm/gen/1x8c4-minmax-fp32-neondot.c:94:16: error: incompatible types when assigning to type ‘int32x4_t’ from type ‘int’
     vacc0x0123 = vcvtnq_s32_f32(vproduct0x0123);
                ^
</code></pre>
<p>What I understand from this error message is that the <code>vcvtnq_s32_f32</code> function, from XNNPACK, is missing. The default C behaviour is to implicitly declare a prototype with <code>int</code> as the return type, which is obviously not correct.</p>
<p>I found <a href="https://github.com/google/XNNPACK/issues/1924" rel="noopener nofollow ugc">this issue</a>, which led me to look around the PyTorch repo that I have checked out and the polyfill does exist, in the <code>pytorch/third_party/XNNPACK/src/xnnpack/intrinsics-polyfill.h</code> file:</p>
<pre><code class="lang-auto">// AArch32 GCC, see
// - https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71233
// - https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95399
#if XNN_ARCH_ARM &amp;&amp; defined(__GNUC__) &amp;&amp; !defined(__clang__) &amp;&amp; defined(__ARM_NEON__)
#include &lt;arm_neon.h&gt;

static XNN_INTRINSIC
int32x4_t vcvtnq_s32_f32(float32x4_t v) {
  return vcvtq_s32_f32(vrndnq_f32(v));
}
#endif  // AArch32 GCC
</code></pre>
<p>So, not only does the polyfill exist, but the affected file, <code>pytorch/third_party/XNNPACK/src/qs8-gemm/gen/1x8c4-minmax-fp32-neondot.c</code>, also includes the polyfill:</p>
<pre><code class="lang-auto">#include &lt;assert.h&gt;

#include &lt;arm_neon.h&gt;

#include &lt;xnnpack/gemm.h&gt;
#include &lt;xnnpack/intrinsics-polyfill.h&gt;
#include &lt;xnnpack/math.h&gt;
</code></pre>
<p>What can I do to make sure that XNNPACK finds and uses the polyfill? This is blocking me from being able to build PyTorch for the Raspberry Pi.</p>

bresam: <p>I run into exact same error, have you already found a solution?</p>

bresam: <p>I’ve switched to 64 bit Raspi OS which works for me.<br>
Torch is available for 64 bit and it works fine.</p>
How to transform a multi-hot tensor to a one-hot tensor?

Anikily: <p>for example,</p>
<pre><code class="lang-auto">[[1. 0. 1. 0. 0. 0.],
[0. 0. 1. 0. 1. 0.]]
</code></pre>
<p>and i want get</p>
<pre><code class="lang-auto">[[[1. 0. 0. 0. 0. 0.],
[0. 0. 1. 0. 0. 0.]],
[[0. 0. 1. 0. 0. 0.],
[0. 0. 0. 0. 1. 0.]]]
</code></pre>
<p>what is the elegant way to achieve this operation in pytorch?</p>

KFrank: <p>Hi Yu!</p>
<aside class="quote no-group" data-username="Anikily" data-post="1" data-topic="145297" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/a5b964/48.png" class="avatar"> Anikily:</div>
<blockquote>
<p>and i want get</p>
<pre><code class="lang-auto">[[[1. 0. 0. 0. 0. 0.],
[0. 0. 1. 0. 0. 0.]],
[[0. 0. 1. 0. 0. 0.],
[0. 0. 0. 0. 1. 0.]]]
</code></pre>
</blockquote>
</aside>
<p>Use <code>nonzero()</code> to find locations of the <code>1</code>s in your multi-hot tensor,<br>
and then (after some fiddling to get the indices in the right form) use<br>
<code>one_hot()</code> to produce your desired one-hot result:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.10.0'
&gt;&gt;&gt; multihot = torch.tensor ([[1, 0, 1, 0, 0, 0], [0, 0, 1, 0, 1, 0]])
&gt;&gt;&gt; count = multihot[0].sum()   # number of ones in first row                     &gt;&gt;&gt; assert  (multihot.sum (dim = 1) == count).all()   # verify same number of ones in each row
&gt;&gt;&gt; onehot = torch.nn.functional.one_hot (multihot.nonzero()[:, 1].view (multihot.size (0), count))
&gt;&gt;&gt; onehot
tensor([[[1, 0, 0, 0, 0],
         [0, 0, 1, 0, 0]],

        [[0, 0, 1, 0, 0],
         [0, 0, 0, 0, 1]]])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

Anikily: <p>Hi K.Frank!<br>
Thanks for your reply.<br>
But i notice that the shape of multihot is 2, 6, but the one hot is 2, 2, 5.<br>
the final dim is not equl in above.<br>
Best,<br>
Yu</p>

KFrank: <p>Hi Yu!</p>
<aside class="quote no-group quote-modified" data-username="Anikily" data-post="3" data-topic="145297" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/a5b964/48.png" class="avatar"> Anikily:</div>
<blockquote>
<p>Hi K.Frank!<br>
But i notice that the shape of multihot is 2, 6, but the one hot is 2, 2, 5.</p>
</blockquote>
</aside>
<p>Yes, you are correct.  That was my oversight.</p>
<p>We can explicitly tell <code>one_hot()</code> the number of classes, rather than<br>
having it “guess:”</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; onehot = torch.nn.functional.one_hot (multihot.nonzero()[:, 1].view (multihot.size (0), count), num_classes = multihot.size (1))
&gt;&gt;&gt; onehot
tensor([[[1, 0, 0, 0, 0, 0],
         [0, 0, 1, 0, 0, 0]],

        [[0, 0, 1, 0, 0, 0],
         [0, 0, 0, 0, 1, 0]]])
</code></pre>
<p>Note the use of <code>num_classes = multihot.size (1)</code>.</p>
<p>Best.</p>
<p>K. Frank</p>
How to unpack tuple in C++

AlphaBetaGamma96: <p>Hi all,</p>
<p>I’m trying to unpack a <code>std::tuple</code> in C++ however using <code>std::get</code> doesn’t work and I’m not sure why. Is there I’m missing?</p>
<pre><code class="lang-auto">#include &lt;torch/torch.h&gt;

int main(){
  int64_t B = 4096;
  int64_t D = 4;
  int64_t N = 6;
  at::Tensor matrices = torch::randn(B,D,N,N);

  std::tuple&lt;at::Tensor, at::Tensor&gt; out = torch::linalg::slogdet(matrices);
  at::Tensor sign = std::get&lt;0&gt;(out); //fails
}
</code></pre>
<p>Thank you!</p>
Pretrained FastRCNN from FasterRCNN

Vincent_Marie: <p>Hi everybody,</p>
<p>My goal is to generate FastRCNN inference with my own proposals. Unfortunately,  there is no implementation of this architecture in Pytorch so i decided to implement it by myself based on the FasterRCNN implementation.</p>
<p>I already tried to make my custom FastRCNN architecture based on FasterRCNN (torchvision.models.faster_rcnn) and GeneralizedRCNN (torchvision.models.generalized_rcnn) by removing the RPN parts and including the proposals parameter in the GeneralizedRCNN forward method.</p>
<p>I successfully loaded a resnet50 backbone but i didn’t found a way to load a pre-trained FastRCNNPredictor which is wrapped into the pre-trained model “fasterrcnn_resnet50_fpn_coco”.</p>
<p>Is it possible to extract the FastRCNNPredictor from a pre-trained model and to add it to my architecture ?</p>

ptrblck: <p>You should be able to directly access the predictor via:</p>
<pre><code class="lang-python">model.roi_heads.box_predictor.state_dict().keys()
# &gt; odict_keys(['cls_score.weight', 'cls_score.bias', 'bbox_pred.weight', 'bbox_pred.bias'])
</code></pre>
<p>and could thus try to load this <code>state_dict</code> into your new model’s submodule.</p>

Vincent_Marie: <p>Thank you for your response.<br>
I managed to successfully load backbone and box_predictor.</p>
<p>I have a general question. My final goal is to export the model in ONNX format in order to use it in C++ with OnnxRuntime. As i said, I based my custom FastRCNN architecture on FasterRCNN. When I export the model in ONNX format, should I expect that the post-processing parts (like box_regression applied to input proposals or box decoding as done in the FasterRCNN architecture) are also exported ? Or do I need to solely export the “processing parts” up to the box_predictor outputs and re-implement the post-processing operations in C++ ?</p>

ptrblck: <p>I’m not familiar enough with the ONNX export process for <code>FasterRCNN</code>, but I would assume that the “entire” model is exported as it’s passed to <code>torch.onnx.export</code>. I.e. if the post-processing steps are part of the model’s <code>forward</code> pass, I would assume they are also in the exported model.<br>
One thing that you could check is if any methods are decorated with <code>torch.jit.ignore</code>, which wouldn’t be scripted.</p>

Vincent_Marie: <p>Ok I will take a look.<br>
Thank you for your quick responses and for your work <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>
Calculate covariance matrix between two sample of the same and different classes

S_M: <p>Hello<br>
Suppose I have a batch with a size of 16 that contains some triplet samples (A, P, N). Where samples A and P are from the same class and the other sample N is from a different class. During the training process, I would like to calculate the covariance matrices Cov(A, P) and Cov(A, N). In each iteration the selection batches are random for example with labels 0 to 9 my first batch would like as:<br>
[[1, 1, 2],<br>
[1, 1, 0],<br>
[2, 2, 1],<br>
[9, 9, 5],<br>
[1, 1, 8],<br>
[2, 2, 4],<br>
[6, 6, 9],<br>
[3, 3, 0],<br>
[3, 3, 1],<br>
[3, 3, 5],<br>
[1, 1, 8],<br>
[6, 6, 7],<br>
[3, 3, 4],<br>
[5, 5, 1],<br>
[5, 5, 2],<br>
[5, 5, 8]]</p>
<p>Thanks</p>
Installing Pytorch with Conda installs CPU only version

Majedshakir: <p>Hi all,</p>
<p>I am trying to install pytorch 1.4 with torchversion 0.5 that are compatible with CUDA. Every time I install them I get “pytorch 1.40 py3.7_cpu_0 [cpuonly] pytorch” same thing for torchvision.</p>
<p>I have installed cuda 10.1 and it is working with my system. I have uninstalled and install PyTorch multiple time and I only get the cpu only.</p>
<p>I use the following command line “conda install pytorch torchvision cudatoolkit=10.1 -c pytorch”</p>
<p>Please advise and thank you for the help.</p>

Majedshakir: <p>I got the answer. I initially install a CPU only version this only install <code>cpuonly 1.0</code>. When I uninstall pytorch to install the cuda pytorch it didn’t remove <code>cpuonly 1.0</code>.</p>
<p>to fix it:</p>
<pre><code class="lang-auto">conda uninstall pytorch
</code></pre>
<pre><code class="lang-auto">conda uninstall cpuonly
</code></pre>
<p>Then install pytorch again normally</p>
<pre><code class="lang-auto">conda install pytorch torchvision cudatoolkit=10.1 -c pytorch
</code></pre>

Zoro_Roronoa: <p>Thanks very much for your answer! <code>conda uninstall cpuonly</code> is the key step.</p>

import_torch_as_tf: <p>nice!!! your answer solved it very well</p>
Finetuning with torch.nn.BatchNorm2d, running statistics changed with num_batches_tracked unchanged

pepper8362: <p>Hi,</p>
<p>I am fine-tuning from a trained model. To freeze BatchNorm2d layers, I set all of them to eval mode during training. But I find a strange thing. After a few epochs, I find <code>running_mean</code> and <code>running_var</code> have changed a bit (most of the change is very small; the absolute sum of the change is like 2e-4, but the largest one reach 1.65). Also, I find <code>num_batches_tracked</code> is not changed for all the BatchNorm2d layers.</p>
<p>I wonder why <code>running_mean</code> and <code>running_var</code> change, if no new batch statistics are accumulated? I am using Automatic Mixed Precision in training, could it be related to this? Any suggestions?</p>
<p>Thank you for your help.</p>

ptrblck: <p>This shouldn’t be the case. Could you post a minimal, executable code snippet to reproduce the issue?</p>

pepper8362: <p>The project is a bit complex. I’ll try to extract a minimal sample.</p>
Backward() twice

bingqing_liu: <pre><code class="lang-auto">a=nn.Parameter(torch.tensor([[2.]]))
b=a*2
c=b.sum()
c.backward()
print(c)
c.backward()#error
</code></pre>
<p>i know this will cause error,</p>
<p>Trying to backward through the graph a second time</p>
<p>but i have a question,the forward history still exists:</p>
<pre><code class="lang-auto">c:
tensor(4., grad_fn=&lt;SumBackward0&gt;)
</code></pre>
<p>why can not i backward twice?is there something secret in pytorch i do not konw?why pytorch not just compute gradient according to the forward history?</p>

ptrblck: <p>The intermediate activation tensors were already freed, which is causing the error.</p>

bingqing_liu: <p>if so,why pytorch still retain the grad_fn？after all,we can not backward anymore.</p>

ptrblck: <p>You can backward through parts which do not need the intermediate activations to calculate the gradient. Removing the <code>.grad_fn</code> would be unnecessarily strict as seen here:</p>
<pre><code class="lang-python">x = torch.randn(1, 1, requires_grad=True)
y = x * 2
w = torch.randn(1, 1, requires_grad=True)
z = y + w

z.mean().backward()
print(x.grad, w.grad)
# &gt; tensor([[2.]]) tensor([[1.]])

# z.mean().backward()
# &gt; RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

z.mean().backward(inputs=w)
print(x.grad, w.grad)
# &gt; tensor([[2.]]) tensor([[2.]])
</code></pre>
Incorrect Output for Bubble Sort Java

Ankit_Dixit: <p>Hello All, I am working on a small project and I have been working on this code for about a week; I am trying to make a program that generates random numbers and then sort them using the Bubble method, but I get this message “Bubble Sort: [I@ad3ba4”. Does anyone see what is wrong I feel like this is so simple but I just can’t find the problem. Any suggestion? I am using the online compiler to run the program here: <a href="https://www.interviewbit.com/online-java-compiler/" class="inline-onebox" rel="noopener nofollow ugc">Online Java Compiler | Java IDE Online - InterviewBit</a></p>
<pre><code class="lang-auto">import java.util.Random;
public class sortLibrary {
    private static void bubbleSort(int[] list) {
        int n = list.length;
        int temp = 0;
        for (int i = 0; i &lt; n; i++) {
            for (int j = 1; j &lt; (n - 1); j++) {
                if (list[j - 1] &gt; list[j - 1]) {
                    temp = list[j - 1];
                    list[j - 1] = list[j];
                    list[j] = temp;
                }
            }
        }
        System.out.println("\nBubble Sort: " + list);
    }
    public static void main(String args[]) {
        System.out.println("Unsorted list:");
        Random numbers = new Random();
        int list[] = new int[20];
        for (int i = 0; i &lt; 20; i++) {
            list[i] = numbers.nextInt(100);
        }
        for (int i = 0; i &lt; 20; i++) {
            System.out.print(list[i] + "  ");
        }
        bubbleSort(list);
    }
}
</code></pre>

ptrblck: <p>I think you might have posted in the wrong discussion board, as this one is focused on PyTorch, which is an ML framework in Python and C++. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>
Reproducibility of nn.Embedding?

Anikily: <p>I am using the nn.Embedding class to generate some word embeddings, and i have tried two setting in my code:<br>
1.<br>
def <strong>init</strong>(self):<br>
self.embedding = nn.Embedding(emb_idim, emb_odim)<br>
def forward(self, indexs):<br>
wemb = self.embedding(indexs)</p>
<ol start="2">
<li>
</li>
</ol>
<p>def <strong>init</strong>(self):<br>
self.embedding = nn.Embedding(emb_idim, emb_odim)<br>
def forward(self, indexs):<br>
query_one_hot = F.one_hot(indxes, max_num_of_indexes)<br>
wemb = query_one_hot.float() @ self.embedding.weight</p>
<p>It seems that the wemb results are equal in 1 and 2 settings after one run, but i got the diferrent testing performance after one epoch.</p>
<p>Could anyone can tell me what’s wrong with my code ?</p>

ptrblck: <p>I cannot reproduce the issue and get a zero error:</p>
<pre><code class="lang-python">device = 'cuda'

embedding1 = nn.Embedding(100, 100).to(device)
embedding2 = nn.Embedding(100, 100).to(device)
embedding2.load_state_dict(embedding1.state_dict())

for _ in range(10):
    indices = torch.randint(0, 100, (32,)).to(device)
    out1 = embedding1(indices)
    out2 = F.one_hot(indices, 100).float() @ embedding2.weight
    
    print('out abs err {}'.format((out1 - out2).abs().max()))
    out1.mean().backward()
    out2.mean().backward()
    print('grad abs err {}'.format((embedding1.weight - embedding2.weight).abs().max()))
    
    embedding1.zero_grad()
    embedding2.zero_grad()
</code></pre>
Are model archives [.mar] files CPU architecture specific?

mgiessing: <p>Hello, does anyone know if .mar files are architecture specific?</p>
<p>E.g. you train a model and create a .mar file on an x86 CPU and then you want to serve/deploy it on another architecture, e.g. ppc64le or ARM. Is that possible or can the .mar file only be served on x86 CPUs then?</p>
<p>To make it even more complex:<br>
Assuming that the previous would work (so no binaries are compiled) what about the endianess? Trained on x86 little endian and then serve/deploy on big endian systems - would that work?</p>
<p>Thanks!</p>
Torch.jit.trace() call forward() three times?

wadewang: <p>I happen to find that torch.jit.trace() will call forward() three times, run the code below:</p>
<pre><code class="lang-python">import torch

class MyCell(torch.nn.Module):
    def __init__(self):
        super(MyCell, self).__init__()
        self.linear = torch.nn.Linear(4, 4)

    def forward(self, x, h):
        print("execute forward")
        new_h = torch.tanh(self.linear(x) + h)
        return new_h, new_h

my_cell = MyCell()
x = torch.rand(3, 4)
h = torch.rand(3, 4)
traced_cell = torch.jit.trace(my_cell, (x, h))
</code></pre>
<p>The output is</p>
<pre><code class="lang-auto">execute forward
execute forward
execute forward
</code></pre>
<p>I also test other model, they all show call forward() three times, so why torch.jit.trace() need call forward() three times, and what is purpose of each time ? Thanks !</p>

ptrblck: <p>TorchScript will optimize the model and uses multiple forward passes to do so.</p>
Difficulty shaping 3 dimension tensors from MFCC transform for input into model

Brennan: <p>Hello!</p>
<p>I am using the LibreSpeech dataset and the MFCC transform from torchaudio to compute the MFCC coeffecients for each waveform.</p>
<p>The MFCC transform returns a tensor with the following shape:</p>
<p>1 x 24 x N</p>
<p>1 = the waveform<br>
24 = the number of MFCC coefficients<br>
N  = the number of time samples (I think?)</p>
<p>I then squeeze this and transpose it so it is now:</p>
<p>N * 24</p>
<p>During training I have tensors of shape:</p>
<p>5 * N * 24</p>
<p>Where 5 is the batch size I am using</p>
<p>How do I reshape this tensor during training or collation so that I can feed it into my Neural net which accepts 24 input_size and outputs 28 (my number of classes).</p>
<p>Many thanks!</p>

nateanl: <p>Hi <a class="mention" href="/u/brennan">@Brennan</a>, what’s the model structure of your neural net? If it’s composed of pure <code>Linear</code> layers, you can move the MFCC dimension to the last.<br>
Suppose your input Tensor <code>X</code> is of shape <code>[5, 24, N]</code>, you can swap the axis of <code>24</code> and <code>N</code> by</p>
<pre><code class="lang-python">X = X.transpose(1, 2)
</code></pre>
<p>Then you feed <code>X</code> to your neural net, the output Tensor will be of shape <code>[5, N, 28]</code>.</p>
<p>If your neural net is different, you can share the details and I’m happy to help <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
Error while obtaining an Inverse Spectrogram

Amit_Kenkre: <p>I am working on a project wherein I’m converting audio of size 60s to a spectrogram, and have the need to convert said spectrogram back to audio, at this stage I get an error “Tensor must have a last dimension of size 2”</p>
<p>the code I used is as follows:</p>
<pre><code class="lang-auto">wave, samp_rate = torchaudio.load(filpath)
transform = torchaudio.transforms.Spectrogram(n_fft=1000)

spec = transform(wave)

rev_trans = torchaudio.transforms.InverseSpectrogram(n_fft=1152)

waveform = rev_trans(spec)  # Here is where the error arises
</code></pre>
<p>after checking for the size of “spec” it says it is a tensor of [2, 501, 5295] size.</p>

nateanl: <p>Hi <a class="mention" href="/u/amit_kenkre">@Amit_Kenkre</a>, are you using torchaudio <code>0.10</code>? If so, you need to set <code>power=None</code> in <code>Spectrogram </code> to return complex-valued Tensor, otherwise it will return the magnitude of the spectrogram.</p>
<p>Second, the <code>n_fft</code> in <code>InverseSpectrogram</code> and <code>Spectrogram</code> must be the same. You may want to change <code>n_fft</code> settings to suit your need.</p>
How to properly train and evaluate nn.Transformer module?

L4zy: <p>Hi all,</p>
<p>Currently I am trying to train the official pytorch version of Transformer in nn module. But I found the tutorial in the comments is not using the nn.Transformer module, and nearly all code using pytorch version of transformer or transformer encoder/decoder are running the model once per iteration.</p>
<p>My question is how should I write train()/evaluate() function to train/evaluate the official nn.Transformer module, what should I pass into the model as tgt in the evaluate/test time?(I think the transformer model should repeat (len_output) times and use each output as the next tgt)<br>
Also, what 2D mask size should I create if the max length of my input and output sequence is different?( (in, out) or (out, out) )</p>
<p>Thanks!</p>

L4zy: <p>I found a tutorial from pytorch <a href="https://pytorch.org/tutorials/beginner/translation_transformer.html" rel="noopener nofollow ugc">here</a>, which use ground truth at training and evaluation time and use greedy decode at inference time. I wonder if that is a valid way to evaluate the model?</p>

merdad: <p>I have the same issue. Did you find out what should we do here ?</p>

SenSai_Onegai: <p>Hey dude, I think for seq2seq problem, during training, the tgt without  (end of sequence) should be the input; while for inference, the tgt should be the single token  (start of sequence), with a for loop generating tokens until  is generated.</p>