akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Issue with PyTorch and ONNX

kshama: <p>I am  finding some issues while converting from pytorch to ONNX please help me in sorting out things .</p>
<p>Please find the issue below</p>
<p>TypeError: i_(): incompatible function arguments. The following argument types are supported:<br>
1. (self: torch._C.Node, arg0: str, arg1: int) -&gt; torch._C.Node</p>
<p>Invoked with: %338 : Tensor = onnx::RandomNormal(), scope: VAEGen<br>
, ‘shape’, 333 defined in (%333 : int[] = prim::ListConstruct(%320, %324, %328, %332), scope: VAEGen<br>
) (occurred when translating randn)</p>
<p>The code for the model is below</p>
<pre><code class="lang-auto">class VAEGen(nn.Module):
    # VAE architecture
    def __init__(self, input_dim):
        super(VAEGen, self).__init__()
        # dim = params['dim']
        # n_downsample = params['n_downsample']
        # n_res = params['n_res']
        # activ = params['activ']
        # pad_type = params['pad_type']

        # content encoder
        self.enc = ContentEncoder(2, 4, 3, 64, 'in', 'relu', pad_type='reflect')
        self.dec = Decoder(2, 4, self.enc.output_dim, 3, res_norm='in', activ='relu', pad_type='reflect')

    def forward(self, images):
        # This is a reduced VAE implementation where we assume the outputs are multivariate Gaussian distribution with mean = hiddens and std_dev = all ones.
        hiddens,_ = self.encode(images)
        if self.training == False:
            noise = Variable(torch.randn(hiddens.size()).cuda(hiddens.data.get_device()))
            images_recon = self.decode(hiddens + noise)
        else:
            images_recon = self.decode(hiddens)
        return images_recon, hiddens

    def encode(self, images):
        hiddens = self.enc(images)
        noise = Variable(torch.randn(hiddens.size()).cuda(hiddens.data.get_device()))
        return hiddens, noise

    def decode(self, hiddens):
        images = self.dec(hiddens)
        return images

</code></pre>
<p>The conversion code for pytorch to ONNX is below</p>
<pre><code class="lang-auto">generator = VAEGen(3).cuda() # Initalize your weights there
    dummy_input = torch.randn(1 ,3, 256, 256).cuda()
    torch.onnx.export(generator, dummy_input, "exported.onnx")
</code></pre>
Grid_Sample loop instead of full blocks

FullStackOfPancakes: <p>Hey all,</p>
<p>Previously, I was using F.grid_sample to process a block of images simultaneously. However, due to the data sizes, I’m moving towards wanting to process each of the images separately so that I don’t have memory size problems on the GPU.</p>
<p>Previously, I was doing something like</p>
<pre><code class="lang-auto">    frame_data = grid_sample(frame_data, row_col_grid, mode='bilinear', padding_mode='zeros')
</code></pre>
<p>frame_data is a &lt;1, 50, 1024, 1024&gt; Tensor and row_col_grid is a &lt;1, 1024, 1024, 2&gt; Tensor.</p>
<p>That was working, other than potential gpu memory issues in the future with larger images.</p>
<p>Now, I’m trying to move to something like the following:</p>
<pre><code class="lang-auto">    for frame in frame_data:
        frame = frame.unsqueeze(0).unsqueeze(0).to('cuda:0')
        frame = grid_sample(frame, row_col_grid, mode='bilinear', padding_mode='zeros')
        frame_list.append(frame.squeeze(0).squeeze(0).to('cpu'))
</code></pre>
<p>When frame hits grid_sample, it is size &lt;1, 1, 1024, 1024&gt; with row_col_grid still &lt;1, 1024, 1024, 2&gt;.</p>
<p>One would think that I’d get the same list of frames at the end, but for some reason, I’m getting frames with only zeros (which I understand to be the padding, I believe).</p>
<p>Any thoughts on how to loop through these images instead of processing them in a block?</p>
<p>(Also, if it wasn’t obvious, I’m using the PyTorch 1.3 with Python 3.7.)</p>
OSError: [Errno 22] Invalid argument while loading pretrained fasttext embeddings

sai_m: <p>I am trying to load pretrained fastext embeddings from .vector_cache on a windows machine, and it’s throwing OSError: [Errno 22] Invalid argument error. And when we download for the first time it has downloaded two files into .vectors_cache folder : wiki.en.vec and wiki.en.vec.pt. what’s the significance of wiki.en.vec.pt file?</p>
How to normalize numpy array by row?

skyraker: <p>I use this:<br>
def data_norm(x):<br>
mu = x.mean(1)<br>
sigma = x.std(1)<br>
return (x-mu)/sigma<br>
but got error, how to deal with it</p>

Eta_C: <p>Do you mean</p>
<pre><code class="lang-python">import numpy as np

def data_norm(x):
    mu = x.mean(axis=1, keepdims=True)
    sigma = x.std(axis=1, keepdims=True)
    return (x-mu)/sigma
</code></pre>

skyraker: <p>Got it! I has been confused by ‘keepdims’</p>

Eta_C: <p>In my understanding, <code>keepdims</code> means that <code>len(output.shape)</code> equals to <code>len(input_shape)</code>.</p>

skyraker: <p>Thx   XD<br>
I need to learn more about it</p>
Cuda extension error

yuanqing_miao: <p>I tried to compile the example of cuda extension but failed. Here is part of the log:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/c/0cbb9f8a1e6f831eb051f5b870444b2061c897ef.png" data-download-href="https://discuss.pytorch.org/uploads/default/0cbb9f8a1e6f831eb051f5b870444b2061c897ef" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/c/0cbb9f8a1e6f831eb051f5b870444b2061c897ef_2_690x40.png" alt="image" data-base62-sha1="1ODI3a5OpwYPg8cRXL21HrgVEGb" width="690" height="40" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/c/0cbb9f8a1e6f831eb051f5b870444b2061c897ef_2_690x40.png, https://discuss.pytorch.org/uploads/default/optimized/3X/0/c/0cbb9f8a1e6f831eb051f5b870444b2061c897ef_2_1035x60.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/0/c/0cbb9f8a1e6f831eb051f5b870444b2061c897ef_2_1380x80.png 2x" data-dominant-color="1A1919"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1888×111 19.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Could anyone do me a favour?</p>

yuanqing_miao: <p>The example of cuda extension is <a href="https://github.com/pytorch/extension-cpp" rel="nofollow noopener">https://github.com/pytorch/extension-cpp</a></p>
<p>I installed pytorch from source and turned NCCL off. Does that matter?</p>
Torch multiprocessing not yielding nontrivial speedup

timtody: <p>I was running into some trouble using DistributedDataPrallel and made a simple sanitation script to measure the speedup yielded by torches multiprocessing.</p>
<pre><code class="lang-auto">import os
import sys
import torch.distributed as dist
import torch.multiprocessing as mp
import time

def setup(rank, world_size):
    os.environ['MASTER_ADDR'] = 'localhost'
    os.environ['MASTER_PORT'] = '12372'
    dist.init_process_group("gloo", rank=rank, world_size=world_size)

def cleanup(): 
    dist.destroy_process_group()

def main(rank, world_size):
    setup(rank, world_size)
    time.sleep(5)
    cleanup()


if __name__ == "__main__":
    world_size = int(sys.argv[1])
    start = time.time()
    mp.spawn(main,
             args=(world_size,),
             nprocs=world_size,
             join=True)
    stop = time.time()
    print("world size: ", world_size, "\ntime elapsed: ", stop - start)
</code></pre>
<p>I’ve tested with world sizes of [1, 4, 8, 16] on a 32 core machine which resulted in the following outputs:</p>
<p>world size: 1<br>
time elapsed: 7.073131561279297<br>
world size: 4<br>
time elapsed: 8.751604080200195<br>
world size: 8<br>
time elapsed: 9.452412605285645<br>
world size: 16<br>
time elapsed: 13.709965944290161<br>
world size: 32<br>
time elapsed: 22.26715612411499</p>
<p>Am I using mp / dist wrong? Or is an overhead per thread in this scope to be expected?</p>
Saving list of dictionaries of tensors as model parameters

thematrixduo: <p>Suppose I have a list of dictionaries of Tensors, which are used (but not necessarily trainable) in a model’s computation. What is the best way to save this list of dictionaries of Tensors when I save the model?</p>

LeviViana: <p>What do you mean by the best ? fastest ? memory optimized ? most compact ? simplest ?I think it depends on your context. I guess JSON serialization or pickling would be good enough for the vast majority of use cases.</p>

thematrixduo: <p>Indeed you can have many definitions of ‘best’. I do know that you can save as this list by pickle or json but then it will be separate from the model (Correct me if I am wrong). I am looking for a way to save this list as model parameters, rather than a separate object.</p>

LeviViana: <p>You can put whatever you want in the state_dict, here is an example:</p>
<pre><code class="lang-python">import torch
net = torch.nn.Linear(2, 2)
d = net.state_dict()
d.update({"custom_stuff": 123})
torch.save(d, "net.pth")

# Load it from the serialized file

new_net = torch.nn.Linear(2, 2)
# By default, strict=True, then you need to force it to False to not raise an error
state_dict = torch.load("net.pth")
custom_stuff = state_dict["custom_stuff"] # Retrieving your stuff after loading
new_net.load_state_dict(state_dict, strict=False) 
# NOTE: you can just del state_dict["custom_stuff"] if you want, then strict can be set True

# Check that the parameters have been loaded as expected

print(net.weight.data.equal(new_net.weight.data))
print(net.bias.data.equal(new_net.bias.data))

</code></pre>

thematrixduo: <p>This looks good, Thanks <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>
Encountered known unsupported method torch.randn/torch.Tensor.get_device/torch.Tensor.cuda

kshama: <p>I am not able to solve this issue. Don’t know what could be the reason for this issue .</p>
<p>Please any feedback to this would be appreciated .</p>
<p>Please find model below and also the way I am trying to convert to the torch model  tensorRT is also provided below .</p>
<pre><code class="lang-auto">class VAEGen(nn.Module):
    # VAE architecture
    def __init__(self, input_dim):
        super(VAEGen, self).__init__()
        # dim = params['dim']
        # n_downsample = params['n_downsample']
        # n_res = params['n_res']
        # activ = params['activ']
        # pad_type = params['pad_type']

        # content encoder
        self.enc = ContentEncoder(2, 4, 3, 64, 'in', 'relu', pad_type='reflect')
        self.dec = Decoder(2, 4, self.enc.output_dim, 3, res_norm='in', activ='relu', pad_type='reflect')

    def forward(self, images):
        # This is a reduced VAE implementation where we assume the outputs are multivariate Gaussian distribution with mean = hiddens and std_dev = all ones.
        hiddens,_ = self.encode(images)
        if self.training == False:
            noise = Variable(torch.randn(hiddens.size()).cuda(hiddens.data.get_device()))
            images_recon = self.decode(hiddens + noise)
        else:
            images_recon = self.decode(hiddens)
        return images_recon, hiddens

    def encode(self, images):
        hiddens = self.enc(images)
        noise = Variable(torch.randn(hiddens.size()).cuda(hiddens.data.get_device()))
        return hiddens, noise

    def decode(self, hiddens):
        images = self.dec(hiddens)
        return images
</code></pre>
<p><a href="https://github.com/NVIDIA-AI-IOT/torch2trt" rel="nofollow noopener">https://github.com/NVIDIA-AI-IOT/torch2trt</a></p>
Dataloader size must match?

alit: <p>Hello,</p>
<p>I am using a dataloader with IMDB dataset and i get this error :</p>
<p>RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 1 and 2 in dimension 1 at …\aten\src\TH/generic/THTensor.cpp:711</p>
<p>class DataProcessing(Dataset):<br>
def <strong>init</strong>(self,data,labels,tokenizer):<br>
self.data = data<br>
self.tokenizer = tokenizer<br>
self.labels = labels<br>
def <strong>getitem</strong>(self, index):<br>
return torch.tensor(self.tokenizer.encode(self.data.iloc[index,:].item()[:3]))<br>
def <strong>len</strong>(self):<br>
return self.data.shape[0]</p>
<p>train_sampler = RandomSampler(dp)<br>
train_dataloader = DataLoader(dp, sampler=train_sampler, batch_size=100)</p>
<p>i reduced the size and now it’s always the same but i still get the error ?</p>
Cannot convert "at::Tensor" to "nc::Ndarray" libtorch

VishnuPJ: <p>Hi all, i am trying to deploy a project using libtorch.I have some post processing steps to do.The output of my model is a Tensor and i have to convert it to ndarray to continue with the post processing.In python this can be easily done with “tensor.numpy()” .Is there any equivalent for that in cpp?<br>
I have tried various methods but none is working.<br>
Please help me with this.</p>

dhpollack: <p>I think you can use <code>tensor.data&lt;float&gt;()</code>, but what are you doing with this c++ numpy implementation that you can’t do directly in libtorch?  Assuming you are using <a href="https://dpilger26.github.io/NumCpp/doxygen/html/index.html" rel="nofollow noopener">this</a>, it doesn’t look like there is anything there that you couldn’t do while keeping your data as a tensor.</p>