akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How to combine temporal and spatial dimensions?

braindotai: <p>How to convert batch of videos containing image sequences, where shape of each batch is -</p>
<p><code>(batch_size, 3, num of images in a video, height of image, width of image)</code></p>
<p>and I want to convert it into -</p>
<p><code>(batch_size, 3 * num of images in a video, height of image, width of image)</code></p>
<p>so its like combining 1st and 2nd dimensions…</p>
<p>Also I don’t want the reshaping procedure to be dependent on batch_size.</p>

ptrblck: <p>You could use the <code>view</code> operation as:</p>
<pre><code class="lang-python">N, C, T, H, W
x = torch.randn(B, C, T, H, W]
x = x.view(x.size(0), -1, x.size(3), x.size(4))
</code></pre>
Pytorch pt and tensorflow pb model comparison

Adithya: <p>Is pytorch pt model just the weight file or the whole graph network?</p>

ptrblck: <p>If you store the <code>state_dict</code> as described <a href="https://pytorch.org/docs/stable/notes/serialization.html">here</a> as the recommended way, the file would only contain all parameters and buffers.<br>
The not recommended way would be to store the model directly, which wouldn’t store the computation graph or the architecture, but the Python objects with all parameters, so that you would have to restore the file and folder structure.</p>
<p><a href="https://pytorch.org/docs/stable/jit.html#torch.jit.save"><code>toch.jit.save</code></a> would save a scripted model, which can be used in another process without redefining the model structure.</p>
How to feed audio into a model?&Can a fully connected layer followed by a conv layer?

Michael-J98: <p>Hi, almighty guys <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20">.<br>
I’m a beginner in digital audio signal processing. And I met some questions during reading a <a href="https://arxiv.org/abs/2006.15321" rel="noopener nofollow ugc">paper</a>.</p>
<ol>
<li>When I get the filter banks outputs from a 10s audio segment, should I send all of them into the model or just several frames?</li>
<li>I’m a little confused about the model architecture in the paper. How does the dense layer connect with the encoder and decoder. Does it convert a  Bx1x1x128 tensor to a Bx128xhxw tensor?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/4/f4f2289d9d4200578947424a9e1e793da00460bd.png" data-download-href="https://discuss.pytorch.org/uploads/default/f4f2289d9d4200578947424a9e1e793da00460bd" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/4/f4f2289d9d4200578947424a9e1e793da00460bd_2_690x304.png" alt="image" data-base62-sha1="yWTkavHEMY5VOG3MttMcL5vmNaJ" width="690" height="304" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/4/f4f2289d9d4200578947424a9e1e793da00460bd_2_690x304.png, https://discuss.pytorch.org/uploads/default/original/3X/f/4/f4f2289d9d4200578947424a9e1e793da00460bd.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/4/f4f2289d9d4200578947424a9e1e793da00460bd.png 2x" data-dominant-color="F4EDF0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">898×396 72 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></li>
</ol>
Updating Pytorch through Anaconda

AladdinPerzon: <p>I installed Pytorch in my anaconda environment using</p>
<pre><code class="lang-auto">conda install pytorch torchvision cudatoolkit=10.2 -c pytorch-nightly
</code></pre>
<p>Using the nightly build I want to make sure I have the latest version after a couple of weeks so I did:</p>
<pre><code class="lang-auto">conda update --all
</code></pre>
<p>But then I get: <code>pytorch            pytorch-nightly::pytorch-1.6.0.dev202~ --&gt; pkgs/main::pytorch-1.3.1-cpu_py37h62f834f_0</code> and the version when printing <code>torch.__version__</code> is 1.3.1. What did I do wrong?</p>
<p><strong>EDIT:</strong> When reinstalling pytorch nightly and the stable version on a new environment cuda wasn’t available, so that was very odd also. Before I got this weird behavior I had done sudo apt-get update, sudo apt-get upgrade but hadn’t rebooted the machine. After rebooting and reinstalling cuda was now available and everything seems to work as intended. Lesson learned: If they ask you to reboot then just reboot.</p>
Big variation in runtime when using for loop vs broadcasting on GPU

naveenreddy61: <p>I am noticing that runtime for loops is very high compared to broadcasting pytorch.</p>
<p>Below, I am posting code snippet to explain my point:<br>
I am finding the mse for 100,000 random datapoints using for loop vs using broadcasting using cuda.<br>
There is big difference in running time ~25s vs ~0.0006s.</p>
<p>What explains this big difference? How can I bring down the ~25 sec runtime?</p>
<p>Replacing torch with numpy in below snippets gives ~0.25s vs ~0.0075s</p>
<pre><code>import torch
import timeit
#Processing mse of 100000 data points thru a loop using pytorch cuda

def torch_loop_cuda_test():
    error = torch.tensor([0.], device = 'cuda')
    for _ in range(100000):        
        value, output = torch.rand(1,device='cuda'), torch.rand(1,device='cuda') 
        error_val = value - output
        error += torch.pow(error_val,2)

print(timeit.timeit(stmt = 'torch_loop_cuda_test()', 
                    globals = globals(), 
                    number = 1))
</code></pre>
<p>output: 24.775256000000013</p>
<pre><code>#Processing mse of 100000 data points thru pytorch cuda broadcasting

def torch_vector_cuda_test():
    values  = torch.rand(100000, device = 'cuda') 
    output = torch.rand(100000, device = 'cuda')
    error = values - output
    error = torch.pow(error, 2)
    torch.sum(error)


print(timeit.timeit(stmt = 'torch_vector_cuda_test()', 
                    globals = globals(), 
                    number = 1))
</code></pre>
<p>output: 0.0006479000001036184</p>

ptrblck: <p>The difference between the code snippets seems to be the sequential vs. parallel execution.<br>
Each CUDA operation will launch a kernel, which will create some overhead.<br>
This overhead of the kernel launch is often not visible, if the actual workload on the GPU is large.</p>
<p>However, in your first example your workload is tiny (two scalar tensors are used), while you are triggering 100k CUDA operations, which would create a large overhead.</p>
<p>That being said, you should synchronize the code via <code>torch.cuda.synchronize()</code> before starting and stopping the timer. In your second example especially you might only profile the actual kernel launch, not the complete operation.</p>

naveenreddy61: <p>Thank you patrick for the reply.<br>
Is there any other way to reduce the runtime for loops that are run inside gpu?</p>

ptrblck: <p>The usual approach would be to use a batched/vectorized operation, i.e. to try to get rid of the Python loop.</p>
Input a list of tensors to a model without the need to manually transfer each item to cuda

Omid_Shams: <p>Hi,</p>
<p>I’d like to input a list of tensors to my model. However, pytorch doesn’t automatically transfer each item in this list to cuda and I need to manually do that each time by .cuda() . Is there a better more elegant way of dealing with this?</p>

Nikronic: <p>Hi,</p>
<p>I think this can help:</p>
<pre><code class="lang-auto">list_of_tensors = [torch.randn(1, 2, 2), torch.randn(1, 2, 2), torch.randn(1, 2, 2)]
b = torch.Tensor(len(list_of_tensors), 2, 2)
b = torch.cat(list_of_tensors, dim=0)
print(b.shape)
b = b.cuda()
print(b.device)
</code></pre>
<p>Please see this thread:</p><aside class="quote" data-post="1" data-topic="8868">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/11129/48/3835_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-to-turn-a-list-of-tensor-to-tensor/8868">How to turn a list of tensor to tensor?</a> 
  </div>
  <blockquote>
    I have a list and there are many tensor in the list 
I want to turn it to just tensor, and I can put it to dataloader 
I use for loop and cat the tensor but it is very slow, data size is about 4,800,000
  </blockquote>
</aside>

<p>Bests</p>
PyTorch output image name

alameer: <p>The code below is used as part of my training model to print prediction. I would like to print each image name as well as the prediction. Can anyone advise how to do this please?</p>
<pre><code class="lang-auto">filename=CellsDataset(args.data_dir,transform=ToTensor(),return_filenames=True)
for epoch in range(50):
    print("Epoch %d" % epoch)
    model.train()
    for images, paths in tqdm(loader_train):
        images = images.to(device)
        targets = torch.tensor([metadata['count'][os.path.split(path)[-1]] for path in paths]) # B
        targets = targets.float().to(device)
        filename=CellsDataset(args.data_dir,transform=ToTensor(),return_filenames=True)
        #print(filename.files)

        output = model(images) # B x 1 x 9 x 9 (analogous to a heatmap)
        #print(output.filename)
        preds = output.sum(dim=[1,2,3]) # predicted cell counts (vector of length B)
        #filename=output.files
        print(preds)
</code></pre>

Nikronic: <p>Hi,</p>
<p>Does not <code>paths</code> contain image names already? if yes so, then you can stack column wise then convert it to a matrix.</p>
<pre><code class="lang-auto">batch_size = 10
paths = np.array(['names']*batch_size).reshape(-1, 1)
preds = torch.randint(0, 5, (batch_size, )).view(-1, 1)
names_preds = np.hstack((paths, preds.numpy()),)
</code></pre>
<p>Bests</p>

alameer: <p>Yes. paths contain the image names. How can i extract the image name against the prediction please? If I try to call path in this way fd.write( ‘,’.join(map(str, paths + ‘\n’))) i get an error message I think i need to transform it somehow. Many Thanks for your help.</p>

Nikronic: <p>Have you tried my snippet? I tried to simulate your case in exact same structure (dtype, etc).<br>
I think it is easier to first construct a matrix of path, labels of batches then saving/printing them.</p>
Will pytorch work on the new upcoming ARM macs

pytorchbegginer12345: <p>Apple has recently announced the new arm macs which will come by the end of the year. has PyTorch started the work on making it work on arm macs</p>

albanD: <p>Hi,</p>
<p>I don’t think any of us have access to that hardware so no-one has tested.<br>
You can definitely build pytorch for arm though (<a href="https://github.com/nmilosev/pytorch-arm-builds">https://github.com/nmilosev/pytorch-arm-builds</a> shows how for raspberry and android phones for example).<br>
Also maybe the compatibility layer from Apple will allow you to use the regular x86 lib just fine?</p>

pytorchbegginer12345: <p>so PyTorch has not brought the developer transition kit by apple . Cause the new macs SOC are going to have machine learning accelerator and neural engine.You can see the SOC here <a href="https://9to5mac.com/wp-content/uploads/sites/6/2020/06/wwdc-2020-14.31.12-PM.png" rel="nofollow noopener">https://9to5mac.com/wp-content/uploads/sites/6/2020/06/wwdc-2020-14.31.12-PM.png</a></p>
ValueError: batch_size should be a positive integer value

alameer: <p>Any idea what is causing this error message</p>
<p>“ValueError: batch_size should be a positive integer value, but got batch_size=Compose(<br>
ToTensor()”</p>
<pre><code class="lang-auto">from torchvision import datasets
import torch.utils.data
from torch.utils.data import DataLoader
from torchvision import transforms
from dataset2 import CellsDataset
from torchvision import datasets
import torch
import torchvision
import torchvision.transforms as transforms


class ImageFolderWithPaths(datasets.ImageFolder):
    """Custom dataset that includes image file paths. Extends
    torchvision.datasets.ImageFolder
    """

# override the __getitem__ method. this is the method that dataloader calls
def __getitem__(self, index):
    # this is what ImageFolder normally returns 
    original_tuple = super(ImageFolderWithPaths, self).__getitem__(index)
    # the image file path
    path = self.imgs[index][0]
    # make a new tuple that includes original and the path
    tuple_with_path = (original_tuple + (path,))
    return tuple_with_path

# EXAMPLE USAGE:
# instantiate the dataset and dataloader
data_dir = "/Users/nubstech/Documents/GitHub/CellCountingDirectCount/Eddata/Healthy_curated"
dataset = ImageFolderWithPaths(data_dir) # our custom dataset
dataloader = DataLoader(dataset)
dataset = DataLoader(data_dir, transforms.Compose([transforms.ToTensor()]))

# iterate over data
for inputs, labels, paths in dataloader:
    # use the above variables freely
   print(inputs, labels, paths)</code></pre>

Nikronic: <p>Hi,<br>
You are setting <code>transforms</code> into <code>batch_size</code>. <code>DataLoader</code> does not accept transforms, you need to apply it on <code>Dataset</code>.</p>
<aside class="quote no-group" data-username="alameer" data-post="1" data-topic="88297">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/9dc877/48.png" class="avatar"> alameer:</div>
<blockquote>
<p><code>dataset = DataLoader(data_dir, transforms.Compose([transforms.ToTensor()]))</code></p>
</blockquote>
</aside>
<pre><code class="lang-auto">dataset = ImageFolderWithPaths(data_dir, transform=transforms.Compose([transforms.ToTensor()]))
dataloader = DataLoader(dataset, batch_size=16)
</code></pre>
<p>Or I think you mistyped <code>Dataloader</code> with <code>ImageFolderWithPaths</code> in the last line. As a  <code>Dataloader</code> does not accept a data direction or transform which on the otherside, your custom dataset does.</p>
<p><a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#loading-and-normalizing-cifar10" rel="noopener nofollow ugc">Official tutorial</a></p>
<p>Bests</p>
Query : Loop for batch per epoch

mukul_01: <p>Hello all,<br>
I have query regarding the placement of optimizer.zero_grad() and optimizer.step()</p>
<p>for idx in range(epoch):<br>
for kdx in range(batch):<br>
y_pred = model(X)<br>
loss = loss_fn(y_pred,y_target)<br>
optimizer.zero_grad()<br>
loss.backward()<br>
optimizer.step()</p>
<p>Do we need to call them once for every epoch or for every batch iteration?</p>

ptrblck: <p>It depends on your use case.<br>
The usual approach would be to zero out the old gradients, calculate the new gradients via <code>backward()</code>, and update the parameters via <code>optimizer.step()</code> for each batch (so once per iteration).</p>
<p>However, you could simulate a larger batch size by accumulating the gradients using multiple batches and call the complete update steps after a specific number of iterations.</p>

mukul_01: <p>Thanks for the reply</p>
Group convolution output order

trougnouf: <p>I am trying to apply a group 1x1 convolution where one (whole) input is convolved with multiple sets of filters (ngroups), that is: conv = nn.Conv2d(ch, ch*ngroups, 1, groups=ch).</p>
<p>What is the correct way to separate the result of this convolution into respective groups?</p>
<p>conv(input).view(bs, ch, ngroups, h, w), or conv(input).view(bs, ngroups, ch, h, w)</p>

trougnouf: <p>I think it’s view(bs, ch, ngroups, h, w) based on the following experiment:</p>
<pre><code class="lang-auto">bs = = h = w = 1
ch = 3
ngroups = 4

&gt;&gt;&gt; input = torch.zeros(bs, 3, h, w)
&gt;&gt;&gt; input[0,1] = 1

&gt;&gt;&gt; conv = torch.nn.Conv2d(ch, ch*ngroups, 1, groups=ch)

&gt;&gt;&gt; conv.bias
Parameter containing:
tensor([ 0.7412,  0.8724, -0.6242, -0.1312, -0.1991, -0.1502,  0.1400,  0.9860,
        -0.7265,  0.2633,  0.3402, -0.7472], requires_grad=True)

&gt;&gt;&gt; conv(input)
tensor([[[[ 0.7412]], #(=bias)
         [[ 0.8724]], #(=bias)
         [[-0.6242]], #(=bias)
         [[-0.1312]], #(=bias)
         [[ 0.7028]], # not bias -&gt; ch1
         [[-0.3131]], # not bias -&gt; ch1
         [[ 0.0071]], # not bias -&gt; ch1
         [[ 0.6137]], # not bias -&gt; ch1
         [[-0.7265]], #(=bias)
         [[ 0.2633]], #(=bias)
         [[ 0.3402]], #(=bias)
         [[-0.7472]]]], grad_fn=&lt;MkldnnConvolutionBackward&gt;)

&gt;&gt;&gt;conv(input).view(bs, ch, ngroups, 1, 1)[0, 1]
Tensor([[[ 0.7028]],

        [[-0.3131]],

        [[ 0.0071]],

        [[ 0.6137]]], grad_fn=&lt;SelectBackward&gt;)
</code></pre>
Tgt and memory masks for transformer decoder layers

andcarnivorous: <p>Hi everyone,</p>
<p>I’ve been looking at previous posts regarding similar issues with understanding how to implement these masks, but things are still not clear to me for my use case.</p>
<p>I am training a Language Model with a transformer encoder-decoder. I know I can just use encoders for LMs, but I want to try this anyway and understand it.</p>
<p>My problem is that after 4 or 5 epochs the perplexity score drops very low, but at inference the model generates only the same last token from the sentence given.</p>
<p>My model is taking input from the <code>get_batch</code> function from the seq2seq tutorial</p>
<pre><code class="lang-auto">def get_batch(source, i):
    seq_len = min(bptt, len(source) - 1 - i)
    data = source[i:i+seq_len]
    target = source[i+1:i+1+seq_len]
    return data, target
</code></pre>
<p>Then, I pass input and output to my model like this</p>
<pre><code class="lang-auto">        data, targets = get_batch(train_data, i)
        optimizer.zero_grad()
        output = model(data, targets, batch=batch, epoch=epoch)
        loss = criterion(output.view(-1, ntokens), targets.view(-1))
</code></pre>
<p>then the model processes the src and tgt like so</p>
<pre><code class="lang-auto">    def forward(self, x, tgt, epoch=0, batch=0, flag="train"):
        x = self.emb(x)
        x = self.posenc(x)
        tgt = self.posenc(self.emb(tgt))
        x = self.transformer(x)#, generate_square_subsequent_mask(x.shape[0]))
        self.trans_activation = x
        if self.num_decoder_layers:
            x = self.decoder(tgt, x, tgt_mask=generate_square_subsequent_mask(tgt.shape[0]), memory_mask=generate_square_subsequent_mask(x.shape[0]))
        if not self.training and flag == "lollo":
            self.save_activations(epoch=epoch, batch=batch)
        return self.linear(x)
</code></pre>
<p>I have seen people shift the tgt sequence before passing it to the model and during the loss calculation, however it is not clear to me why this should be done since the tgt is already shifted and a mask is applied. Moreover a memory mask in my case is square since the seq len of my src and tgt are the same, right?</p>
<p>this is my inference function</p>
<pre><code class="lang-auto">def generate_sent(frase, model, TEXT, words=5):
    """
    This function gets a list of a split string and 
    returns a new string with the next N number of words 
    predicted by the model
    """
    
    for i in range(words):
        frasevec = TEXT.numericalize([frase]).to("cuda")
        model.eval()
        var = model(frasevec, frasevec).squeeze(1)
        print(var.shape)
        most = torch.argmax(var[-1])
        frase.append(TEXT.vocab.itos[most.cpu().detach().numpy()])
    
    return " ".join(frase)
</code></pre>
Regarding quantization

Ganga: <p>“”"import torch</p>
<p>import torch.nn as nn</p>
<p>m = nn.quantized.Conv2d(16, 33, 3, stride=2)</p>
<p>input = torch.randn(20, 16, 50, 100)</p>
<p>q_input = torch.quantize_per_tensor(input, scale=1.0, zero_point=0, dtype=torch.qint32)</p>
<p>output = m(input)"""</p>
<p>i got error like this can anyone help me out</p>
<p>RuntimeError: Could not run ‘quantized::conv2d’ with arguments from the ‘CPUTensorId’ backend. ‘quantized::conv2d’ is only available for these backends: [QuantizedCPUTensorId].</p>
Not able to view the forward method of a pretrained model

Sanju: <p>Hi ,</p>
<p>I loaded a pretrained shuffle-net-v2 model from torch.hub . I am able to perfrom inference on it . However I am not able to access the forward method implemented in it. I need to access the forward method since I want to print the intermediate activations of each layer and also to know exact operation performed ( in the sense that when I see the model description, it does not show where the residual connections are and so on …)</p>
<p>Any help in this regard would be greatly appreciated.</p>
<p>Thankyou in advance .</p>

SandPhoenix: <p>I dont know if I understood you very well but I think that you need to add this kind of thing in your model’s definition since the begining like for example.<br>
in the forward function, put each output of each layer inside a list and then return the whole vector after that and therefore you’d be able to access any output of any layer.<br>
This worked for me before.</p>

Sanju: <p>Yes … I was able to do that when I trained my own model from scratch i.e I was printing the activations after each layer from within the forward method . But when I take pre-trained model from the hub, how do I do it ?</p>

SandPhoenix: <p>As far as I know, I am not sure so I can be wrong. I think that in this case you won’t be able to do it</p>

Sanju: <p>Thanks for the quick reply !!</p>
Why gradients of nn.CTCLoss is the objective function derivatives with respect to the unnormalised outputs u(t,k)

yanz0920: <p>hi, I found that the gradients of nn.CTCLoss is the objective function derivatives with respect to the unnormalised outputs u(t,k), why is not the objective function derivatives with respect to the input(log probs obtained with log_softmax)? Does it compute gradients again when backpropagate through softmax layer?</p>
<p>thank you!</p>

albanD: <p>Hi,</p>
<p>I am not sure to grasp your question exactly.<br>
Could you share some code that shows the mismatch you’re talking about?</p>

yanz0920: <aside class="quote no-group" data-username="albanD" data-post="2" data-topic="87377">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p>ac</p>
</blockquote>
</aside>
<p>thanks for your answer. The code is in pytorch/aten/src/ATen/native/LossCTC.cpp, line 300.<img src="https://discuss.pytorch.org/uploads/default/original/3X/7/f/7fa9b43c5dfa746e3f7747048fea4ab36dee668a.png" alt="微信截图_20200706111226" data-base62-sha1="idme8oa93U7QXfo7poC2MZ8HR4S" width="688" height="241"></p>

albanD: <p>Hi,</p>
<p>Thanks for the link. This should be OK no?<br>
All these function’s gradients are checked with finite difference. So it is very unlikely that it is not computing the backward corresponding to their forward.</p>

yanz0920: <p>Hi，I can’t find the softmax op and log op in the forward functions(ctc_loss_cpu_template) in the same file, line 37. It makes me confused.</p>

albanD: <p>Hi,</p>
<p>The doc seems to indicate that it takes “Log_probs” as input. So I don’t think the log is included.<br>
That would explain the backward formula <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
ERROR: No matching distribution found for torchvision==0.4.2

fgauthier: <p>I created a requirements.txt using pipreqs and I wanted to install torchvision 0.4.2 in a conda environnement. I can install torchvision outside of a conda environnement easily by running</p>
<blockquote>
<p>pip install -r requirements.txt</p>
</blockquote>
<p>However when I run this command in a conda environnement I always get the following error</p>
<blockquote>
<p>ERROR: Could not find a version that satisfies the requirement torchvision==0.4.2 (from -r requirements.txt (line 2)) (from versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3, 0.5.0, 0.6.0, 0.6.1)<br>
ERROR: No matching distribution found for torchvision==0.4.2 (from -r requirements.txt (line 2))</p>
</blockquote>
<p>Should I just upgrade to a more recent version of torchvision or is there a reason why I can’t install torchvision 0.4.2 in a conda environnement?</p>
Symbolic registry for custom Python operations

druzhkov.paul: <p>Having a custom C++ operation it’s possible to register a symbolic function for it using</p>
<p><code>register_op(name, symbolic, namespace, opset)</code></p>
<p>With custom Python operation it’s possible to wrap it with an <code>autograd.Function</code> having static <code>symbolic</code> method, but it’s much less agile in term of different opsets’ support. Is there a way to register symbolic for a Python operation in the same way, as it’s done for C++ ones.</p>
Expected cuda:0 but got CPU error

SandPhoenix: <p>Hello everyone,<br>
So, I have a problem and I hope that you guys can help me figure out a solution for this.<br>
so I wrote a model and trained it and I am trying to test it out.<br>
and I get this error so here is the full sityuation:</p>
<pre><code class="lang-auto">device = 'cuda' if torch.cuda.is_available() else 'cpu'
print(device)
Model.eval()
    with torch.no_grad():
        total_correct = 0
        size_of_set = 0
    for images, labels in DataLoader:
        images = images.to(device)
        print(images.device)
        labels = labels.to(device)
        preds = Model(images)
        total_correct+= get_correct(preds,labels)
</code></pre>
<p>So actually, the first print shows that cuda is available.<br>
but the second one shows that the images are given to the cpu…<br>
This is weird and then I get this error:<br>
RuntimeError: expected device cuda:0 but got device cpu</p>

harsha_g: <p>Did you move your <code>Model</code> as well to the <code>device</code> using <code>Model.to(device)</code>?</p>

SandPhoenix: <p>thx for your reply.<br>
Actually, I did and it moved to another error that is related to my code. I think that I did something wrong.</p>
Is it possible to back propagate in a TorchScript?

rossimat: <p>PyTorch 1.5.0</p>
<p>Hello, I am using PyTorch to minimise a cost function. No training involved. I was wondering whether using TorchScript rather than pure Python could have led to a speed up. However, I am not sure whether it is possible to back propagate in TorchScript.</p>
<p>Let us consider the following toy class:</p>
<pre><code class="lang-auto">class Example(nn.Module):
  def forward(self, x):
    y = torch.tensor([0], dtype=x.dtype)
    y.requires_grad = True
    return y
</code></pre>
<p>If I call <code>example_scripted = torch.jit.script(Example())</code>, I get the following error:</p>
<pre><code class="lang-auto">RuntimeError:
Tried to set an attribute: grad on a non-class: Tensor:
</code></pre>
<p>The issue seems the <code>requires_grad</code>. I am wondering whether it is possible to use TorchScript to back propagate. Is it?</p>
<p>Please see also <a href="https://github.com/pytorch/pytorch/issues/40561" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/40561</a></p>
<p>Thank you.</p>

googlebot: <p>Yeah, backprop works, but some python code won’t compile. In your (contrived) case,  tensor(…).requires_grad_(True) would compile, but you don’t usually need to be explicit about requires_grad, as it propagates with operations, and there is also .detach().</p>

lev: <p>Yes you can train see this book:<br>
<a href="https://pytorch.org/deep-learning-with-pytorch" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/deep-learning-with-pytorch</a></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/1/e1fa81909363d4c18ebcb5001f6c377570a543e5.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/e1fa81909363d4c18ebcb5001f6c377570a543e5" title="Picture1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/1/e1fa81909363d4c18ebcb5001f6c377570a543e5_2_690x137.jpeg" alt="Picture1" data-base62-sha1="wf68xoRjYvz04iKyQ5UOSEx3lxr" width="690" height="137" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/1/e1fa81909363d4c18ebcb5001f6c377570a543e5_2_690x137.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/e/1/e1fa81909363d4c18ebcb5001f6c377570a543e5_2_1035x205.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/1/e1fa81909363d4c18ebcb5001f6c377570a543e5_2_1380x274.jpeg 2x" data-dominant-color="F5F5F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Picture1</span><span class="informations">1552×309 109 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>