akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Adaptive Loss function

algeriapy: <p>I am trying to implement Adaptive Loss function, which has the formulas<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/7/2/7284ae89d02431d4c4bf8bcdecc092591f992e75.png" alt="545545" data-base62-sha1="gl4HsRh2UbSRyDJKBkgc5rGc80R" width="318" height="93"></p>
<p>here my code</p>
<pre><code class="lang-auto"> def Adaptive_loss(preds, labels, sigma):
     l2 = (1+sigma)*(torch.mean(preds - labels)**2)
     l1 = sigma + torch.mean(torch.abs(preds - labels))
     return l2/l1
</code></pre>
<p>The issue I got very very worse performance compared with using MSE loss<br>
Could anyone check if I have any mistake</p>

harsha_g: <p>You are taking <code>mean</code> in the numerator and denominator separately instead of taking the <code>mean</code> after <code>l2/l1</code>. In other words, <code>mean(l1/l2)</code> is not <code>mean(l1)/mean(l2)</code></p>

algeriapy: <p>Thanks a lot, your note fixed the issue</p>

Jason_Lucas: <aside class="quote no-group" data-username="algeriapy" data-post="1" data-topic="85354">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/34f0e0/48.png" class="avatar"> algeriapy:</div>
<blockquote>
<pre><code class="lang-auto"> def Adaptive_loss(preds, labels, sigma):
     l2 = (1+sigma)*(torch.mean(preds - labels)**2)
     l1 = sigma + torch.mean(torch.abs(preds - labels))
     return l2/l1
</code></pre>
</blockquote>
</aside>
<p>I am having difficulties implementing adaptive loss. Can you kindly share your solution function with me?</p>
Pytorch equivalent of tensor flow

Anubhav1107: <p>I am trying to follow up on this code in Pytorch. I have been trying for days but reading tensor flow docs then PyTorch docs has made me totally confused.</p>
<p>input_data = Input(shape=(256, 64, 1), name=‘input’)</p>
<p>inner = Conv2D(32, (3, 3), padding=‘same’, name=‘conv1’, kernel_initializer=‘he_normal’)(input_data)<br>
inner = BatchNormalization()(inner)<br>
inner = Activation(‘relu’)(inner)<br>
inner = MaxPooling2D(pool_size=(2, 2), name=‘max1’)(inner)</p>
<p>inner = Conv2D(64, (3, 3), padding=‘same’, name=‘conv2’, kernel_initializer=‘he_normal’)(inner)<br>
inner = BatchNormalization()(inner)<br>
inner = Activation(‘relu’)(inner)<br>
inner = MaxPooling2D(pool_size=(2, 2), name=‘max2’)(inner)<br>
inner = Dropout(0.3)(inner)</p>
<p>inner = Conv2D(128, (3, 3), padding=‘same’, name=‘conv3’, kernel_initializer=‘he_normal’)(inner)<br>
inner = BatchNormalization()(inner)<br>
inner = Activation(‘relu’)(inner)<br>
inner = MaxPooling2D(pool_size=(1, 2), name=‘max3’)(inner)<br>
inner = Dropout(0.3)(inner)</p>
<h1>CNN to RNN</h1>
<p>inner = Reshape(target_shape=((64, 1024)), name=‘reshape’)(inner)<br>
inner = Dense(64, activation=‘relu’, kernel_initializer=‘he_normal’, name=‘dense1’)(inner)</p>
<h2>RNN</h2>
<p>inner = Bidirectional(LSTM(256, return_sequences=True), name = ‘lstm1’)(inner)<br>
inner = Bidirectional(LSTM(256, return_sequences=True), name = ‘lstm2’)(inner)</p>
<h2>OUTPUT</h2>
<p>inner = Dense(num_of_characters, kernel_initializer=‘he_normal’,name=‘dense2’)(inner)<br>
y_pred = Activation(‘softmax’, name=‘softmax’)(inner)</p>
<p>model = Model(inputs=input_data, outputs=y_pred)</p>
<p>and I tried</p>
<p>class Net(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Net,self).<strong>init</strong>()</p>
<pre><code>    self.input_data = input_size
    self.conv1 = nn.Conv2d(32,3,3)
    self.conv2 = nn.Conv2d(64,3,3)
    self.conv3 = nn.Conv2d(128,3,3)
    
    #self.conv_bn = nn.BatchNorm2d()
    self.dropout = nn.Dropout(0.3)
    self.maxp = torch.nn.MaxPool2d((2,2))
    
    
    #CNN to RNN
    self.linear1 = nn.Linear(256*62*62,64)
    
    #RNN
    self.lstm = torch.nn.LSTM(256, 10,bidirectional = True)
    
    #output
    self.linear2 = nn.Linear(64,num_of_chars)
    
def forward(self,x,input_size):
    
     
    x = self.conv1(input_size)
    x = nn.BatchNorm2d(x)
    x = F.relu(x)
    x = self.maxp(x)
    x = self.conv2(x)
    x = nn.BatchNorm2d(x)
    x = F.relu(x)
    x = self.maxp(x)
    x = self.dropout(x)
    x = self.conv3(x)
    x = nn.BatchNorm2d(x)
    x = F.relu(x)
    x = self.maxp(x)
    x = self.dropout(x)
    x = x.view((64,1024))
    x = self.linear1(x)
    
    x = self.lstm(x)
    x = self.lstm(x)
    
    x = self.linear2(x)
    x = nn.Softmax(x,dim=1)
    return x
</code></pre>
<p>But the model summary is not at all same. I am very confused by the parameters. Any help would be appreciated. Tell me if you need anything. Thanks</p>

Usmankhan_1234: <p>What is the input size of pytorch model</p>
How to implement other DP methods?

Frankozay: <p>Hello everyone, I’m a beginner in differential privacy,  I think that use Opacus is easy to implement DP-SGD, my question may be silly but I wonder if there is a way to use DP independently?<br>
I don’t know if I’ve expressed clear enough, I mean if I add noise to dataset or output(like DP-logits) instead of gradients,  is there a tool in Opacus to compute the DP part(like epsilon and delta)? If not, is there some advice to achieve this?<br>
Thanks for your help!!</p>

ashkan_software: <p>Hi <a class="mention" href="/u/frankozay">@Frankozay</a></p>
<p>Thank you for your interest in Opacus.</p>
<p>Your question is not silly!</p>
<p><strong>Opacus</strong> is a library to train a machine learning model with differential privacy with minimal code changes.</p>
<blockquote>
<p>is there a tool in Opacus to compute the DP part(like epsilon and delta)</p>
</blockquote>
<p>Yes, but this accounts for privacy of adding noise to the gradients, not anything else. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>I think the best is to search for tools that are for DP, and not necessarily DP in Machine Learning</p>
<p>Hope this helps</p>

Frankozay: <p>I got this and I’m looking forward to search other tools, thanks for your reply:)!!</p>
How does one set the pytorch distributed hostname, port and GLOO_SOCKET_IFNAME so that DDP works?

Brando_Miranda: <p>How does one solve the:</p>
<pre><code class="lang-auto">[W ProcessGroupGloo.cpp:558] Warning: Unable to resolve hostname to a (local) address. Using the loopback address as fallback. Manually set the network interface to bind to with GLOO_SOCKET_IFNAME. (function operator())
</code></pre>
<p>error in pytorch?</p>
<p>I read:</p>
<ul>
<li><a href="https://discuss.pytorch.org/t/pytorch-distributed-example-code-hang-deadlock/83335/12" class="inline-onebox">PyTorch distributed example code hang. Deadlock? - #12 by Brando_Miranda</a></li>
<li><a href="https://pytorch.org/docs/stable/distributed.html" class="inline-onebox" rel="noopener nofollow ugc">Distributed communication package - torch.distributed — PyTorch 1.7.1 documentation</a></li>
<li>and more<br>
but they weren’t terribly helpful.</li>
</ul>
<p>My questions are:</p>
<ol>
<li>what does this error mean?</li>
<li>how do I resolve it when using pytorch?</li>
</ol>
<p>I am trying to run this in my local machine for debugging purposes. It doesn’t work in pycharm, pycharm debug or the terminal locally. The clusters seem to work for some reason.</p>
<p>Why is this and how do I fix it?</p>
<hr>
<p>cross posted:</p>
<ul>
<li><a href="https://www.quora.com/unanswered/How-does-one-set-the-PyTorch-distributed-hostname-port-and-GLOO_SOCKET_IFNAME-so-that-DDP-works" class="inline-onebox" rel="noopener nofollow ugc">How does one set the PyTorch distributed hostname, port, and GLOO_SOCKET_IFNAME so that DDP works? - Quora</a></li>
<li><a href="https://stackoverflow.com/questions/66426441/how-does-one-set-the-pytorch-distributed-hostname-port-and-gloo-socket-ifname-s" class="inline-onebox" rel="noopener nofollow ugc">python - How does one set the pytorch distributed hostname, port and GLOO_SOCKET_IFNAME so that DDP works? - Stack Overflow</a></li>
<li><a href="https://www.reddit.com/r/pytorch/comments/lw1tkv/how_does_one_set_the_pytorch_distributed_hostname/" rel="noopener nofollow ugc">https://www.reddit.com/r/pytorch/comments/lw1tkv/how_does_one_set_the_pytorch_distributed_hostname/</a></li>
</ul>

ralzq01: <p>Hi, I also get same error. Have you solved it?<br>
I can initialize process group cross nodes with nccl backend, but failed in gloo.</p>

ralzq01: <p>Just resolved it, by adding an environment configuration: <code>GLOO_SOCKET_IFNAME=eth0</code></p>
Access weights, biases in model - pruning

grid_world: <p>To implement L1-norm, unstructured, layer-wise pruning with torch.nn.utils.prune l1_unstructured, radom_unstructured methods, I have a toy LeNet-300-100 dense neural network as-</p>
<pre><code class="lang-auto">class LeNet300(nn.Module):
    def __init__(self):
        super(LeNet300, self).__init__()
        
        # Define layers-
        self.fc1 = nn.Linear(in_features = 28 * 28 * 1, out_features = 300)
        self.fc2 = nn.Linear(in_features = 300, out_features = 100)
        self.output_layer = nn.Linear(in_features = 100, out_features = 10)
        
        self.weights_initialization()
    
    
    def forward(self, x):
        x = F.leaky_relu(self.fc1(x))
        x = F.leaky_relu(self.fc2(x))
        x = self.output_layer(x)
        return x
    
    
    def weights_initialization(self):
        '''
        When we define all the modules such as the layers in '__init__()'
        method above, these are all stored in 'self.modules()'.
        We go through each module one by one. This is the entire network,
        basically.
        '''
        for m in self.modules():
            if isinstance(m, nn.Linear):
                nn.init.kaiming_normal_(m.weight)
                nn.init.constant_(m.bias, 1)
    
    
    def shape_computation(self, x):
        print(f"Input shape: {x.shape}")
        x = self.fc1(x)
        print(f"dense1 output shape: {x.shape}")
        x = self.fc2(x)
        print(f"dense2 output shape: {x.shape}")
        x = self.output_layer(x)
        print(f"output shape: {x.shape}")
        del x
        return None

# Initialize architecture-
model = LeNet300().to(device)
</code></pre>
<p>This has 266610 trainable parameters. To prune this with 20% for the first two dense layers and 10% for the output layer until 99.5% sparsity, you need 25 pruning rounds. The pruning is done with-<br>
<code>l1_unstructured(module = fc, name = 'weight', amount = 0.2)</code></p>
<p>Iterating through the layers is done with-</p>
<pre><code class="lang-auto">for name, module in model.named_modules():
    if name == '':
        continue
    else:
        print(f"layer: {name}, module: {module}")
</code></pre>
<p>However, for a particular module, how to access its weights and biases besides using-<br>
<code>module.weight, module.bias</code><br>
?</p>
<p>The idea is to use a layer-wise (for now) pruning function as-</p>
<pre><code class="lang-auto"># Prune multiple parameters/layers in a given model-
for name, module in model.named_modules():
  
    # prune 20% of weights/connections in for all hidden layaers-
    if isinstance(module, torch.nn.Linear) and name != 'output_layer':
        prune.l1_unstructured(module = module, name = 'weight', amount = 0.2)
    
    # prune 10% of weights/connections for output layer-
    elif isinstance(module, torch.nn.Linear) and name == 'output_layer':
        prune.l1_unstructured(module = module, name = 'weight', amount = 0.1)
</code></pre>
<p>Bias pruning will also be included.</p>
RuntimeError: Failed to run torchsummary. See above stack traces for more details. Executed layers up to: []

huynth: <p>i am trying to run torchsummary but getting this error. I don’t understand why, please help me. This is my code:</p>
<pre><code class="lang-auto">from gc_layer import GatedConv2d, GatedDeConv2d
import torch
import torch.nn as nn
from base_model import BaseModel

class Discriminator(BaseModel):
    def __init__(self, channels = 64):
        super(Discriminator, self).__init__()
        self.channels = channels
        input_dim = 3
        self.init_weights()

        self.gt_conv1 = GatedConv2d(input_dim+1, channels, kernel_size=3, dilation=1, padding='same')
        self.lk1 = nn.LeakyReLU()

        layers = []
        for i in range(1,7):
            mult = (2**i) if (2**i) &lt; 8 else 8
            in_mult = (2**i) if (2**i) &lt; 9 else 16
            layers.append(nn.Sequential(GatedConv2d(channels*in_mult, channels*mult, kernel_size=3, stride=2, padding='same', dilation=1),
                            nn.LeakyReLU()))

        self.layers = nn.Sequential(*layers)

    def forward(self, inputs, mask):
        x_in = torch.cat([inputs, mask], dim=1)
        output = self.gt_conv1(x_in)
        output = self.lk1(output)
        for i in range(1,7):
            output = self.layers[i](output)
        return output


if __name__ == "__main__":
    model = Discriminator()
    print(model)
    from torchsummary import summary
    print(summary(model, [(3,256,256), (1,256,256)], 1))```
And this is error:

</code></pre>
<p>Traceback (most recent call last):<br>
File “/home/huynth/miniconda3/envs/inpainting/lib/python3.8/site-packages/torchsummary/torchsummary.py”, line 140, in summary<br>
_ = model.to(device)(*x, *args, **kwargs)  # type: ignore[misc]<br>
File “/home/huynth/miniconda3/envs/inpainting/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 1102, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
TypeError: forward() takes 3 positional arguments but 4 were given</p>
<p>The above exception was the direct cause of the following exception:</p>
<p>Traceback (most recent call last):<br>
File “/home/huynth/Hypergraph-Inpainting/models/discriminator.py”, line 38, in <br>
print(summary(model, [(3,256,256), (1,256,256)], 1))<br>
File “/home/huynth/miniconda3/envs/inpainting/lib/python3.8/site-packages/torchsummary/torchsummary.py”, line 143, in summary<br>
raise RuntimeError(<br>
RuntimeError: Failed to run torchsummary. See above stack traces for more details. Executed layers up to: []</p>
<pre><code class="lang-auto"></code></pre>

James_TRUMP: <p>1.cd  …/lib/python3.7/site-packages/torchsummary/ (you can find it’s parent dir by “import torchsummary   print(torchsummary.<strong>file</strong>)” in python<br>
2. vim torchsummary.py<br>
3.modify “total_input_size = abs(np.prod(input_size) * batch_size * 4. / (1024 ** 2.))” to “total_input_size = abs(np.sum([np.prod(in_tuple) for in_tuple in input_size]) * batch_size * 4. / (1024 ** 2.))”</p>
Understanding pytorch intra op parallelism

Puneeth_Naik: <p>I am trying to understand the application of pytorch’s intra op parallelism in the context of inference of a pytorch model. My question is: <em>Is there use to pytorch intra op parallelism when model inference happens on the GPU?</em> My thinking is that intra op parallelism gives great results when the inference takes place on the CPU. Any clarification in this regard will be of great help.<br>
Thanks.</p>
Where can I find the implementation (raw code) of rnn_tanh and rnn_relu

Ji_Zhang: <p>I’m a beginner of pytorch, I would like to see how a RNN is realized in the raw code of pytorch.<br>
Sofar, I found the class RNN call functions named _VF.rnn_tanh and _VF.rnn_relu.<br>
How can I found the raw cods of them?<br>
THX.</p>

khushi-411: <p>Hi, <a class="mention" href="/u/ji_zhang">@Ji_Zhang</a>,<br>
The implementation of <code>rnn_tanh</code> and <code>rnn_relu</code> are available <a href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/RNN.cpp#L1629-L1655" rel="noopener nofollow ugc">here</a>. Thanks!</p>

Ji_Zhang: <p>thanks a lot, khushi</p>
Autocast: Does support each input has diffrent dtype in custom op

dancedpipi: <p>I have a custom op looks like below:</p>
<pre><code class="lang-auto">def staticmethod_combined(custom_decorator):
    return lambda func: staticmethod(custom_decorator(func))

def MyCustomOp(torch.autograd.Function):
    @staticmethod_combined(custom_fwd(cast_inputs=torch.float16))
    def forward(ctx, t1, t2, t3):
        ## do forward
        return output

    @staticmethod_combined(custom_bwd)
    def backward(ctx, grad_out):
         ## do backward
         return grad_t1, grad_t2, grad_t3
</code></pre>
<p>I want t1 and t2 is <code>torch.float16</code> tensor, and t3 to be <code>torch.float32</code>. Now my solution is to manually add explicit type conversions like:</p>
<pre><code class="lang-auto">  t3 = t3.float()
</code></pre>
<p>I wanner:</p>
<ol>
<li>Is there a more elegant way of type conversion</li>
<li>Do I need to convert grad back to half in backward after manually converting the type of t3, such as <code>grad_t3.half()</code>
</li>
</ol>
<p>thanks for any help~</p>
Question about torch.transforms.resize

Ardeal: <p>Hi,</p>
<p>with the following input and code, the output is obviously wrong.<br>
is there anything wrong with my code about how to use <code>torch.transforms.Resize</code>?</p>
<p>Is there any detailed explanation about how <code>torch.transforms.Resize</code> is done?</p>
<pre data-code-wrap="python"><code class="lang-python">image = []
for i in range(9):
    row = []
    for j in range(9):
        row.append(i*10+j)
    image.append(row)
image = np.array(image)
image = torch.unsqueeze(torch.from_numpy(image).type(torch.float), 0)

tempimage = transforms.Resize((4, 4))(image)
</code></pre>
<h1><a name="input-1-channel-9x9-image-1" class="anchor" href="#input-1-channel-9x9-image-1"></a>input 1-channel 9x9 image:</h1>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/0/b04c6d81247b64740ddf7b8a80cc0e3b2eb802fb.png" alt="image" data-base62-sha1="p9BMofvd6Waps7Le2BwW6MDWbI7" width="678" height="295"></p>
<h1><a name="outout-1-channel-4x4-image-2" class="anchor" href="#outout-1-channel-4x4-image-2"></a>outout 1-channel 4x4 image</h1>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/5/65d0789493e33bdc611caa74e1a6e570d08760d6.png" alt="image" data-base62-sha1="ewGS2sYsyemYSbeBPeOJKe8W6jQ" width="316" height="160"></p>

tom: <p>Could you describe in more detail how it is obviously wrong?<br>
Quite likely, if the discussion in the documentation ( <a href="https://pytorch.org/vision/main/generated/torchvision.transforms.Resize.html" class="inline-onebox" rel="noopener nofollow ugc">Resize — Torchvision main documentation</a> ) is not enough, you’d want to look a the source code how bilinear interpolation is implemented.</p>
<p>Best regards</p>
<p>Thomas</p>

Ardeal: <p><a class="mention" href="/u/tom">@tom</a></p>
<p>Let me describe the details:</p>
<p>for the left-top corner of the input image, the element are : 0, 1, 2, 10, 11, 12,  but the corresponding element in output image are: 6.875.<br>
In my example, it is a down-sampling case the resize value should be in-between the input values.</p>
<p>Could you lease tell the source code link so that I could have a look?</p>

Ardeal: <p><a class="mention" href="/u/tom">@tom</a><br>
do you have any more comments about my questions?</p>
Where to find the reference note mentioned in pytorch source code?

cq_chu: <p>when I read the cpp code of pytorch, some reference meterial are listed there but I cannot find, such as: <a href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/core/dispatch/Dispatcher.h#L484" rel="noopener nofollow ugc">pytorch/Dispatcher.h at master · pytorch/pytorch (github.com)</a> and I cannot find the <code>Note: Argument forwarding in the dispatcher</code> . So where can I find these reference meterial?</p>

ptrblck: <p>Copy/pasting the posted text into the GitHub search points to <a href="https://github.com/pytorch/pytorch/blob/84a06d71936e61ceeee2abb9c9cb7bf5ee6440dd/aten/src/ATen/core/boxing/impl/make_boxed_from_unboxed_functor.h#L17">this section</a>.</p>

cq_chu: <p>Oh, I got it ~ Thank you !! <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=12" title=":smile:" class="emoji" alt=":smile:" loading="lazy" width="20" height="20"></p>
Transformers for time series forecasting

fatcat: <p>Hi everyone,<br>
I’m trying to implement a transformer model for time series forecasting. E.g., using the previous ten time steps x_1, x_2, …, x_10 to predict the next five time steps x_11,x_12,…,x_15. When predicting x_11, x_1 to x_10 should be fed into the encoder and x_10 should be fed into the decoder, and the output decoder is \hat{x_11}, i.e., the predicted value of x_11. When predicting the next time step x_12, should I append \hat{x_11} to the input of the encoder, i.e., should I use x_1, x_2,…,\hat{x_11} as the input of the encoder and \hat{x_11} as the input of the decoder to predict x_12?</p>

HamidGadirov: <p>Hi <a class="mention" href="/u/fatcat">@fatcat</a>, I think that is fine to use the output of the decoder (\hat{x_11) as a new input for your encoder since you don’t have ground truth for the for x_11. \hat{x_11 in this case will be your best approximation.</p>
Is there anyway to monitor the length of buffer of shuffler/prefetcher datapipe?

roytseng: <p>I’m benchmarking my data pipeline design. And I would like to know more information about the data pipeline at runtime.</p>
<p>For example, I would like to know the occupancy status of the prefetch buffer, to see if the buffer size setting is too large or small.</p>
<p>Or in another scenario, I want to benchmark the speed of a data pipeline but not consider the time to fill up the shuffle buffer. (Currently, I sleep the main thread for a while before trying to get data from the data iterator)</p>
<p>Really appreciate it if someone can shed some light on this!</p>

nivek: <p>We currently do not have that functionality; though we are tracking that <a href="https://github.com/pytorch/data/issues/210" rel="noopener nofollow ugc">potential feature as a GitHub issue</a>.</p>
<p>Feel free to comment on that issue to show your interest/use case and add any suggestion that you may have. We are also open to accept a PR that implement that feature.</p>
Module fusion with customized stride and padding in conv2d

superbtum: <p>I try to refer to this tutorial <a href="https://pytorch.org/tutorials/intermediate/custom_function_conv_bn_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Fusing Convolution and Batch Norm using Custom Function — PyTorch Tutorials 1.13.0+cu117 documentation</a> to practice module fusion in my personal project. But I found it difficult to use customized stride and padding settings as this may lead to shape being unmatched between X and grad_X. I try to add padding to the gradient matrix but sometimes the shape difference between X and grad_X is not an even number and it is tricky to add padding. Any idea is highly appreciated. Thanks.</p>

soulitzer: <p>Yes the backward implementation of conv2d does not support different strides and padding, instead you can probably use the convolution backwards implemented in <a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/grad.py" rel="noopener nofollow ugc">torch/nn/grad.py</a></p>

superbtum: <p>I made a minor change to the code, trying to make it adapt to different padding in the <code>convolution_backward</code> function:</p>
<pre><code class="lang-auto">  if X.size() != grad_X.size():
       diff_xaxis = X.size(-2) - grad_X.size(-2)
       diff_yaxis = X.size(-1) - grad_X.size(-1)
       p2d = (diff_yaxis &gt;&gt; 1, diff_yaxis - (diff_yaxis &gt;&gt; 1), diff_xaxis &gt;&gt; 1, diff_xaxis - (diff_xaxis &gt;&gt; 1))
       grad_X = F.pad(grad_X, p2d, "constant", 0.)
  if weight.size() != grad_input.size():
       diff_xaxis = weight.size(-2) - grad_input.size(-2)
       diff_yaxis = weight.size(-1) - grad_input.size(-1)
       p2d = (diff_yaxis &gt;&gt; 1, diff_yaxis - (diff_yaxis &gt;&gt; 1), diff_xaxis &gt;&gt; 1, diff_xaxis - (diff_xaxis &gt;&gt; 1))
       grad_input = F.pad(grad_input, p2d, "constant", 0.)
</code></pre>
<p>And for all conv and transpose conv, include the padding as well.<br>
Not sure if this is correct. But I found it slower with resnet50.</p>
RT semantic segmentation in Jetson nano

MD_DIDARUL_ISLAM: <p>Hello everyone,<br>
I have a semantic segmentation model in pytorch (DeepLabV3 architecture with MobileNetv3Large backbone). I want to deploy that model in jetson nano in real-time. I have 2 questions:</p>
<ol>
<li>When I do the prediction using jetson nano’s CPU, it takes around 7s for a frame, but for CUDA/GPU it takes like 38s. What am I doing wrong here ?</li>
<li>What other techniques I could apply to minimize my model, like pruning, truncate to make it faster? I have seen several approaches, but personally I was not successful for my model.<br>
I will be really grateful if somebody can help me.</li>
</ol>
<p>Thank you very much<br>
Didarul</p>

jcaip: <p>Hi Didarul,</p>
<p>For 1) it might be better to ask in Jetson Nano’s forums, they probably know better.<br>
For 2) you can check out our quantization tutorials <a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html" rel="noopener nofollow ugc">here</a> for examples on how to speed up your model with quantization.</p>

MD_DIDARUL_ISLAM: <p>Thank you very much, Jesse !</p>
A new idea about optimizer running and neural network design

ChineseBest: <p>Hi all, now I have many datasets with different feature number but same feature type. For example, d1=1000<em>200, d2=1000</em>400.<br>
Therefore, I intend to design a three layer neural network while for each dataset, they will share the later two layers but not the first layer. For the first layer, each dataset has their own firsrt layer dataset.</p>
<p>Therefore, my optimization is like:<br>
model = model()<br>
opt = opt(model.parameters())</p>
<p>opt.zero_grad()<br>
model.layer1 = layer[i]</p>
<p>pre = model(x)<br>
loss = (true, pre)</p>
<p>loss.backward()<br>
opt.step()</p>
<p>Is it a correct design? I think the first layer will also be optimized.</p>

ptrblck: <p>I don’t know where the <code>layer</code> list is defined but I would recommend to avoid manipulating the model after it’s creation since the parameter set could change and your optimizer might not have valid references to them. Since the shape of your input tensors differs maybe you could add a condition to the <code>forward</code> and use the appropriate layer for the current input?</p>

ChineseBest: <p>Thanks a lot. But since I did not observe any erros in my training and indeed I learnt something based on my model, it is still pretty confusing for me to understand the difference between your solution and my plan. I will try to explore it. Thanks a lot.</p>
Report only Rouge-2-F

AfonsoSalgadoSousa: <p>Hi. I wanted to report just Rouge-2-F, but I cannot find a way to do so. When I attach the RougeN metric, it is a single instance of ignite.metrics.Metric, but when printing it carries Rouge-2-P, Rouge-2-R, Rouge-2-F and a dictionary with the 3. From the dictionary I can get what I want, but want to drop the rest. How can I do it? Thanks in advance for any help you can provide.</p>

vfdev-5: <p>Hi <a class="mention" href="/u/afonsosalgadosousa">@AfonsoSalgadoSousa</a> , you can use metric arithmetics to get “Rouge-2-F” only value:</p>
<pre><code class="lang-python">from ignite.metrics import RougeN

# We define the metric as 
# m = RougeN(ngram=2, multiref="best")
# and apply getitem operator on resulting dictionary output:
m = RougeN(ngram=2, multiref="best")["Rouge-2-F"]

candidate = "the cat is not there".split()

references = [
    "the cat is on the mat".split(),
    "there is a cat on the mat".split()
]

m.update(([candidate], [references]))

print(m.compute())
&gt; 0.4
</code></pre>
<p>More on metrics arithmetrics:</p>
<ul>
<li><a href="https://pytorch.org/ignite/metrics.html#metric-arithmetics" class="inline-onebox" rel="noopener nofollow ugc">ignite.metrics — PyTorch-Ignite v0.4.10 Documentation</a></li>
</ul>
AttributeError: 'LinearPackedParams' object has no attribute '_load_state_dict_pre_hooks'

11137: <p>when i used the FX to create a quantized FXmodule ，i create on module.py file by "to_folder()"method of FXGraph。but i can not use this created module。<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/a/9aea99e9bbf5c4c111781740cd88c1f0031d72da.png" data-download-href="https://discuss.pytorch.org/uploads/default/9aea99e9bbf5c4c111781740cd88c1f0031d72da" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/a/9aea99e9bbf5c4c111781740cd88c1f0031d72da_2_681x499.png" alt="image" data-base62-sha1="m6s9ZQzQYRpPhAi3WlU2wUN5HlM" width="681" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/a/9aea99e9bbf5c4c111781740cd88c1f0031d72da_2_681x499.png, https://discuss.pytorch.org/uploads/default/optimized/3X/9/a/9aea99e9bbf5c4c111781740cd88c1f0031d72da_2_1021x748.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/9/a/9aea99e9bbf5c4c111781740cd88c1f0031d72da.png 2x" data-dominant-color="F2F1F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1143×838 102 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

11137: <p><code>Traceback (most recent call last):   File "quantizedmodule/module.py", line 31, in &lt;module&gt;     model = FxModule()   File "quantizedmodule/module.py", line 13, in __init__     self.load_state_dict(torch.load(r'/home/data/qxh/code/zyy_dlworks/test/quantizedmodule/state_dict.pt'))   File "/home/qxh22/.conda/envs/pytorch18/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1483, in load_state_dict     load(self)   File "/home/qxh22/.conda/envs/pytorch18/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1481, in load     load(child, prefix + name + '.')   File "/home/qxh22/.conda/envs/pytorch18/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1481, in load     load(child, prefix + name + '.')   File "/home/qxh22/.conda/envs/pytorch18/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1478, in load     state_dict, prefix, local_metadata, True, missing_keys, unexpected_keys, error_msgs)   File "/home/qxh22/.conda/envs/pytorch18/lib/python3.7/site-packages/torch/nn/quantized/modules/linear.py", line 84, in _load_from_state_dict     missing_keys, unexpected_keys, error_msgs)   File "/home/qxh22/.conda/envs/pytorch18/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1379, in _load_from_state_dict     for hook in self._load_state_dict_pre_hooks.values():   File "/home/qxh22/.conda/envs/pytorch18/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1186, in __getattr__     type(self).__name__, name)) AttributeError: 'LinearPackedParams' object has no attribute '_load_state_dict_pre_hooks'</code></p>

jcaip: <p>Hi, can you share the code you are using to save your state_dict?</p>
URL error for GTZAN?

Sam_Lerman: <p>Standard loading of GTZAN via <code>torchaudio.datasets.GTZAN</code> with <code>download=True</code>:</p>
<p><code>&lt;urlopen error [Errno 60] Operation timed out&gt;&gt;&gt;&gt;</code></p>

nivek: <p>Hi,</p>
<p>Looks like the issue is being <a href="https://github.com/pytorch/audio/issues/2743" rel="noopener nofollow ugc">tracked on GitHub</a>.</p>
Fast CRPS Metric

Samh: <p>Is there any sort of support for a method of getting the continuous ranked probability score (<a href="https://www.lokad.com/continuous-ranked-probability-score" class="inline-onebox" rel="noopener nofollow ugc">Continuous Ranked Probability Score (CRPS)</a>) for mixture density network quickly in pytorch to use as a loss function? Thanks</p>

jerryzh168: <p>this does not look like related to quantization, could you post in a different sub-category? thanks</p>

mgm: <p>Hi <a class="mention" href="/u/samh">@Samh</a> I have the same problem, so just wondering if you found a solution for it? Thanks</p>
Request Advice on Feature Importance

Jordan_Howell: <p>Hi,</p>
<p>Will anyone give some advice on how to get feature importance out of a model?  I have seen partial dependence plots used in tree based models but I need to be able to use it with a pytorch model I have developed.</p>
<p>Any advice on how I can figure out which of my variables are the most important?</p>

alldbi: <p>Hi,</p>
<p>I assume that the prediction of your model is differentiable with respect to the input. If so, you can compute the expectation of the absolute value of the gradient for the variables. Check this paper out: <a href="https://ieeexplore.ieee.org/document/8999179" rel="nofollow noopener">https://ieeexplore.ieee.org/document/8999179</a>.</p>

googlebot: <p>I haven’t yet tried it, but github has pytorch/captum project for this, <a href="https://captum.ai/" rel="nofollow noopener">https://captum.ai/</a></p>

Giorgio: <p>After two years, is still captum the to-go framework to decide what the best features are? thanks</p>
<p><a href="https://pytorch.org/tutorials/beginner/introyt/captumyt.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/beginner/introyt/captumyt.html</a></p>
What exactly is the motion field created by _gen_affine_grid()

Fourier_Transformati: <p>I moved the imgs 50 pixels towards the positive x-axis using <a href="https://pytorch.org/vision/stable/generated/torchvision.transforms.functional.affine.html" rel="noopener nofollow ugc">torchvision.transforms.functional.affine</a>.</p>
<p>The transformation is fine, however, since i would like to get the optical flow of the entire image, I went deeper into the source code and figure out that, it uses a <a href="https://pytorch.org/docs/stable/_modules/torch/nn/functional.html#affine_grid" rel="noopener nofollow ugc">grid</a> to create the new image.<br>
The deafault uses theta, which is the affine transformation matrix to compute the grid.<br>
In my case, the theta would be:<br>
[1 0 -50]<br>
[0 1    0]<br>
However, when i plot the grid [1,H,W,2] it is something like this:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/e/a/ea1f0c5b1ec12e8a340f26df66a10974a174016f.png" alt="image" data-base62-sha1="xp8fpX2LV8RAXMNjPEyMYoRdFVl" width="561" height="366"><br>
clearly we can see the 50, but i was expecting a uniform vector field that points to the right, I have no idea why i am getting a vector field like this, could someone exlpain what the grid is? what is its relation with optical flow?<br>
Both the source code of generating grid from theta, and using grid to transform the image is implemented in C, and the doc does not explain much.</p>
Stack expects each tensor to be equal size -- error

mfazio: <p>Hi,</p>
<p>I am getting this error which I understand why is happening but I don’t know how to fix it. <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=10" title=":confused:" class="emoji" alt=":confused:"></p>
<p>I am using a custom dataset to load images from a directory:</p>
<pre><code class="lang-auto">class DiffPatternsDataSet(Dataset):
    def __init__(self, main_dir):
        self.main_dir = main_dir
        self.all_imgs = os.listdir(main_dir)

    def __len__(self):
        return len(self.all_imgs)

    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()
        img_loc = os.path.join(self.main_dir, self.all_imgs[idx])
        image = torch.from_numpy(np.genfromtxt(img_loc)).unsqueeze(0)
        filename = self.all_imgs[idx]
        tiles = split_pattern(image, tile_size=80, stride=2)
        return tiles, filename
</code></pre>
<p>The split_pattern function basically splits an image using the Unfold function, then eliminates the patches that contain only white noise. Therefore, for each image I get a tiles tensor of dimension (<span class="hashtag">#tiles</span>, channels, width, height) and the number of tiles is different depending on the image. So when I use the loader, it wants to do a torch.stack and finds tensors of different dimensions.</p>
<p>Is there any solution for this? Maybe I am not approaching the problem correctly. I want to train my model with the tiles I get from the images (plus each tile needs to have the original image filename attached). Maybe I should do things differently in the getitem?</p>
<p>Any help or input will be greatly appreciated!</p>

mfazio: <p>Just for future reference if anyone sees this: I solved my problem by using a custom collate_fn for the loader. Instead of using torch.stack I generated a list with all my tiles so the tensor size mismatch was no issue.</p>

CHIEF: <p>Would you be able to share your custom collate_fn code please?</p>
YOLO Negative class in Pytorch?

jjaistein: <p>Hi all, While training for YOLO darknet, I can leave an annotation file empty and still darknet accepts the image and considers it as a negative class on a whole. But when I tried to do the same in PyTorch (providing no bounding box) I got an error while training stating that the tensor was empty.</p>
<p>Can someone suggest me how to specify negative classes in PyTorch ?</p>
Reduce number of output parameters of a trained ResNext model

unimatrix007: <p>I have a trained ResNext model, with <code>model.fc.in_features = 2048</code>. I want to remove the last fc classification layer and have just the 2048 feature vector.<br>
Replacing the <code>module.fc = nn.Identity()</code> solves this, but I was wondering how could I reduce this feature vector to eg. 512 without retraining?<br>
Setting <code>module.fc = nn.MaxPool1d(8, stride=8)</code> throws<br>
<code>RuntimeError: Expected 3-dimensional tensor, but got 2-dimensional tensor for argument #1 'self' (while checking arguments for max_pool1d)</code></p>
How to choose the size of hidden size value?

laro: <p>According to some posts I read, the <strong>hidden_size</strong> parameter (GRU) affect underfitting or overfitting.</p>
<p>I read about some rules of thumb (choosing <strong>hidden_size</strong>):</p>
<ol>
<li>value between the input layer size … and the output layer size.<br>
or</li>
<li>(Number of inputs + outputs) * (2/3)</li>
</ol>
<p>So if I have:</p>
<ul>
<li>input sequence with the the shape (10, 2) (i.e sequence of length 10 with 2 features)</li>
</ul>
<p>And</p>
<ul>
<li>output is (5,1) (i.e sequence of length 5 with 1 feature)</li>
</ul>
<p>Do I need to set the <strong>hidden_size</strong> with the value of 10 ? (10+5)*2/3 ?</p>
<ul>
<li>Am I right ?</li>
</ul>
<p>I saw a lot of examples (from kaggle) of models using GRU with hidden_size between 32-256.</p>
<p>According to my example above, what is the right (make sense) value of <strong>hidden_size</strong> ?</p>
This attribute exists on the Python module, but we failed to convert Python type: 'list' to a TorchScript type

HwanUng_Lee: <p>An error occurs when executing the code below.</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

class AAA(nn.Module):
    def __init__(self):
        super(AAA, self).__init__()

    def forward(self, x):
        return x


class BBB(nn.Module):
    def __init__(self, other: AAA=None):
        super(BBB, self).__init__()

        print(type([other]))
        self.other = [other]

    def forward(self, x):
        src = self.other[0]           # !!!Error!!!
        return x


if __name__ == '__main__':
    from utils.functions import init_console
    init_console()

    net_a = AAA()
    net_b = BBB(other=net_a)

    script_a = torch.jit.script(net_a)
    script_b = torch.jit.script(net_b)

    print('done')
</code></pre>
<blockquote>
<p>Module ‘BBB’ has no attribute ‘other’ (This attribute exists on the Python module, but we failed to convert Python type: ‘list’ to a TorchScript type. Could not infer type of list element: Cannot infer concrete type of torch.nn.Module. Its type was inferred; try adding a type annotation for the attribute.):<br>
File “C:/xxx/test.py”, line 20<br>
def forward(self, x):<br>
src = self.other[0]<br>
~~~~~~~~~~ &lt;— HERE<br>
return x</p>
</blockquote>
<p>Is there a way to make it work?</p>

nima_rafiee: <p>Hey</p>
<p>Have you found any solution to this problem?</p>
Is there something wrong with the backward function?

jaycola99: <pre><code class="lang-auto">model = UNet(3, 2)  #input_channels, classes
model.to(device=device)

criterion = nn.CrossEntropyLoss()  
optimizer = torch.optim.Adam(model.parameters(), 0.001)  
scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, 'max')
grad_scaler = torch.cuda.amp.GradScaler(enabled=False)

for epoch in range(epochs):
    model.train()  
    for batch in train_data_loader:
        epoch_loss = 0
        image = batch['image']   #torch.as_tensor(img.copy()).float().contiguous()
        label = batch['label']     #torch.as_tensor(mask.copy()).long().contiguous()
        image = image.to(device=device, dtype=torch.float32)
        label = label.to(device=device, dtype=torch.long)
        with torch.cuda.amp.autocast(enabled=False):
            logits = model(image)
            loss = criterion(logits, label)

        # backprop
        optimizer.zero_grad(set_to_none=True)  
        grad_scaler.scale(loss).backward()
        grad_scaler.step(optimizer)
        grad_scaler.update()
        epoch_loss += loss.item()

    print('epoch', epoch, ' loss:', loss.item())
</code></pre>
<p>error info</p>
<pre><code class="lang-auto">/tmp/ipykernel_22/2557856581.py in &lt;module&gt;
     34         # backprop
     35         optimizer.zero_grad(set_to_none=True)  # 梯度清0
---&gt; 36         grad_scaler.scale(loss).backward()
     37         grad_scaler.step(optimizer)
     38         grad_scaler.update()

/opt/conda/lib/python3.7/site-packages/torch/_tensor.py in backward(self, gradient, retain_graph, create_graph, inputs)
    361                 create_graph=create_graph,
    362                 inputs=inputs)
--&gt; 363         torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
    364 
    365     def register_hook(self, hook):

/opt/conda/lib/python3.7/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)
    173     Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
    174         tensors, grad_tensors_, retain_graph, create_graph, inputs,
--&gt; 175         allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
    176 
    177 def grad(

RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED
</code></pre>

srishti-git1110: <aside class="quote no-group" data-username="jaycola99" data-post="1" data-topic="165338">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jaycola99/48/54636_2.png" class="avatar"> jaycola99:</div>
<blockquote>
<p><code>CUDNN_STATUS_NOT_INITIALIZED</code></p>
</blockquote>
</aside>
<p>Hi,<br>
Please try following <a href="https://discuss.pytorch.org/t/cuda-error-cublas-status-internal-error-when-calling-cublascreate-handle/114341/10">this</a> thread and see if it helps.</p>

jaycola99: <p>thanks. I found the error, the number of categories changed after data pre-processing.</p>
How to predict the label/probability for an input of multiple similar views of an object with a sharing weights model?

Chenoille: <p>I want to do a binary classification on an particular object of which I have n images of it at different angles. I want to know if the object contains one feature or not, and containing this feature is dominant in the classification step. In other words, if I see that feature in at least one of the n views, the complete model should classify the input as “containing the feature”. I have one label for all n images (angles of view). The object images taken respect a certain symmetry, such that each views could be interchanged without affecting the output. For example, my object could be a cylinder and the views are obtained when we rotate around its center axis, thus all n views looks similar.</p>
<p>To classify, I want to do the following:</p>
<ol>
<li>Use the same model for each view, (sharing weights)</li>
<li>and to terminate via special voting along the n outputs of the same model.</li>
</ol>
<p>Step 1 is to be reasonable since all my objects view are similar.</p>
<p>Now, I don’t know what is the most efficient way for the voting part. I guess I should not learn it since I know the voting conditions. But is it better to output a label out of voting or is it possible to output some values for later binary cross-entropy?</p>
<p>I thank you in advance for your help.</p>

azhanmohammed: <aside class="quote no-group" data-username="Chenoille" data-post="1" data-topic="165158">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/chenoille/48/45737_2.png" class="avatar"> Chenoille:</div>
<blockquote>
<p>and to terminate via special voting along the n outputs of the same model.</p>
</blockquote>
</aside>
<p>A better way to solve this might be using <a href="http://vis-www.cs.umass.edu/mvcnn/" rel="noopener nofollow ugc">MultiView CNN network</a>. It takes multiple images from different views of the same object, passes them through the same network, just like you want in point 1. And then gives the final output. It might help you in your use case for sure.</p>

Chenoille: <p>Thank you, that is interesting and I will test it.</p>
Torchvision transforms ToTensor not working

Observarun: <p>I’m working with spectrograms, each image is 288 * 432. ToTensor() isn’t working for me, I believe it should have brought height and width of each one between 0 and 1.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/d/8/d885c70f62910281e907a3a16f79ac978c5d2866.png" alt="Picture1" data-base62-sha1="uTrKxUqTWjzvTrX1RuzaVXDwFqC" width="481" height="315"><br>
Can someone please help?<br>
Thank you.</p>

srishti-git1110: <p>Hi,<br>
Please refer to the <a href="https://pytorch.org/vision/stable/generated/torchvision.transforms.ToTensor.html" rel="noopener nofollow ugc">docs</a> and check whether your images do not fall into the cases when the image is returned without scaling. (range should be 0-255 etc.)</p>

Observarun: <p>Thanks, I understand. What do you think is a good way to bring the resolution in [0-255]? RandomResizedCrop()? Or something else?</p>
<p>Thank you.</p>

srishti-git1110: <aside class="quote no-group" data-username="Observarun" data-post="3" data-topic="165261">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/observarun/48/46518_2.png" class="avatar"> Observarun:</div>
<blockquote>
<p>RandomResizedCrop()</p>
</blockquote>
</aside>
<p>This is used to re-size the image (change the dimensionality) rather than to interfere with the pixel values.</p>
<p>As long as the pixel values are in the range 0-255 and the PIL image is in one of the modes (L, LA, P, I, F, RGB, YCbCr, RGBA, CMYK, 1), the scaling should work fine. See:</p>
<pre><code class="lang-auto">import numpy
from PIL import Image
import torch
import torchvision.transforms as transforms

imarray = numpy.random.rand(300, 300, 3) * 255
im = Image.fromarray(imarray.astype('uint8')).convert('RGB')

trans = transforms.ToTensor()
im_tens = trans(im)

im_tens = torch.where(im_tens&gt;1.0, 1.0, 0.0)
print(im_tens.sum()) # tensor(0.)
</code></pre>
<p>You could use manual scaling in case your pixel values are outside 0-255.</p>

Observarun: <p>Thanks, Srishti. I was trying manual transformation. Wrote the following, not sure why it’s not working.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/d/3df32c3c846f11478b94050d121ed06f7ac1fb89.png" alt="Picture1" data-base62-sha1="8Q27vbjk8qDVdlLR0ZNIAvCyU53" width="163" height="120"></p>
<p>Could you please have a look?</p>
<p>Thank you.</p>

srishti-git1110: <p>Hi,<br>
Could you please post your code enclosed with three backticks ```.<br>
Please also provide an example of what transformations you’d like to do with your images.</p>