akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Too much gap between train and validation accuracy

username: Donato_Caf
cooked: <p>Hi, I’m doing Transfer Learning on a senet50 pretrained model, but I got weird results on the accuracy.<br>
After one epoch my train accuracy is about 70% while validation on 25%. I read something about BatcNorm so I tried to retrain, setting to eval, rise the momentum, add batch accumulation but I got the same results.<br>
What can I do? The code looks fine.<br>
Train set contains 211588 images while validation 65060</p>
Custom object detected segmentation and detect colors of object

username: nrllah
cooked: <p>hi everyone. ı am new this topic.</p>
<p>I want to make a program about detect products and their color for fashion. ı have some dataset there is no problem. ı wanted to start with tensorflow for this project. but my friend said use pytorch. ı am searching but ı havent good tutorial for my project.</p>
<p>Can you help me pls ?  thank you</p>
Can we split a large, pytorch-built-in `nn.Module` to multiple-gpu?

username: JiaRu2016
cooked: <p>eg. I have and <code>nn.Linear</code> whose <code>in_features = 500e4</code> and <code>out_features = 3000</code>, so number of trainable parameters consume 55 GB (500e4 * 3000 * 4 / 1024 ** 2) memory. My single GPU has 12 GB and I have 8 GPU on one machine. How can I parallel this “atomic” built-in module on multiple gpu?</p>

username: albanD
cooked: <p>Hi,</p>
<p>I’m afraid we don’t provide any construct to do this automatically.<br>
But you can simply create 8 different Linear that each take a subset of the input and split the input yourself and call each of these Linears and then add all the results (assuming your split on the input size here given that it is the biggest).</p>
Conditional execution of forward hooks

username: ErikJ
cooked: <p>I am using some forward hooks on intermediate layers for debugging/visualisation purposes and I am wondering if it is possible to execute the “<em>hook</em>” code only on each <em>n-th</em> batch? In other words is there any global modifier which toggles <em>hook</em> execution on and of?</p>

username: mariosasko
cooked: <p>I see two options.</p>
<p>When registering a hook, the handle is returned that can be used as a context-manager. When this context-manager exits, the hook gets deactivated. This is not probably documented in the offical docs but you can check the source code for that.</p>
<p>The snippet below shows how this works:</p>
<pre><code class="lang-python">with my_net.layer.register_forward_hook(my_hook):
    # here the hook is active
    my_net(input)

# here the hook no longer gets called
</code></pre>
<p>Or you could write your own decorator that executes a hook every N calls.</p>
<pre><code class="lang-python">def execute_every(n):
    def decorator(f):
        cnt_exec = 0
        
        @functools.wraps(f)
        def wrapper(*args, **kwargs):
            nonlocal cnt_exec
            print(cnt_exec)
            cnt_exec += 1
            if cnt_exec % n == 0:
                f(*args, **kwargs)
    
        return wrapper
    return decorator

@execute_every(5)
def my_hook(...):
    ...

my_net.layer.register_forward_hook(my_hook)
</code></pre>

username: ErikJ
cooked: <p>Oh it’s that simple <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> Cool and thank you!</p>
Tensordot does not support Bool

username: gabinkbl
cooked: <p>It seems that tensordot does not support tensors of dtype Bool (unlike numpy).<br>
a = torch.tensor([[False, True], [False, True]])<br>
b = torch.tensor([[False, True], [False, True]])<br>
torch.tensordot(a, b, dims=1)<br>
Traceback (most recent call last):<br>
File “”, line 1, in <br>
File “…\torch\functional.py”, line 705, in tensordot<br>
return _VF.tensordot(a, b, dims_a, dims_b)<br>
RuntimeError: _th_addmm_out not supported on CPUType for Bool</p>
<p>A possible workaround:<br>
torch.tensordot(a.float(), b.float(), dims=1).bool()<br>
requires doing redundant operations.</p>
<p>I believe, it would be useful for tensordot to support Bool.</p>

username: albanD
cooked: <p>Hi,</p>
<p>You can check if there is already an issue asking for this feature.<br>
But we will most likely be happy to accept a PR adding this.</p>

username: gabinkbl
cooked: <p>Thanks. I Will do this.</p>

username: agolynski
cooked: <p>Xref <a href="https://github.com/pytorch/pytorch/issues/44969" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/44969</a></p>
Backward pass only uses ~50% GPU utilization?

username: gebbissimo
cooked: <p>Hello there!</p>
<p>The graph below visualizes my GPU utilization per timestamp as a progress bar. It is mostly 99%, but has some drops in between. Interestingly, those drops correlate consistently to the times of the “loss.backward()” call. Any idea why the backward pass might not use 100% GPU or how to improve it?</p>
<p>If helpful: The final loss is an L1-loss between two RGB images. One of those images is created using F.grid_sample() and some pinhole camera backprojection and projection functions.</p>
<p>Thanks in advance!</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/7/27c42e5bba003a27ab7d365f19fce5ee39c53a9b.png" data-download-href="https://discuss.pytorch.org/uploads/default/27c42e5bba003a27ab7d365f19fce5ee39c53a9b" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/7/27c42e5bba003a27ab7d365f19fce5ee39c53a9b.png" alt="image" data-base62-sha1="5FMW7jRo2av6lBJBwiLCEuQBqk3" width="229" height="500" data-dominant-color="484949"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">352×768 8.81 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
CUDA error: invalid argument (getDeviceFromPtr

username: Jakov_Gaspar
cooked: <p>Hi,</p>
<p>I’m loading a model trained in python via torchscript to C++.<br>
I’m doing it according to the tutorial: <a href="https://pytorch.org/tutorials/advanced/cpp_export.html" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/cpp_export.html</a></p>
<p>I have a network operating with CUDA.<br>
I’m transforming the input tensor from a vector via tensor::from_blob (data is a vector containing all values of a matrix) :</p>
<p>torch::Device device(at::kCUDA);<br>
inputs.push_back(torch::from_blob(data.data(), {375,375}, device));</p>
<p>I  get this error:</p>
<p>terminate called after throwing an instance of ‘c10::Error’<br>
what():  CUDA error: invalid argument (getDeviceFromPtr at …/…/aten/src/ATen/cuda/CUDADevice.h:13)</p>
<p>i don’t know what to do, if I set the input to torch::ones everything works fine:</p>
<p>torch::Device device(at::kCUDA);<br>
inputs.push_back(torch::ones({375, 375}, device));</p>
<p>(no error)</p>
<p>Thank you</p>

username: Jakov_Gaspar
cooked: <p>Found the solution and posting it here, in case someone else needs it. i had to call insert a clone to make everything work:</p>
<p><code>inputs.push_back(torch::from_blob(data.data(), {375,375}).clone().to(device));</code></p>

username: kb1ooo
cooked: <p>The clone is not required for me.  Just make sure you don’t pass the device in the options argument for <code>torch::from_blob</code>.</p>
Cannot train my model on ImageNet in a reasonable time

username: tal123
cooked: <p>I built a custom model that is similar to MobileNet-v2 in its structure, but has less layers overall. The problem is that my model is useless, as I cannot afford fully training it on ImageNet.</p>
<p>I work on a PC with a GeForce GTX 1060 GPU with 6GB, so I cannot use the settings from the torchvision training reference. All the workarounds I tried, using different kinds of subsets of the ImageNet dataset with different learning-rate strategies and different batch sizes have failed to produce a usable model that can be transferred for other tasks.</p>
<p>Are there any recommendations what I can do to have the model pretrained?</p>
<p>Specifically, I’d be glad to know:</p>
<ol>
<li>whether you think Knowledge Distillation should make a difference</li>
<li>whether there exists any service that lets you just upload a Python source file with a PyTorch network class and that automatically trains it for you on ImageNet, so you can later download its trained weights?</li>
</ol>
Mixed precision and Spectral norm

username: dougsouza
cooked: <p>I started playing around with new Amp interface. The thing is: I am training GANs and my models use spectral norm.</p>
<p>I would like to know how things work with mixed precision when using spectral norm. Are my spectraly normalized weights eligible for fp16 precision? Do I need to do something extra to get things working (such as increasing spectral norm eps?)</p>
<p>Thanks</p>

username: ptrblck
cooked: <p>AMP will autocast operators as described in <a href="https://pytorch.org/docs/stable/amp.html#ops-that-can-autocast-to-float16">this list</a>. Since <code>spectral_norm</code> uses some of them (e.g. <code>matmul</code>) this operation would be performed in FP16. If you want to keep the calculation in FP32, you can disable <code>autocast</code> for this call.</p>

username: dougsouza
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>, thank for you answer. I have some questions about AMP, maybe eu can clarify for me.</p>
<p>1 - About weight initilization: let’s say my initializer produces very small weights, will them flush to zero when run inside autocast?<br>
2 - What about model inputs: if I am working with vector of very small floats as inputs to my model, what happens to them?</p>
<p>Thanks</p>

username: ptrblck
cooked: <ol>
<li>No, that shouldn’t be the case as the compute and accumulate <code>dtype</code> would still be performed in FP32 even if FP16 inputs are passed.</li>
<li>If depends on the range of your values and if they are representable. Theoretically it would be possible, e.g. via:</li>
</ol>
<pre><code class="lang-python">x = torch.tensor(2**(-149)).cuda()
print(x)
&gt; tensor(1.4013e-45, device='cuda:0')
print(x.half())
&gt; tensor(0., device='cuda:0', dtype=torch.float16)
</code></pre>
<p>but then I would doubt that the small FP32 values would contribute to your model in any way.<br>
Note that a <code>GradScaler</code> should be used for mixed-precision training together with <code>autocast</code> to avoid gradient underflow.</p>
<p>It would be interesting to know more about your use case, i.e. what kind of model “needs” values close to zero to perform properly.</p>

username: dougsouza
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>, there is no requirements to have inputs close to zero. I just wanted to know how AMP handled these cases.</p>
<p>Thanks for your help!</p>
how to encrypt binary model file with libtorch in an offline environment

username: xinheblue
cooked: <p>we have a model file trained with pytorch python api and deployed offline in client’s computer with libtorch c++ api to do inference, to prevent our competitors to use our model to produce exactly the same result, we need to encrypt the model, that is, the model file can only infered with a given password in the program, rather than just loading the model and pass the picture and get a result</p>
<p>how can we manage to do this, thanks!</p>

username: MSA
cooked: <p>Hi,<br>
<a class="mention" href="/u/xinheblue">@xinheblue</a> did you find a solution with Pytorch? We can probably encrypt the .pt with RSA encryption but for now, we didn’t test anything.</p>
<p>Thanks,</p>
Can't install previous PyTorch versions with Pycharm

username: Kimonili
cooked: <p>Hi,</p>
<p>I want to use the torch 1.5.1 and torchvision 0.6.1. I accidentally downloaded the current latest version of pytorch but I run on a bug after some several runs related with the sampling from a categorical distribution.</p>
<p>For that reason, I wanted to downgrade to a version that the same code had worked perfectly in the past. I tried all of these <a href="https://pytorch.org/get-started/previous-versions/#linux-and-windows" rel="nofollow noopener">commands</a> but despite the fact that they were installed succesfully based on Pycharm’s terminal message, when I import them in the code I get this <code>ModuleNotFoundError: No module named 'torch</code>.</p>
<p>Does anyone know why is this happening?</p>
TypeError: torch.nn.functional.relu is not a Module subclass

username: Anand_Deep
cooked: <p>I am trying to create a pytorch model and add relu activation after every linear layer. But I am getting</p>
<pre><code class="lang-auto"> TypeError : torch.nn.functional.relu is not a Module subclass
</code></pre>
<p><strong>Model Class :</strong></p>
<pre><code class="lang-auto">class Model(nn.Module):
    def __init__(self, in_features=4, out_features=3):
        super().__init__()
        
        self.net = nn.Sequential(
            nn.Linear(in_features, 8),
            F.relu,

            nn.Linear(8,9),
            F.relu,

            nn.Linear(9, out_features) )

    def forward(self,x):
       x = self.net(x)
       return x
</code></pre>

username: Diego
cooked: <p><code>nn.Sequential</code> can only be populated with pytorch modules (<code>nn.Module</code>) class, and <code>F.relu</code> is just a function, not a module. You should change the <code>F.relu</code> in your code to <code>nn.ReLU()</code>, dont forget the parethesis <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: Anand_Deep
cooked: <p>thanks ! you saved my day</p>
How to implement onnx::slice op in pytorch?

username: alanzhai219
cooked: <p>I need to implement a net according to onnx which has a slice op. But it seems there is no such op in pytorch. How to implement it by using other ops?<br>
</p><aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Slice" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Slice" target="_blank" rel="nofollow noopener">onnx/onnx/blob/master/docs/Operators.md#Slice</a></h4>
<pre><code class="lang-md">## Operator Schemas
*This file is automatically generated from the
            [def files](/onnx/defs) via [this script](/onnx/defs/gen_doc.py).
            Do not modify directly and instead edit operator definitions.*

For an operator input/output's differentiability, it can be differentiable,
            non-differentiable, or undefined. If a variable's differentiability
            is not specified, that variable has undefined differentiability.

### ai.onnx (default)
|**Operator**|**Since version**|
|-|-|
|&lt;a href="#Abs"&gt;Abs&lt;/a&gt;|&lt;a href="Changelog.md#Abs-13"&gt;13&lt;/a&gt;, &lt;a href="Changelog.md#Abs-6"&gt;6&lt;/a&gt;, &lt;a href="Changelog.md#Abs-1"&gt;1&lt;/a&gt;|
|&lt;a href="#Acos"&gt;Acos&lt;/a&gt;|&lt;a href="Changelog.md#Acos-7"&gt;7&lt;/a&gt;|
|&lt;a href="#Acosh"&gt;Acosh&lt;/a&gt;|&lt;a href="Changelog.md#Acosh-9"&gt;9&lt;/a&gt;|
|&lt;a href="#Add"&gt;Add&lt;/a&gt;|&lt;a href="Changelog.md#Add-13"&gt;13&lt;/a&gt;, &lt;a href="Changelog.md#Add-7"&gt;7&lt;/a&gt;, &lt;a href="Changelog.md#Add-6"&gt;6&lt;/a&gt;, &lt;a href="Changelog.md#Add-1"&gt;1&lt;/a&gt;|
|&lt;a href="#And"&gt;And&lt;/a&gt;|&lt;a href="Changelog.md#And-7"&gt;7&lt;/a&gt;, &lt;a href="Changelog.md#And-1"&gt;1&lt;/a&gt;|
|&lt;a href="#ArgMax"&gt;ArgMax&lt;/a&gt;|&lt;a href="Changelog.md#ArgMax-13"&gt;13&lt;/a&gt;, &lt;a href="Changelog.md#ArgMax-12"&gt;12&lt;/a&gt;, &lt;a href="Changelog.md#ArgMax-11"&gt;11&lt;/a&gt;, &lt;a href="Changelog.md#ArgMax-1"&gt;1&lt;/a&gt;|
|&lt;a href="#ArgMin"&gt;ArgMin&lt;/a&gt;|&lt;a href="Changelog.md#ArgMin-13"&gt;13&lt;/a&gt;, &lt;a href="Changelog.md#ArgMin-12"&gt;12&lt;/a&gt;, &lt;a href="Changelog.md#ArgMin-11"&gt;11&lt;/a&gt;, &lt;a href="Changelog.md#ArgMin-1"&gt;1&lt;/a&gt;|
|&lt;a href="#Asin"&gt;Asin&lt;/a&gt;|&lt;a href="Changelog.md#Asin-7"&gt;7&lt;/a&gt;|
</code></pre>

  This file has been truncated. <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Slice" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


username: freezek
cooked: <p>you mean slice operation? maybe this help:</p>
<p><a href="https://pytorch.org/cppdocs/notes/tensor_indexing.html" rel="nofollow noopener">https://pytorch.org/cppdocs/notes/tensor_indexing.html</a></p>
Unable to create custom pytorch dataset class for .npy image file

username: IamSparky
cooked: <p>I want to create custom pytorch dataset class  , but getting the below error :–&gt;<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/9/f978cd741ef3b83cc7623e0242086dea2d992051.png" data-download-href="https://discuss.pytorch.org/uploads/default/f978cd741ef3b83cc7623e0242086dea2d992051" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/9/f978cd741ef3b83cc7623e0242086dea2d992051_2_690x173.png" alt="image" data-base62-sha1="zAVHV9Tq7XOx78mSbw1Ik2gD5Qt" width="690" height="173" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/9/f978cd741ef3b83cc7623e0242086dea2d992051_2_690x173.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/9/f978cd741ef3b83cc7623e0242086dea2d992051_2_1035x259.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/9/f978cd741ef3b83cc7623e0242086dea2d992051_2_1380x346.png 2x" data-dominant-color="F7F7FA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1479×371 26.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Here’s the link to <a href="https://colab.research.google.com/drive/1qXhwwDZsqCk2Tb76H-y7mWLNLYeC5Zsx?usp=sharing" rel="noopener nofollow ugc">my notebook</a> and <a href="https://drive.google.com/file/d/1Y5GLe1WcklZNZYxIR20X7gsEAH8bvDiS/view?usp=sharing" rel="noopener nofollow ugc">image data</a></p>
<p>please help me in resolving the problem .</p>

username: KaiHoo
cooked: <pre><code class="lang-auto">img = cv2.resize(img, dsize=(128, 128), interpolation=cv2.INTER_CUBIC)
img = self.aug(img)
img = np.transpose(img, (2,0,1)).astype(np.float32) 
</code></pre>
<p>pytorch transforms only works on tensors, but you apply them on numpy arrays: img = self.aug(img)</p>

username: IamSparky
cooked: <p>How to resolve this part so that I can run my dry run testing syntaxes?</p>
<pre><code class="lang-auto">img = train_data[0]

plt.imshow(np.transpose(img, (1,2,0)) , cmap='gray')
</code></pre>
Difference in unstructured pruning with L1 vs L2 norm

username: Xu_Han
cooked: <p>Question as simple as the title. What is the difference in results between L1Unstructured and LnUnstructured when using the torch.nn.utils.prune class. If say, I pruned 20% a dense layer’s weights with L1 and L2 norm,  respectively, wouldn’t both simply prune the lowest 20% of the weight elements in absolute value?</p>
Sample selection in RoI (Faster/Mask R-CNN)

username: sigma_x
cooked: <p>In my dataset there are 3 classes (+background)<br>
I set</p>
<pre><code class="lang-auto">'box_positive_fraction':0.75
'box_batch_size_per_image':1024
</code></pre>
<p>in the Mask R-CNN constructor arguments. Yet after the sampling (<code>select_training_samples</code>) there are roughly 90% Class 0, 6% Class 1, 2% Class 2 and 3, which is passed on to the loss function. Why is the final breakdown different from the 0.75? Is it how it’s supposed to work?</p>
DataParallel throws an error “AttributeError: 'DataParallel' object has no attribute 'loss'”

username: jiang_ix
cooked: <p>Hello. I’m implementing my model on multi gpus. But it throws an error “AttributeError: ‘DataParallel’ object has no attribute ‘loss’”. So I want to know whether the DataParallel only duplicate the forward() function of our model ? And isn’t the loss() function duplicated? So, how should I do? Thanks a lot.</p>
<pre><code class="lang-auto">class MyModule1(nn.Module):
  def __init__(self):
    super(MyModule1, self).__init__()
    self.fc1 = nn.Linear(2, 1)

  def forward(self, x):
    x = self.fc1(x)
    return x

  def loss(self, x):
    loss = ......
    return loss

model1 = MyModule1()
model1 = DataParallel(model1)
model1.to('cuda:0')
x = torch.tensor([10, 5], dtype=torch.float).to('cuda:0')
print model1.loss(x)
</code></pre>

username: blackberry
cooked: <p>Because, <code>model1</code> is now an object of class <code>DataParallel</code>, and it indeed does not have such a function or attribute.<br>
You should do <code>model1.module.loss(x)</code><br>
But, then, it will run only on one GPU.</p>

username: ptrblck
cooked: <p>This, or if it’s possible you could try to call <code>self.loss</code> in your <code>forward</code>.<br>
(Not sure if that fits your use case <a class="mention" href="/u/jiang_ix">@jiang_ix</a> )</p>

username: lxy444
cooked: <p>So how should I do if I want use multiple gpus?</p>

username: Xiaokang_Wang
cooked: <p>So we can do not data.parallelization on a customized method. Could someone confirm this? If so, I will not search in this directory.</p>

username: ptrblck
cooked: <p>You would have to call your custom method in <code>forward</code> to make it work with <code>nn.DataParallel</code>.</p>
Intuition for batch[:,c] in the book

username: Sokhib_Tukhtaev
cooked: <p>I am reading a book “Deep learnin with Pytorch” and the code is like below:</p>
<pre><code class="lang-auto">print(batch.shape) # torch.Size([4, 3, 256, 256])
batch /= 255.0

n_channels = batch.shape[1]
for c in range(n_channels):
    print("smth.shape", smth.shape)
    mean = torch.mean(batch[:,c])
    std = torch.std(batch[:,c])
    batch[:, c] = (batch[:, c] - mean) / std
</code></pre>
<p>I can’t get the intuition of <code>batch[:,c]</code>. It returns <code>torch.Size([4, 256, 256])</code> where <code>4</code> is batch_size, <code>256</code> is <code>height</code> and <code>width</code> of an image. <code>:</code> means everything and <code>c</code> is looped over 0, 1 and 2. In the first iteration it will be <code>batch[:,0]</code>. Why is that returning <code>torch.Size([4, 256, 256])</code>?</p>

username: tom
cooked: <p>You index into the second index position (dimension 1) of the tensor [4, <strong>3</strong>, 256, 256], so that dimension is dropped in the resulting view.</p>

username: JVGD
cooked: <p>Yes, you are right that gives you a tensor of <code>torch.Size([4, 256, 256])</code>.</p>
<p>The intuition here is that you want to normalize image before feeding it to the network. The normalization usually comprises 2 operations: make the input distribution a 0-mean and 1-std. In this piece of code you are normalizing per channel as each channel (or each RGB color) distribution might have different characteristics so you want to normalize per channels. This assumes there is a correlation between channels of the same color, which might makes sense.</p>
<p>However in practice I found that you can also normalize the whole batch of images aggregating all the channels without any issue:</p>
<pre><code class="lang-auto"># torch.Size([4, 3, 256, 256]) 
print(batch.shape)

# Norm
batch_mean = batch.mean()
batch_std = batch.std()
batch = (batch - batch_mean) / batch_std
</code></pre>
<p>You can also take a look at <code>albumentaitons</code> library specifically to <code>albumentations.transforms.Normalization</code> which does the same automatically for you</p>

username: tom
cooked: <p>Yeah, so from the top of my head, I think the books mentions that depending on your problem type, you’d take (per-channel) averages over the entire Dataset. This is what is usually done for image classifiers (ImageNet).<br>
I’m sure you can get good results without splitting by channel, given that the mean is something like [0.485, 0.456, 0.406] and the std [0.229, 0.224, 0.225], so they’re not that far apart.<br>
When you use the batch, it is a bit funny because the image depends on its neighbors, I’d probably avoid that just because I’d feel dirty if I did that. On the other hand batch norm is arguably quite successful.</p>
<p>Best regards</p>
<p>Thomas</p>

username: JVGD
cooked: <p>Yeah you are completely right! Better not to normalize the whole batch!</p>
<p>I took a look to my code and the normalization is being done in a subclass of <code>torch.utils.data.Dataset</code>. So you are right. It is better just to normalize by sample.</p>
Transfer Learning: Best practices

username: bing
cooked: <p>Hi,<br>
I am trying to use transfer learning for image classification.<br>
Can anyone suggest best practices,<br>
I have seen multiple tutorial where few people adds single fc layers after removing the initial fc layer<br>
Also, it seems like a rabbit hole where we can freeze-unfreeze the different number of sub-modules. It seems not very efficient to me.<br>
Can you guys share your best practices, thumb rule etc. (I know it is a very comprehensive question but I am thinking a lot and really not able to converge at a precise mental model on how to approach it.)</p>

username: ptrblck
cooked: <p>I don’t think there is a simple answer to this question.</p>
<aside class="quote no-group" data-username="bing" data-post="1" data-topic="96760">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/839c29/48.png" class="avatar"> bing:</div>
<blockquote>
<p>I have seen multiple tutorial where few people adds single fc layers after removing the initial fc layer<br>
Also, it seems like a rabbit hole where we can freeze-unfreeze the different number of sub-modules. It seems not very efficient to me.</p>
</blockquote>
</aside>
<p>I think that’s unfortunately still often the case and you will read in a lot of papers that the training hyperparameters were “tuned” or “heuristically determined”.</p>
<p>You could have a look at e.g. <a href="https://www.fast.ai/">FastAI</a> which explains the best practices for some use cases.</p>
About “Deprecation of the volatile flag”

username: ky_Pa
cooked: <p>Hi,<br>
The 0.4 version deprecated the keyword volatile. I don’t understand the official document thoroughly. Can you explain the relationship among volatile, requires_grad, and with torch.no_grad()?In addition, how to modify the following two pieces of code when volatile=true or false. Thank!</p>
<pre><code class="lang-auto">def make_variable(tensor, volatile=False):
    """Convert Tensor to Variable."""
    if torch.cuda.is_available():
        tensor = tensor.cuda()
    return Variable(tensor, volatile=volatile)
</code></pre>
<pre><code class="lang-auto">    # evaluate network
    for (images, labels) in data_loader:
        images = make_variable(images, volatile=True)
        labels = make_variable(labels).squeeze_()

        preds = classifier(encoder(images))
        loss += criterion(preds, labels).item()

        pred_cls = preds.data.max(1)[1]
        acc += pred_cls.eq(labels.data).cpu().sum()

    loss /= len(data_loader)
    acc /= len(data_loader.dataset)

    print("Avg Loss = {}, Avg Accuracy = {:2%}".format(loss, acc))</code></pre>

username: ptrblck
cooked: <p>Instead defining the <code>volatile</code> flag in the <code>Variable</code> you are now wrapping code blocks into a <code>with torch.no_grad()</code> block.</p>
<p>Your code snippet would thus change to:</p>
<pre><code class="lang-python">with torch.no_grad():
    # evaluate network
    for (images, labels) in data_loader:
        labels = labels.squeeze_()

        preds = classifier(encoder(images))
        loss += criterion(preds, labels).item()

        pred_cls = preds.max(1)[1]
        acc += pred_cls.eq(labels).cpu().sum()
</code></pre>
<p>Also, don’t use the <code>.data</code> attribute anymore, as it might have unwanted side effects.</p>

username: ky_Pa
cooked: <p>Thanks for your reply. Can volatile=False be the default value and can be removed directly? Or can the entire first paragraph of code be removed?</p>

username: ptrblck
cooked: <p>You can remove the usage of <code>Variable</code>s completely.<br>
The equivalent to <code>volatile=True</code> is the <code>with torch.no_grad()</code> block, so you should use it if you want to save memory and don’t want to calculate gradients for these operations.</p>

username: ky_Pa
cooked: <p>So how to modify it when encountering <code>volatile=False</code> or <code>requires_grad</code> ?</p>

username: ptrblck
cooked: <p>I’ve posted a code snippet in my <a href="https://discuss.pytorch.org/t/about-deprecation-of-the-volatile-flag/96292/2">previous post</a>.<br>
Isn’t this working for you?</p>

username: ky_Pa
cooked: <p>It can work. Thank you very much. What I asked above is an additional question.</p>

username: ptrblck
cooked: <p>In that case I misunderstood the question.<br>
For <code>volatile=False</code> you can just use plain tensors without using the <code>no_grad</code> guard.</p>
Train.py use early stoping

username: 111296
cooked: <p>I use this web(<a href="https://github.com/dusty-nv/pytorch-classification/blob/5107de352cade326e7aacd44bf8d625b7055fd4e/train.py" rel="nofollow noopener">https://github.com/dusty-nv/pytorch-classification/blob/5107de352cade326e7aacd44bf8d625b7055fd4e/train.py</a>)to use train.py. I want to add the early stopping method to train.py. How to do it?</p>

username: hash-ir
cooked: <p>You can either use the validation loss or accuracy to determine the early stopping. Its better to do it on the epoch validation loss. In the validate function, you can keep track of the running loss and compute the epoch loss outside of the loop. In addition, save the best validation loss (initialized with a very large value) and introduce a patience variable (initialized with 0). Both can be global variables or passed as an argument to the validate function:</p>
<pre><code class="lang-auto">def validate(val_loader, model, criterion, num_classes, args):
    running_loss = 0.0
    for i, (images, target) in enumerate(val_loader):
        running_loss += loss.item() * images.size(0)
    epoch_val_loss = running_loss / len(val_loader)
    if epoch_val_loss &lt; best_val_loss:
        best_val_loss = epoch_val_loss
        patience = 0 # reset patience if the loss improved
    else:
        patience += 1 # increase patience counter

    if patience == patience_threshold: # loss didn't decrease for 'patience_threshold' epochs
        return early_stopping # can be a flag to indicate early stopping and break out of the epoch loop
</code></pre>
<p>You can do it similarly using the accuracy metric. This is a very crude way to do early stopping. There can be more formal ways and modifications. Feel free to change it according to your needs.</p>

username: 111296
cooked: <p>I am very sorry to say. Would it be convenient for you to show or write the early_stopping code with train.py to me?</p>

username: 111296
cooked: <p>How to use dropout method on train.py</p>
How to use loop length as an optimization parameter

username: edeguine
cooked: <p>Hi, I am trying to use the length of a loop as an optimization parameter.</p>
<p>Here is an example code illustrating the problem (the real problem is more complex):</p>
<pre><code class="lang-auto">import torch

positive_iter = torch.tensor([10.0], requires_grad=True)
negative_iter = torch.tensor([20.0], requires_grad=True)

optimizer = torch.optim.Adam([positive_iter, negative_iter], lr=0.02, betas=(0.5, 0.999))

for i in range(100):
    loss = torch.tensor([0.0], requires_grad=True)
    for p in range(int(positive_iter)):
        loss = loss + torch.rand(1)
    for n in range(int(negative_iter)):
        loss = loss - torch.rand(1) * 2

    loss = torch.abs(loss)
    loss.backward()
    optimizer.step()

    print(i, loss.item(), positive_iter.item(), negative_iter.item())
</code></pre>
<p>It does not seem to work:</p>
<pre><code class="lang-auto">0 16.121417999267578 10.0 20.0
1 16.20305633544922 10.0 20.0
2 14.316006660461426 10.0 20.0
3 15.411490440368652 10.0 20.0
4 13.62910270690918 10.0 20.0
5 20.098087310791016 10.0 20.0
6 16.164840698242188 10.0 20.0
7 11.07910442352295 10.0 20.0
8 15.867802619934082 10.0 20.0
9 18.05478286743164 10.0 20.0
...continued
90 11.088855743408203 10.0 20.0
91 13.985483169555664 10.0 20.0
92 15.015034675598145 10.0 20.0
93 16.076112747192383 10.0 20.0
94 19.00041389465332 10.0 20.0
95 20.184921264648438 10.0 20.0
96 18.890005111694336 10.0 20.0
97 14.863526344299316 10.0 20.0
98 16.529191970825195 10.0 20.0
99 14.828125 10.0 20.0
</code></pre>
<p>Please advise on how to make this work</p>

username: KaiHoo
cooked: <p>I do not think the positive_iter is differential. Can you do the math to show the<br>
derivative?</p>
Calculate the mean of a torch tensor, ignoring the zero paddings

username: Ian_De_Oliveira
cooked: <p>Hi all,</p>
<p>I am trying to calculate the mean of a tensor element-wise but ignoring the zero paddings, is this possible in Pytorch?</p>
<p>For example if I have the following tensor:</p>
<p>a = torch.FloatTensor([[1, 3, 6, 0, 9 ],<br>
[1, 2, 3, 4, 0])</p>
<p>torch.mean(a,dim=0)</p>
<p><span class="hashtag">#result</span><br>
tensor([1.0000, 2.5000, 4.5000, 2.0000, 4.5000])</p>
<h1>result which I need</h1>
<p>tensor([1.0000, 2.5000, 4.5000, <strong>4.000</strong>, <strong>9.000</strong>])</p>
<p>Any help is greatly appreciated<br>
Ian</p>

username: KaiHoo
cooked: <p>a.sum(0)/(a!=0).sum(0)</p>
How are batch norm statistics affected during multiple forward passes?

username: Rakshit_Kothari
cooked: <p>Consider a simple Siamese network. I have inputs from two datasets, <code>x1</code> and <code>x2</code>. What is the correct technique in forwarding inputs?</p>
<pre><code class="lang-auto">net = Siamese()
X = torch.cat([x1, x2], dim=0)
O = net(X)
o1 = O[:x1.shape[0], ...]
o2 = O[x2.shape[0]:, ...]
</code></pre>
<p>or alternatively,</p>
<pre><code class="lang-auto">net = Siamese()
o1 = net(x1)
o2 = net(x2)
</code></pre>
<p>Note that <code>x1</code> and ‘x2’ are from different datasets.</p>
Estimated remaining time

username: iLegolas
cooked: <p>How to get estimated remaining time for training a model in pytorch? In keras it shows automatically.<br>
I have just started learning pytorch.</p>

username: hash-ir
cooked: <p>You can use <a href="https://tqdm.github.io/" rel="nofollow noopener">tqdm</a>. Install it using <code>pip install tqdm</code></p>
<pre><code class="lang-auto">from tqdm import tqdm 
from tqdm.notebook import tqdm # better version for jupyter notebook

for i in tqdm(range(n_epochs), desc="Training: "): 
   # training code
</code></pre>

username: iLegolas
cooked: <aside class="quote no-group" data-username="hash-ir" data-post="2" data-topic="96779">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hash-ir/48/28920_2.png" class="avatar"> hash-ir:</div>
<blockquote>
<p><code>desc=</code></p>
</blockquote>
</aside>
<p>Thank you very much <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=12" title=":smile:" class="emoji" alt=":smile:" loading="lazy" width="20" height="20"> <a class="mention" href="/u/hash-ir">@hash-ir</a> .</p>
<p>A good example for tqdm .<a href="https://www.kaggle.com/abhishek/very-simple-pytorch-training-0-59" rel="noopener nofollow ugc">https://www.kaggle.com/abhishek/very-simple-pytorch-training-0-59</a></p>

username: hash-ir
cooked: <p>I am glad it helped. Please accept this as a solution if it worked for you!</p>
BCEWithLogitsLoss autoencoders

username: mehran2020
cooked: <p>Hello all,</p>
<p>I have an auto-encoder trained with BCEWithLogitsLoss(). Say the output dimension is 10, and I feed a data batch of size 20 into the network during testing. Say, y_hat is my output vector (i.e., reconstruction of the input of dimension 10), and say y is my actual input data:</p>
<blockquote>
<p>criterion = nn.BCEWithLogitsLoss()<br>
e = criterion(y_hat, data)</p>
</blockquote>
<p>Unfortunately, e has is just a scalar! What I need is individual error vectors. In particular, I want e to be NOT aggregated, and have the dimensionality of (batchsize, data_dimension)= (20,10). Then I can average my ‘e’ across the second dimension and end up with an average error vector of size 20!</p>
<p>This way for 20 input data, I will have 20 individual error values and I can compute my performance (e.g., AUC). Any ideas how I can do this?</p>
<p>Thanks a lot</p>
<p>P.S.<br>
If it was just BCE() error function, I could have just written the math and computed it manually, but BCEWithLogitsLoss() is tricky to code from scratch (I think!)</p>

username: ptrblck
cooked: <p>If you don’t want to compute the loss average you can pass <code>reduction='none'</code> to <code>nn.BCEWithLogitsLoss</code>.</p>

username: mehran2020
cooked: <p>Thank you so much for the fast response. You guys are amazing!</p>
3d model not using GPU

username: Jaideep_Valani
cooked: <p>I am currently using this 3d model in public kernel. This model is not using gpu even after moving model and input to available gpu. What could be reason…</p>
<p><a href="https://www.kaggle.com/raviet/osic-final-easy-nn-using-image-and-tabular-data" class="onebox" target="_blank" rel="nofollow noopener">https://www.kaggle.com/raviet/osic-final-easy-nn-using-image-and-tabular-data</a></p>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a>  any help i can get.</p>

username: BramVanroy
cooked: <p>Are you sure you are using the GPU? What do you get when you print out the device variable? How do you know it’s not using the GPU?</p>
Load Checkpoint into DenseNet model

username: asoin29
cooked: <p>Loading a checkpoint (stored as a “pth.tar”) into DenseNet to then later convert it to Keras. The code below works but gives issues with formatting during conversion later. First, I simply loaded the state dict from the “pth.tar” without changing classifier weight and bias tensor shapes but was getting <strong>torch.size tensor mismatch</strong>. I’ve initialized those required tensor shapes using the data attribute. Am I not supposed to be messing with the “.data” attribute? Basically, what do I need to change in my code so I can load such “pth.tar” checkpoints and still have the model work for downstream tasks?</p>
<pre><code class="lang-auto">
new_state_dict = OrderedDict()

model = models.densenet121(pretrained=True)

# to load
model.classifier.weight.data = torch.randn(42, 1024)
model.classifier.bias.data = torch.randn(42)

checkpoint = torch.load('../iter_196800.pth.tar')

for k, v in checkpoint['model_state'].items():
    if k[:13] == 'module.model.':
        name = k[13:]  # remove `module.model`
    else:
        name = k
    new_state_dict[name] = v

checkpoint['model_state'] = new_state_dict

model.load_state_dict(checkpoint['model_state'])

#optimizer.load_state_dict(checkpoint['optimizer'])

model.eval()
</code></pre>
Lazy generation of large random matrix

username: Luca_Mingarelli
cooked: <p>Is it possible to lazy-generate a larger-than-memory random matrix with pytorch? Ideally I am looking to generate a 1e9 x 5000 matrix and compute the scalar product with another 5k x 5k matrix.<br>
I had a look at keops, but I didn’t manage to get too far with that.</p>
<p>Is it be possible to do this in a distributed manner, for example by interfacing somehow with pyspark?</p>

username: mrshenli
cooked: <p>Hey <a class="mention" href="/u/luca_mingarelli">@Luca_Mingarelli</a></p>
<p><code>torch.distributed</code> package can help to distribute tensors across multiple nodes, but, as of today, you still need to implement the distribute matrix multiplication on your own.  If you are looking for features like Mesh-TensorFlow. It is not yet available.</p>
<p>This is a very interesting request. Do you mind share some some details of the application?</p>

username: Luca_Mingarelli
cooked: <p>Hi <a class="mention" href="/u/mrshenli">@mrshenli</a>,</p>
<p>Say I want to perform N monte carlo simulations each consisting in a dot product of a vector of length M (M=5k) times a square matrix M x M. One way is a very lengthy and never ending loop. A matrix multiplication is however much faster. This is particularly useful in credit risk, to estimate distributions of losses. With numpy you can only use an in-between solution by making batches, but being able to lazily perform such operation gives massive advantages. Currently I solved the problem by a combination of xarray and dask, but would be great to see this in pytorch as well.</p>
Use gradient in intermediate step in encoder-decoder

username: fedetask
cooked: <p>I am trying to reproduce <a href="https://arxiv.org/pdf/1909.13789.pdf" rel="nofollow noopener">Hamiltonian Generative Networks</a>.</p>
<p>They encode the sequence of images into a latent space <code>z_0 = (p_0, q_0)</code> where <code>p</code> and <code>q</code> are both k-dimensional vectors, corresponding to momentum and position at time <code>t=0</code>.  <code>z_0</code> is then transformed into a scalar value <code>h</code> with another network. Then, they compute <code>z_1 = (p_1, q_1)</code> where <code>p_1 = p_0 - dh/dq_0</code> and <code>q_1 = q_0 + dh/dp_0</code>.  <code>z_1</code> is then fed to the decoder network and the reconstruction loss is used to train the whole system.</p>
<p>The issue is the following: how can I build a graph in pytorch that uses the gradients of a subset of the network to compute values (in this case <code>p</code> and <code>q</code>) that are used by the rest of the network? Does pytorch allow something like this?</p>

username: googlebot
cooked: <p>You can calculate (subgraph) gradients during the forward pass. When doing this inside autograd.Function, you can plug precalculated gradients into the outer graph. Not sure if this will help with your case, but here is an illustrative snippet:</p>
<pre><code class="lang-auto">class LpdfGenAsymLaplace(torch.autograd.Function):
	@staticmethod
	def forward(ctx, x_in, p0, g_in):
		if any(ctx.needs_input_grad):
			with torch.enable_grad():
				x = x_in.detach().requires_grad_()
				g = g_in.detach().requires_grad_()
				lpdf = _pdf(x,p0,g)
				dLpdf_dX, dLpdf_dG = torch.autograd.grad(lpdf, (x,g), torch.ones_like(lpdf), retain_graph=True, create_graph=True)
				ctx.save_for_backward(dLpdf_dX, dLpdf_dG)
				ctx.mark_non_differentiable(p0)
				return lpdf
		else:
			return _pdf(x_in,p0,g_in)
		
	@staticmethod
	def backward(ctx, gr_in):
		dLpdf_dX, dLpdf_dG = ctx.saved_tensors
		return dLpdf_dX * gr_in, None, dLpdf_dG * gr_in
</code></pre>
Inference with HalfTensor for speed up

username: wlike
cooked: <p>Can we first train a model using default torch.Tensor, which is torch.FloatTensor,<br>
and convert it to torch.HalfTensor for inference?</p>
<p>Or can we directly use torch.HalfTensor for training and inference?</p>

username: alexis-jacq
cooked: <p>You can change the nature of your tensor when you want, using <code>my_tensor.half()</code> or <code>my_tensor.float()</code>, my instincts would tell me to use the whole network with floats and to just change the output into half at the very last time in order to compute the loss.</p>

username: wlike
cooked: <p>we can use model.half() to convert model’s parameters and internal buffers<br>
to half precision. But we do NOT get significant improvement as expected<br>
to be 2~4x faster by using HalfTensor.</p>

username: Elias_Vansteenkiste
cooked: <aside class="quote no-group" data-username="wlike" data-post="3" data-topic="5739">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/w/e0b2c6/48.png" class="avatar"> wlike:</div>
<blockquote>
<p>But we do NOT get significant improvement as expected</p>
<p>to be 2~4x faster by using HalfTensor.</p>
</blockquote>
</aside>
<p>Hi wlike,</p>
<p>Did you figure out why you didn’t get a speedup? Which Nvidia GPU cards did you use? Did they support float16 like the V100 or P100 or was it the 1080(ti) or Titan which does not support fast float16?<br>
How was your GPU memory consumption when you changed to HalfTensor?</p>
<p>Thanks in advance</p>

username: Tgaaly
cooked: <p>thanks for sharing this! this is valuable info! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=6" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: Edwardmark
cooked: <p>We did not see any speed up, how about you?</p>

username: Edwardmark
cooked: <p>I am using 2080ti, but cannot see any improvements when changing from fp32 to fp16 when do inference with batch_size 1. Why is that? And how can I speed up the inference speed? The gpu usage is reduced from 1905MB to 1491MB anyway. Thanks.</p>

username: ptrblck
cooked: <p>Answered <a href="https://discuss.pytorch.org/t/cnn-fp16-slower-than-fp32-on-tesla-p100/12146/4">here</a>.</p>

username: kwanUm
cooked: <p>Did not observe speedup. Actually inference has <em>slowed down</em> for me. Using V100 GPU and running a WaveGAN architecture.</p>
How to check my model working

username: Ganga
cooked: <p>I have fused the learnable parameters of 2 layers (bnorm-&gt;linear) , as i wanted to check whether my fused model is working proper or not I have tested 5000 images(Imagenet datasets) falling under 1000 classes, its accuracy is same as before(before fusing)</p>
<p>Is there any other method to check my fused model is working or not?</p>
Deep q network not learning and step not stepping towards target

username: Ryan_Mccauley
cooked: <p>Hi, I am trying to create a simple deep q network for rl with conv2d layers. I can’t figure out what I am doing wrong, and the only thing I can see that doesn’t seem right is when I get the model predicition for a state after the optimizer step it doesn’t seem to get closer to the target.</p>
<p>I am using pixels from pong in openai gym with single channel 90x90 images, a batch size of 32 and replay memory.</p>
<p>As an example if I try with a batch size of one, and try running self(states) again right after the optimizer step the output is as follows:</p>
<pre><code class="lang-auto">current_q_values -&gt; -0.16351485  0.29163417  0.11192469 -0.08969332  0.11081569  0.37215832
q_target -&gt;         -0.16351485  0.5336551   0.11192469 -0.08969332  0.11081569  0.37215832
self(states) -&gt;     -0.8427617   0.6415581   0.44988257 -0.43897176  0.8693738   0.40007943
</code></pre>
<p>Does this look as what would be expected for a single step?</p>
<p>The network with loss and optimizer:</p>
<pre><code>    self.in_layer = Conv2d(channels, 32, 8)
    self.hidden_conv_1 = Conv2d(32, 64, 4)
    self.hidden_conv_2 = Conv2d(64, 128, 3)
    self.hidden_fc1 = Linear(128 * 78 * 78, 64)
    self.hidden_fc2 = Linear(64, 32)
    self.output = Linear(32, action_space)

    self.loss = torch.nn.MSELoss()
    self.optimizer = torch.optim.Adam(
        self.parameters(), lr=learning_rate) # lr is 0.001

def forward(self, state):
    in_out = fn.relu(self.in_layer(state))
    in_out = fn.relu(self.hidden_conv_1(in_out))
    in_out = fn.relu(self.hidden_conv_2(in_out))
    in_out = in_out.view(-1, 128 * 78 * 78)
    in_out = fn.relu(self.hidden_fc1(in_out))
    in_out = fn.relu(self.hidden_fc2(in_out))
    return self.output(in_out)
</code></pre>
<p>Then the learning block:</p>
<pre><code>        self.optimizer.zero_grad()

        sample = self.sample(self.batch_size)
        states = torch.stack([i[0] for i in sample])
        actions = torch.tensor([i[1] for i in sample], device=device)
        rewards = torch.tensor([i[2] for i in sample], dtype=torch.float32, device=device)
        next_states = torch.stack([i[3] for i in sample])
        dones = torch.tensor([i[4] for i in sample], dtype=torch.uint8, device=device)

        current_q_vals = self(states)
        next_q_vals = self(next_states)
        q_target = current_q_vals.clone()
        q_target[torch.arange(states.size()[0]), actions] = rewards + (self.gamma * next_q_vals.max(dim=1)[0]) * (~dones).float()

        loss = fn.smooth_l1_loss(current_q_vals, q_target)
        loss.backward()

        self.optimizer.step()
</code></pre>
<p>Thanks.</p>