akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Aim of grad_tensor of backward(), and how to compute Jocobian-vector product

Godspeed: <p>b25 = torch.tensor([[1.,2.,3.,4.,5.],[6.,7.,8.,9.,10.]], requires_grad= True)<br>
def func2(x):<br>
y = x**2<br>
return y</p>
<p>y = func2(b25)</p>
<p>vector = torch.tensor([[100., 1., 1., 1., 1.],[1., 1., 1., 1., 1.]])<br>
Jocobian = torch.autograd.functional.jacobian(func2, b25, create_graph=False, strict=False)</p>
<p>y.backward(vector)</p>
<p>The Jocobian is a tensor of (2,5,2,5), because y is (2,5), b25 is also (2,5).<br>
Jocobian is the derivatives of dy/db25<br>
According to the Jocobian-vector product, we can multiply Jocobian with vector to get the result that is same as y.backward(vector). But how could tensor of (2,5,2,5) and vector (2,5) to derive a (2,5) final gradient?</p>
<pre><code class="lang-auto">Jocobian = tensor([[
         [[ 2.,  0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.,  0.]],

         [[ 0.,  4.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.,  0.]],

         [[ 0.,  0.,  6.,  0.,  0.],
          [ 0.,  0.,  0.,  0.,  0.]],

         [[ 0.,  0.,  0.,  8.,  0.],
          [ 0.,  0.,  0.,  0.,  0.]],

         [[ 0.,  0.,  0.,  0., 10.],
          [ 0.,  0.,  0.,  0.,  0.]]],


        [[[ 0.,  0.,  0.,  0.,  0.],
          [12.,  0.,  0.,  0.,  0.]],

         [[ 0.,  0.,  0.,  0.,  0.],
          [ 0., 14.,  0.,  0.,  0.]],

         [[ 0.,  0.,  0.,  0.,  0.],
          [ 0.,  0., 16.,  0.,  0.]],

         [[ 0.,  0.,  0.,  0.,  0.],
          [ 0.,  0.,  0., 18.,  0.]],

         [[ 0.,  0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0., 20.]]]])
</code></pre>
<p>This is final gradients compute by pytorch, how can I get the same result by using Jocobian and vector above?</p>
<p>b25.grad<br>
tensor([[200.,   4.,   6.,   8.,  10.],<br>
[ 12.,  14.,  16.,  18.,  20.]])</p>
Labels starting from 0 or 1

over9k: <p>Hello!</p>
<p>Is there any requirement for labels for start from 0 all the way to 1, 2, 3, number of classes? Or can I have labels start from 1, for example?</p>
<p>This is just a matter of having to apply a label encoder manually to the dataset or not.</p>
<p>Thanks</p>

KFrank: <p>Hello Over!</p>
<aside class="quote no-group" data-username="over9k" data-post="1" data-topic="106909" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/o/a87d85/48.png" class="avatar"> over9k:</div>
<blockquote>
<p>Is there any requirement for labels for start from 0 all the way to 1, 2, 3, number of classes?</p>
</blockquote>
</aside>
<p>Well, it depends on what you do with the labels.</p>
<p>In the common case that you use <a href="https://pytorch.org/docs/1.7.0/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss" rel="noopener nofollow ugc">CrossEntropyLoss</a> as your loss<br>
function, per its documentation, your labels (the <code>target</code>) must be<br>
integer class labels that run from <code>[0, nClass - 1]</code>, inclusive.  That<br>
is, the labels start from 0.</p>
<p>Best.</p>
<p>K. Frank</p>
Implementation of Cosine Annealing with Warm up

hsiangyu: <p>Hi there, I am wondering that if PyTorch supports the implementation of <strong>Cosine annealing LR with warm up</strong>, which means that <strong>the learning rate will increase in the first few epochs and then decrease as cosine annealing</strong>. Below is a demo image of how the learning rate changes.<br>
I only found <em>Cosine Annealing</em> and <em>Cosine Annealing with Warm Restarts</em> in PyTorch, but both are not able to serve my purpose as I want a relatively small lr in the start.<br>
I would be grateful if anyone gave me advices or instructions.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/4/14391e53728abd2894f69a47b23916bc48d32f52.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/14391e53728abd2894f69a47b23916bc48d32f52" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/14391e53728abd2894f69a47b23916bc48d32f52_2_690x345.jpeg" alt="image" data-base62-sha1="2STVmgoYLisrkAkbrNSwkZGbjma" width="690" height="345" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/14391e53728abd2894f69a47b23916bc48d32f52_2_690x345.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/1/4/14391e53728abd2894f69a47b23916bc48d32f52.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/4/14391e53728abd2894f69a47b23916bc48d32f52.jpeg 2x" data-dominant-color="F5F8F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">864×432 78 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

googlebot: <p>Ignite has these:<br>
<a href="https://pytorch.org/ignite/contrib/handlers.html#ignite.contrib.handlers.param_scheduler.create_lr_scheduler_with_warmup" rel="noopener nofollow ugc">create_lr_scheduler_with_warmup</a><br>
<a href="https://pytorch.org/ignite/contrib/handlers.html#ignite.contrib.handlers.param_scheduler.ConcatScheduler" rel="noopener nofollow ugc">ConcatScheduler</a></p>
Combine n-gram word embeddings

BlaBla24: <p>Hi, I am willing to concatenate char, 2-gram and 3-gram word embeddings.</p>
<pre><code class="lang-auto">x_char = torch.randint(low=0, high=26, size=(3, 16), dtype=torch.long)
x_2_gram = torch.randint(low=0, high=60, size=(3, 8), dtype=torch.long)
x_3_gram = torch.randint(low=0, high=100, size=(3, 4), dtype=torch.long)

class NN(nn.Module):
    def __init__(self, one_gram_vocab_size, two_gram_vocab_size, three_gram_vocab_size, embedding_dim):
        super().__init__()
        self.one_gram_embedding = nn.Embedding(one_gram_vocab_size, embedding_dim)
        self.two_gram_embedding = nn.Embedding(two_gram_vocab_size, embedding_dim)
        self.three_gram_embedding = nn.Embedding(three_gram_vocab_size, embedding_dim)
    
    def forward(self, x_one_gram, x_two_gram, x_three_gram):
        one_gram_embedding = self.one_gram_embedding(x_one_gram)
        two_gram_embedding = self.two_gram_embedding(x_two_gram)
        three_gram_embedding = self.three_gram_embedding(x_three_gram)
        
        print(one_gram_embedding.shape, two_gram_embedding.shape, three_gram_embedding.shape)
        concat = torch.cat((one_gram_embedding, two_gram_embedding, three_gram_embedding), 0)
        return concat


model = NN(26, 60, 100, 5)
model(x_char, x_2_gram, x_3_gram)
</code></pre>
<p>Given different input shapes I don’t get how I can concatenate embedding outputs. Thanks!</p>

vdw: <p>Well, all your three embedding layers have the same <code>embedding_dim</code>, say, 100. So each one-gram, bigram and trigram gets mapped to a 100-dim vector. So you can concatenate these three vectors a long both dimensions, either to get a (1, 300) vector or a (3, 100) vector.</p>

BlaBla24: <p>Thanks! How can I create a (1, 300) vectors from this setting? Given that created tokenised n-grams are of different shape I am getting an error</p>
<blockquote>
<p>Sizes of tensors must match except in dimension 2. Got 16 and 8 in dimension 1 (The offending index is 1)</p>
</blockquote>
Deploy on Libtorch to device use .ipa

ducnm: <p>Hi, everyone.<br>
From this tutorial <a href="https://www.youtube.com/watch?v=JFy3uHyqXn0" class="inline-onebox" rel="noopener nofollow ugc">PyTorch Mobile Runtime for iOS - YouTube</a>, I can run an app on my device, everything is easy and it has a good result if deploy directly from Xcode.<br>
But when deploy my app by .ipa file through <a href="http://installonair.com/" rel="noopener nofollow ugc">http://installonair.com/</a> my app have many problems:</p>
<ul>
<li>Crash and stop an app.</li>
<li>Return bad result (I used same pic for test)</li>
</ul>
<p>Please help me, thanks.</p>
Back Propagation for Multi-Label Learning in Pytorch

Thiru: <p>How to implement BPMLL loss in pytorch?</p>
<p>(OR)</p>
<p>Is there any official repo that has BPMLL loss implemented for pytorch?</p>

Alexey_Demyanchuk: <p>Hi. If I understand correctly, and you are talking about <a href="https://en.wikipedia.org/wiki/Multi-label_classification" rel="noopener nofollow ugc">Multi Label Classification</a>, then <a href="https://pytorch.org/docs/stable/generated/torch.nn.BCELoss.html#bceloss" rel="noopener nofollow ugc">Binary Cross Entropy</a> and its more stable <a href="https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html#bcewithlogitsloss" rel="noopener nofollow ugc">version</a> should be  a good start for your search <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Thiru: <p>Yeah I tried with Binary Cross Entropy but my Macro AUC is not boosting up even after rigorous hyperparam tuning using raytune. The dataset is highly imbalanced. I referred the internet where they stated that BP_MLL loss is the perfect thing for multi-label classification.</p>
How to append an int tensor to a list tensor?

Z_Mario: <p>In python, I can use:</p>
<pre><code class="lang-auto">A = [1,2]
B = 3
A.append(B)
</code></pre>
<p>to get <code>[1,2,3]</code>. However, In pytorch,</p>
<pre><code class="lang-auto">A = torch.tensor([1,2])
B = torch.tensor(3)
</code></pre>
<p>How to get:</p>
<pre><code class="lang-auto">tensor([1,2,3])
</code></pre>
<p>Thank you very much</p>

Alexey_Demyanchuk: <p>Hello. I believe it could be done in many ways. A reasonable choice for me would be <code>.cat</code> method. Just make sure that the tensor <code>B</code> is also 1-dimensional as <code>A</code>.<br>
This should work:</p>
<pre><code class="lang-auto">A = torch.tensor([1,2])
B = torch.tensor([3])
torch.cat((A,B))
</code></pre>

AlperEkmekci: <p>Your B tensor is zero dimesional, so you can’t use <code>torch.cat()</code> function to concatenate them.</p>
<p>1)Concatenate them as python array and convert them to tensor<br>
2)Add dimension with <code>unsqueeze()</code> method and use <code>torch.cat()</code></p>

Z_Mario: <p>I use 2) and solve my problem. Thanks</p>
Increasing GPUmemory

nafise: <p>Hi</p>
<p>can anyone help me ? I want to increasing my gpu memory.<br>
I got the error below when I ran my code:</p>
<p>Your session crashed after using all available RAM</p>

AlperEkmekci: <p>Unfortunately you can’t increase you GPU vRAM except buying a new one. GPU vRAM and computer RAM are different things. Error you described tells about your overusage PC RAM not vRAM.</p>
<p>What do you want to do exactly, please share more detail including terminal log maybe.</p>
<p>Greetings.</p>
Looking for a function that resets model to an older model before validation error increased

Zador: <p>Hi,</p>
<p>so in the passed I have been implementing the following strategy inspired by the function torch.optim.lr_scheduler.ReduceLROnPlateau:</p>
<ul>
<li>after a patience number of epochs calculate the validation error of the current model</li>
<li>if validation error of current model is lower than the previously calculated validation error then save the current model</li>
<li>if the above condition does not hold then reset the model to the previously saved model and step down the learning rate</li>
</ul>
<p>The above strategy is similar to ReduceLROnPlateau in that it steps down the learning rate when the validation error did not decrease after a patience number of epochs, however, using my method I am not stuck with the model that has already overfitted.</p>
<p>The biggest issue with my approach is that the way I have written the code is it requires a lot of coding to apply it to different applications (I am yet to write it in a manner in which it is generalisable). I would very much rather not do this each time.</p>
<p>Is there any chance that there is some function in pytorch that does something similar to this? If that were the case it would be awesome and I would appreciate it if someone could inform me about it. If that is not the case is there a reason why it is not? It shouldn’t be to hard to implement. Is there some inherent flaw with the strategy that I might be overlooking?</p>
[libtorch] How to convert std::vector<torch::Tensor> to torch::TensorList?

wizardk: <p>How to convert std::vector&lt;torch::Tensor&gt; to torch::TensorList?</p>

LeviViana: <p>It works for me:</p>
<pre><code class="lang-cpp">#include &lt;torch/torch.h&gt;
#include &lt;iostream&gt;


int main(){
    torch::Tensor x = torch::rand({2, 3, 2});
    std::cout &lt;&lt; "Torch Tensor" &lt;&lt; std::endl;
    std::cout &lt;&lt; x &lt;&lt; std::endl;
    std::vector&lt;torch::Tensor&gt; v;
    v.push_back(x);
    torch::TensorList z = torch::TensorList(v);
    std::cout &lt;&lt; "Torch TensorList 1st element" &lt;&lt; std::endl;
    std::cout &lt;&lt; z[0] &lt;&lt; std::endl;
    return 0;
}
</code></pre>
<p>And it should work always, since <a href="https://github.com/pytorch/pytorch/blob/f5178bf151cdca4268a7268ae6aa961f683bbb8b/aten/src/ATen/templates/TensorBody.h#L53" rel="noopener nofollow ugc">TensorList is an ArrayRef of tensors</a> and <a href="https://github.com/pytorch/pytorch/blob/master/c10/util/ArrayRef.h#L83-L90" rel="noopener nofollow ugc">ArrayRef implements a constructor from std::vector</a>.</p>

wizardk: <p>Thanks! After posting this post, I realized that I could cast std::vector&lt;torch::Tensor&gt; to torch::TensorList.</p>
How to get rid of “cuda unspecified launch failure” error that happens after a few epochs?

Berna_Yilmaz: <p>I am getting a CUDA error: unspecified launch failure error when training.It does not work 5 times and it gives this error. It works without error on the 6th stage. But this changes a lot. Sometimes it gives an error 8 times,and 9th works.</p>
<pre><code class="lang-auto">for i, (inputs, target, _) in enumerate(train_loader):
		print(torch.cuda.is_available())
		print(len(inputs))
		input_var = [input.cuda() for input in inputs]
</code></pre>
<p>Output :</p>
<pre><code class="lang-auto">True
136
Traceback (most recent call last):

  File "....\train.py", line 273, in &lt;module&gt;
    train(train_loader, model, criterion, optimizer, epoch)

  File ".....\train.py", line 75, in train
    input_var = [input.cuda() for input in inputs]

  File "......\train.py", line 75, in &lt;listcomp&gt;
    input_var = [input.cuda() for input in inputs]

RuntimeError: CUDA error: unspecified launch failure
</code></pre>
<p>Do you have any idea how I can fix the error? Thanks.</p>
<ul>
<li>Windows 10</li>
<li>NVIDIA GeForce GTX 1060</li>
<li>Torch 1.6</li>
<li>Cuda 10.1</li>
</ul>

ptrblck: <p>Could you update to the latest stable release and check, if this error is still raised after some epochs?<br>
Also, could you check <code>dmesg</code> for any xid errors?</p>
Replace parameters of model

FCF: <p>I have a Deep Convolutional LSTM NN, for which I am trying to do a very basic post trained quantization. Just convert the trained weights from float32 to int8.</p>
<p>I’m new to this so I might have does this wrong but what I’ve done is taken a list of model.parameter after training, of which there are 18 tensors. Converted the value range from the float range to int8 range and then converted the data type to int8. The list output (qparameter) looks right, now I’m just unsure how to put these values back into a model to test.</p>
<pre><code class="lang-auto">qparameters = []

def convert_float_to_int8(i):

    while i &lt; 18:
        x = list(model.parameters())[i]

        OldMin = torch.min(x)
        OldMax = torch.max(x)

        NewMin = float(-128)
        NewMax = float(127)

        OldRange = (OldMax-OldMin)
        NewRange = (NewMax-NewMin)

        y = (((x - OldMin)*NewRange)/OldRange)+NewMin
        y = torch.round(y)
        y = y.type(torch.int8)
        print(x.dtype)
        print(y,y.dtype)

        qparameters.append(y)
        i += 1


convert_float_to_int8(0)
print(qparameters)

</code></pre>

Vasiliy_Kuznetsov: <p>Hi <a class="mention" href="/u/fcf">@FCF</a>, you can check out <a href="https://pytorch.org/tutorials/recipes/recipes/dynamic_quantization.html" class="inline-onebox" rel="noopener nofollow ugc">Dynamic Quantization — PyTorch Tutorials 1.7.1 documentation</a> for an e2e example of this.</p>
Are c10::Dicts reference types

Prashant: <p>I know that torch::jit::Modules are reference types: passing them around to functions / assigning them to variables doesn’t do an expensive copy of the entire model.</p>
<p>Is that also true for torch::Tensor? For c10::Dict and all the other c10 types?</p>

glaringlee: <p><a class="mention" href="/u/prashant">@Prashant</a><br>
For torch::Tensor and c10::Dict, yes.<br>
For all other c10 types, I don’t have a list on which c10 type copy values and which copy refs/ptrs, I think we can only talk about it type by type.</p>

Prashant: <p>Hmm thanks for the info. Let’s say I’m interested in c10::Tuple and I want to identify if its a ref type or not. How do I tell?</p>

glaringlee: <p><a class="mention" href="/u/prashant">@Prashant</a><br>
I am not aware of c10::Tuple in our code base (or do you mean c10::Ivalue::Tuple?) . We do have TupleType and TupleTypePtr.<br>
I assume you use it as an example. Normally, the implementation of these c10 types are located in<br>
c10/utils or aten/src/ATen/core, you can search the implementation there.</p>
<p>If you are talking about Ivalue::Tuple (or any types under Ivalue), take a look at here:<br>
‘<a href="https://github.com/pytorch/pytorch/blob/5741de883af2ac63cec537fc8a554610a5cea853/aten/src/ATen/core/ivalue.h#L143" class="inline-onebox" rel="noopener nofollow ugc">pytorch/ivalue.h at 5741de883af2ac63cec537fc8a554610a5cea853 · pytorch/pytorch · GitHub</a>’<br>
If you don’t call deepCopy, I believe there will be no value copy (buy swap) happen.</p>
Torch.manual_seed(0) doesn't work with randint?

Richard_S: <p>I am using torch.manual_seed(0)  before randint to always generate the same numbers, but it doesnt work…</p>
<pre><code>torch.manual_seed(0)
test_numbers = np.random.randint(1, 500,  100 , dtype=np.uint8)
</code></pre>
<p>everytime the array is different… why?</p>

ptrblck: <p><code>torch.manual_seed</code> will seed the PyTorch random number generator, not the ones from 3rd party libraries such as numpy, so you would either need to use <code>torch.randint</code> or seed numpy via <code>np.random.seed</code>.</p>
Is there any upper-level API to disable the detection of the weight's version

ymsdu2004: <p>We are trying to use pipelined model parallelism to speed up model training. This work requires ourselves to manage different versions of weights. This is ok in version 1.4 pytorch, but the following error will be reported after 1.5，<br>
In our scenario, we confirm that the version of the weight is reasonable， so  Is there any upper-level API to disable the detection of the weight’s version</p>
<pre><code class="lang-auto">raceback (most recent call last):
  File "bert_main.py", line 294, in &lt;module&gt;
    main()
  File "bert_main.py", line 210, in main
    model_args.model_name_or_path) else None
  File "/workspace/pipedream-1.5BW/lib/runtimecontrol.py", line 521, in train
    self.runtime_control.run_forward_backward()
  File "/workspace/pipedream-1.5BW/lib/runtimecontrol.py", line 194, in run_forward_backward
    data_provider=data_provider)
  File "/workspace/pipedream-1.5BW/lib/callableunit.py", line 157, in backward
    result = self._backward(**kwargs)
  File "/workspace/pipedream-1.5BW/lib/callableunit.py", line 317, in _backward
    result = self._backward_g0(**kwargs)
  File "/workspace/pipedream-1.5BW/lib/callableunit.py", line 392, in _backward_g0
    return self._comm_backward(outputs, kwargs)
  File "/workspace/pipedream-1.5BW/lib/callableunit.py", line 174, in _comm_backward
    torch.autograd.backward(forward_output, backward_gradient)
  File "/opt/conda/lib/python3.7/site-packages/torch/autograd/__init__.py", line 146, in backward
    allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [768]] is at version 9; expected version 4 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
</code></pre>

ptrblck: <p>I don’t think there is a clean way to disable this check, so could you explain your use case a bit? I.e. which operation is manipulating the parameters and causing this check to fail, while you think it shouldn’t?</p>
How to set up torch::nn::SequentialImpl with 'torch::narrow'?

MatthewV: <p>Hello, How do I use <code>torch::narrow</code> in a <code>torch::nn::SequentialImpl</code>? I am new to the c++ api. My first attempt was to create a <code>Functional</code> module, which I understand wraps a free function, but this doesn’t compile:</p>
<pre><code class="lang-auto">#include &lt;torch/torch.h&gt;

class NarrowingImpl : public torch::nn::SequentialImpl {
public:
    NarrowingImpl() {
        using namespace torch::nn;

        // This was my first attempt, but it doesn't work
        push_back(Functional(torch::narrow, 0, 1, 1));

        // This works, but with slightly different semantics (end vs length)
        // push_back(Functional(functional::_narrow_with_range, 0, 1, 2));
    }
};
TORCH_MODULE(Narrowing);

int main() {
  Narrowing nrw;
  torch::Tensor x = torch::rand({2, 3, 4});

  std::cout &lt;&lt; x &lt;&lt; std::endl &lt;&lt; std::endl;

  std::cout &lt;&lt; nrw-&gt;forward(x) &lt;&lt; std::endl;
}
</code></pre>
<p>As a side note, I was able to get it to work with <code>functional::_narrow_with_range</code>, but I’m not sure if that is the correct way to go about this.</p>
<p>Any help would be much appreciated. Thank you!</p>

glaringlee: <p><a class="mention" href="/u/matthewv">@MatthewV</a><br>
You can use functional::_narrow_with_range. The only potential issue is that this is an internal function, might change without public notification.</p>
<p>The reason you can not use torch::narrow directly is because there are 2 versions of torch::narrow,<br>
Tensor narrow(const Tensor &amp; self, int64_t dim, int64_t start, int64_t length);<br>
Tensor narrow(const Tensor &amp; self, int64_t dim, const Tensor &amp; start, int64_t length);<br>
We can not deduce the function type with parameter bindings at the moment, if you want to use torch::narrow in this case, you can wrap it with another name, for eg.<br>
torch::Tensor aaa(const torch::Tensor&amp; t, int a, int b, int c){ return torch::narrow(t, a, b, c);}<br>
Then you can do Functional(aaa, 0, 1, 1);</p>

MatthewV: <p>Thank you for the explanation and alternative method for using torch::narrow. Thanks!!</p>
Inference slow on Windows 10 (CPU)

fabio_carbone: <p>Hi. I have dual boot computer (Ubuntu 18 and Windows 10). Pytorch 1.1.0<br>
I trained a <a href="https://github.com/NVIDIA/pix2pixHD" rel="nofollow noopener">pix2pixHD</a> network, and when I try to generate an image on CPU, I get very different performance. On Windows the processing is very slow.</p>
<p>Ubuntu 18:  <strong>2 seconds</strong><br>
Windows 10:  <strong>9 seconds</strong></p>
<p>This is the code I am using to measure time, at this <a href="https://github.com/NVIDIA/pix2pixHD/blob/cf94f863ba6c833a2e5fe439d1f4777d6d4c240a/models/pix2pixHD_model.py#L165" rel="nofollow noopener">line</a>:</p>
<pre><code class="lang-auto">xtime = datetime.datetime.now()
fake_image = self.netG.forward(input_concat)
ytime = datetime.datetime.now()
print(ytime - xtime)
</code></pre>
<p>What could cause the problem?<br>
What can I try?</p>
<p>Thanks!</p>

lxtong01: <p>hi, did you solve the problem?</p>

aswath8: <p>Is the problem solved?</p>
<p>I’m having a similar issue:<br>
I tried <a href="https://github.com/nianticlabs/monodepth2" rel="noopener nofollow ugc">monodepth2</a> on windows and linux for inference on a single image.<br>
The inference took <strong>2 seconds on windows(Ryzen 9 4900 HS)</strong> compared to just <strong>0.45 seconds on Ubuntu(i3 processor).</strong><br>
Does anyone know the cause of this discrepency?</p>
How to speed up Libtorch C++ inference on CPU?

maudzung: <p>Hi everyone,</p>
<p>I convert the trained Pytorch model to torchscript, then I use Libtorch C++ version 1.5.1 cpu to deploy my implementation on CPU. However, the inference time of the torchscript model is unstable (fluctuate from 5ms to 30ms with a batch of 30 images with a size of 48x48x3. If I export the model to onnx and deploy it using onnxruntime, the runtime is more stable and faster a bit. But I’d like to use Libtorch in my implementation.</p>
<p>Can you please give some suggestions to improve the runtime for torchscript models on CPU? Do I need to set any environment variables in the <em>.bashrc</em> file?<br>
Thank you in advance!</p>

Prashant: <p>Try setting OMP_NUM_THREADS=1</p>
Is it ok to use huge batch size?

Maunish_Dave: <p>I am making a market prediction model that is a classification model.<br>
I have 2M rows of data. with 130 features.<br>
I have made a simple NN model.<br>
Currently, I am using a batch size of 50,000 and it occupies 1GB of 16 Gb available in google collab.<br>
So is it ok to use such a huge batch size and should I increase batch size or such huge batch sizes should be avoided?</p>

LeviViana: <p>It should be ok. Moreover, you should be able to increase your learning rate. The only thing that comes to mind that could be useful is to perform some warm up rounds before increasing batch size. <a href="https://research.fb.com/wp-content/uploads/2017/06/imagenet1kin1h5.pdf" rel="noopener nofollow ugc">This article</a> can give you some interesting insights.</p>
Feeding 1d PackedSequence data to an LSTM

torch_bearer: <p>I am trying to train an LSTM on audio signal data. I gave used <code>pad_sequnce()</code> and <code>pack_padded_sequence()</code> to get resultant <code>PackedSequence</code> data.</p>
<p>However this data is 1 dimensional. I have checked it using <code>x.data.shape</code> in <code>forward()</code> function and it is a 1d tensor. (<code>x</code> here is a <code>PackedSequence</code> data)</p>
<p>And I’ve used <code>batch_first=True</code> parameter also when defining lstm.</p>
<p>I’m  getting this error:</p>
<pre><code class="lang-auto">~/.conda/envs/ml/lib/python3.8/site-packages/torch/nn/modules/rnn.py in check_input(self, input, batch_sizes)
    172         expected_input_dim = 2 if batch_sizes is not None else 3
    173         if input.dim() != expected_input_dim:
--&gt; 174             raise RuntimeError(
    175                 'input must have {} dimensions, got {}'.format(
    176                     expected_input_dim, input.dim()))

RuntimeError: input must have 2 dimensions, got 1
</code></pre>
<p>Would adding a dummy dimension to this data work?</p>
<p>How can I do this to <code>PackedSequence</code> data?</p>

torch_bearer: <p>I solved this by adding a newaxis to sequence data before padding and packing in <code>collate_fn()</code> of dataloader.</p>
An error while loading Pytorch pretrained densenet121 on Kaggle notebook

Nadir_Tariverdiyev: <p>Hi all</p>
<p>Im getting an error while loading densenet121 in my notebook.<br>
Here is an error<br>
<code>URLError: &lt;urlopen error [Errno -3] Temporary failure in name resolution&gt;</code><br>
Pytorch version is 1.6.0<br>
How can I solve it?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/6/76c171ea9f823611bb79194d0778c96931d85ce7.png" data-download-href="https://discuss.pytorch.org/uploads/default/76c171ea9f823611bb79194d0778c96931d85ce7" title="image_2020_11_17T22_30_40_114Z"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/6/76c171ea9f823611bb79194d0778c96931d85ce7_2_690x235.png" alt="image_2020_11_17T22_30_40_114Z" data-base62-sha1="gWyNjQhzUA9CQm5ZEetsFIPoFcH" width="690" height="235" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/6/76c171ea9f823611bb79194d0778c96931d85ce7_2_690x235.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/6/76c171ea9f823611bb79194d0778c96931d85ce7_2_1035x352.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/6/76c171ea9f823611bb79194d0778c96931d85ce7.png 2x" data-dominant-color="242830"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image_2020_11_17T22_30_40_114Z</span><span class="informations">1329×453 54 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>The error seems to be raised by the server. Are you still seeing this issue or are you able to load the pretrained parameters?</p>

Nadir_Tariverdiyev: <p>I don’t see it anymore.<br>
Thanks</p>
Function for picking an optimizer

talachem: <p>Hello,<br>
I am trying to write a function which returns an optimizer from a dictionary, but I get the following error message:<br>
<code>ValueError: optimizer got an empty parameter list</code></p>
<p>The Function is as follows:</p>
<pre><code class="lang-auto">def pickOptimizer(opt, *args, **kwargs):
    optimizers = {
        'sgd': optim.SGD(*args, **kwargs),
        'asgd': optim.ASGD(*args, **kwargs),
        'adam': optim.Adam(*args, **kwargs),
        'adamw': optim.AdamW(*args, **kwargs),
        'adamax': optim.Adamax(*args, **kwargs),
        'rms': optim.RMSprop(*args, **kwargs),
        'rprop': optim.Rprop(*args, **kwargs),
        'adah': AdaHessian(*args, **kwargs)
    }
    return optimizers[opt]
</code></pre>
<p>If I write <code>params=net.parameters()</code> explicitly into the function it works as I expect otherwise I get an error. For this to work I need to put this function into my main document.<br>
Thank you for any help.</p>

googlebot: <p>you’re exhausting the iterator, make a list of parameters or don’t construct all the optimizers</p>

talachem: <p>I see, but is there a way to make the choice of optimizer easier than just re-writing the code? Need to be able to run is with the help of bash scripts and I want to adjust the network this way.</p>

googlebot: <p>I use this</p>
<pre><code class="lang-auto">def _get_core_optimizer(name):
	import torch.optim as optim
	return getattr(optim, name)
</code></pre>

talachem: <p>Thank you very much, now it works as I wanted.</p>
Method for dataset loads huge data

Hu_Penglong: <p>Hi, everyone.<br>
i have a large dataset.every line is the word ids.<br>
I can’t read it into the memory, i can’t figure out how to solve the problem.<br>
i also read the pd.read_csv method, but i don’t know how to do it in the <strong>get_item</strong> method of dataset class.</p>
Image reconstruction with 4 channel image

burak_dl: <p>Hi there,</p>
<p>I am dealing with an image reconstruction task with 4 channel image. After applying model surgery, as in <a href="https://discuss.pytorch.org/t/how-to-transfer-the-pretrained-weights-for-a-standard-resnet50-to-a-4-channel/52252/2">here</a>, by changing the first convolutional layer’s input channel to 4, I ended up having (batch_size, 3, height, width)  output tensors.</p>
<p>However, input image still contains 4 channels and it raises a mismatch error when using L1 loss, as expected. I discarded the last channel of input image but it doesnt seem like a convenient approach to me. Any suggestions on this?</p>
<p>Thanks</p>

Alexey_Demyanchuk: <p>Hey. Why not change the model, so reconstructed output tensor will be of shape <code>(batch_size, 4, height, width)</code>?</p>

burak_dl: <p>Hi!<br>
Let me give it a try. Thanks for the suggestion.</p>
The difference between tf.image.crop_and_resize vs torchvision.ops.roi_align

anony_anony: <p>I find it is apparently different between the two APIs( tf.image.crop_and_resize vs torchvision.ops.roi_align). I provide the example here:</p>
<pre><code>import numpy as np
b = np.arange(100*100).reshape(1, 100, 100, 1)
boxes = [12, 30, 42, 53]
x1 = boxes[0] / (100-1)
x2 = boxes[2] / (100-1)
y1 = boxes[1] / (100-1)
y2 = boxes[3] / (100-1)

import tensorflow as tf
res = sess.run(tf.image.crop_and_resize(b, [[y1, x1, y2, x2]], [0], [7, 7])).transpose([0, 3, 1, 2])
# transpose is for comparison with the result of pytorch
print("tensorflow:", res)

# pytorch 1.7
from torchvision.ops import roi_pool, roi_align
res1 = roi_align(torch.from_numpy(b.transpose([0, 3,1,2]).astype(np.float32)), torch.from_numpy(np.asarray([[0, boxes[0], boxes[1], boxes[2], boxes[3]]]).astype(np.float32)), output_size=(7, 7))
print(res1)
</code></pre>
<p>The two results are:</p>
<p>tf:</p>
<pre><code>[[[[3012.     3017.     3022.     3027.     3032.     3037.
3042.    ]
[3395.3333 3400.3333 3405.3333 3410.3333 3415.3333 3420.3333
3425.3333]
[3778.6667 3783.6667 3788.6667 3793.6667 3798.6667 3803.6667
3808.6667]
[4162.     4167.     4172.     4177.     4182.     4187.
4192.    ]
[4545.333  4550.333  4555.333  4560.333  4565.333  4570.333
4575.333 ]
[4928.6665 4933.6665 4938.6665 4943.6665 4948.6665 4953.6665
4958.6665]
[5312.     5317.     5322.     5327.     5332.     5337.
5342.    ]]]]
</code></pre>
<p>pytorch:</p>
<pre><code>tensor([[[[3178.4287, 3182.7141, 3187.0000, 3191.2856, 3195.5713, 3199.8569,
       3204.1428],
      [3507.0000, 3511.2856, 3515.5710, 3519.8569, 3524.1426, 3528.4285,
       3532.7141],
      [3835.5715, 3839.8579, 3844.1426, 3848.4285, 3852.7144, 3857.0000,
       3861.2859],
      [4164.1426, 4168.4287, 4172.7139, 4177.0000, 4181.2856, 4185.5713,
       4189.8574],
      [4492.7139, 4497.0000, 4501.2856, 4505.5713, 4509.8569, 4514.1426,
       4518.4287],
      [4821.2861, 4825.5718, 4829.8574, 4834.1431, 4838.4287, 4842.7144,
       4847.0000],
      [5149.8574, 5154.1421, 5158.4292, 5162.7144, 5167.0005, 5171.2856,
       5175.5713]]]])
</code></pre>
<p>The results are really different!</p>
<p>Does anyone know the reason? Thanks in advance.</p>
The relation between accuracy and loss

Lincoln10153: <p>I know that the lower the validation loss, the higher the validation accuracy can not be guaranteed. However, I do not know if this condition is normal, as shown below.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/f/af6d23c68f0a91655f90863b47a2ffe6f0c66d55.png" alt="image" data-base62-sha1="p1To45IJzMCE32g0Mo3KPr79tm5" width="565" height="365"></p>
<p>As we can see, after about epoch 25, there is overfitting (validation loss increases largely). I think when there is overfitting, the validation accuracy should lower. However, I found that after about epoch 25, the validation accuracy still increases. This confuses me.</p>
<p>And also, I found that the highest testing accuracy is not necessarily at the point with lowest validition loss or the point with highest validation accuracy when I used the testing set to test. This also confuses me. Totally, I trained 300 epoches. And I found that the testing accuracy of epoch 300 is usually good whether it has the highest validition accuracy or not.</p>

PLarsen79: <p>It seems after epoch 25 the model is narrowing down on perfecting the prediction of the training data. The overfitting of the model is after that disregarding any more higher level semantics learned during the first 25 epochs and exchanging it for in detail being able to predict your specific training data, validation is not examples from your training data right?. The generalisation seems to be lost due to too small training data corpus.</p>
<p>I have no clue what happened there after ~125 epochs - did you change training data?</p>

Lincoln10153: <aside class="quote no-group" data-username="PLarsen79" data-post="2" data-topic="106742">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/plarsen79/48/32386_2.png" class="avatar"> PLarsen79:</div>
<blockquote>
<p>validation is not examples from your training data right</p>
</blockquote>
</aside>
<p>Hi,<br>
Thanks for your reply.</p>
<p>The validation set is not examples from my training data. The validation set is independent.</p>
<p>And I did not change the training data after about 125 epochs.</p>
<p>Look forward to your reply!</p>

Lincoln10153: <p>Hi,</p>
<p>my training set, validation set, and testing set are all independent.</p>

SimonW: <p>you might just be observing an instance of this mysterious effect in DL: <a href="https://github.com/thegregyang/LossUpAccUp" class="inline-onebox">GitHub - thegregyang/LossUpAccUp: Loss and accuracy go opposite ways...right?</a> <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Lincoln10153: <p>Hi,</p>
<p>Thanks!<br>
I will check this link.</p>
How was conv2d implemented in pytorch?

fangyh: <p>Is there any python code which implements the forward pass of the following one?</p>
<pre><code class="lang-auto">torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)
</code></pre>

albanD: <p>Hi,</p>
<p>No there is none.<br>
There are specific cpp/cuda kernels to do this. In the case of gpu, it mostly uses cudnn own implementation that use many different algorithms itself.</p>
<p>The classic implementation that we use on CPU is based on matrix multiplication and would look like this in python (note that this will use more memory and be slower that the direct conv2d implementation):</p>
<pre><code class="lang-auto">import torch 

inp = torch.randn(1, 3, 10, 12)
w = torch.randn(2, 3, 4, 5)

# Handmade conv
inp_unf = torch.nn.functional.unfold(inp, (4, 5))
out_unf = inp_unf.transpose(1, 2).matmul(w.view(w.size(0), -1).t()).transpose(1, 2)
out = torch.nn.functional.fold(out_unf, (7, 8), (1, 1))
# or equivalently (and avoiding a copy),
# out = out_unf.view(1, 2, 7, 8)

# Check that the result is correct
print((torch.nn.functional.conv2d(inp, w) - out).abs().max())
</code></pre>

evinpinar: <p>Thanks for the answer. Do you know if this handmade conv will give exactly same results with Conv2d? When we train it for some task, will they be exactly same? Or are there any other hidden optimizations/different way of calculating gradients in the cuda implementation?</p>

albanD: <p>Hi,</p>
<p>It will give the same result up to numerical precision of floating point numbers.<br>
So it might not be bit equivalent no.</p>

kraken-d: <p>I found the gradient of weight using this method and it matched the pytorch one but the gradient of the input doesn’t match the pytorch one. Could you please help me to implement it using fold and unfold.</p>
A problem in transform compose

111445: <p>Hi,<br>
I want to add the adjust_brightness function to my transform.Compose,but that function has a integrant parameter img so that i can only apply this to my loop. Is there any methods to make it a compatible part in  transform.Compose for other similar datasets?  I’m a beginner in  pytorch.<br>
Thanks in advance!</p>
<pre><code class="lang-auto">
import matplotlib.pyplot as plt
import numpy as np
import torch
torch.manual_seed(60)
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import torchvision
from torchvision import datasets, transforms
batchSize=1
transform = transforms.Compose([transforms.ToTensor(),
                                transforms.Normalize([0.5],[0.5])])
#Load the training data
trainset = datasets.FashionMNIST(root='./data', train=False,
                                       download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=batchSize,
                                          shuffle=True, num_workers=0)
def imshow(img):
    img = img / 2 + 0.5     # unnormalize back to range [0, 1]
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0))) #rearrange dimensions to numpy format for disply
    plt.show()
for images, labels in trainloader:  # only take first element of dataset
    if labels==torch.tensor([9]):
        images = transforms.functional.adjust_brightness(img = images,brightness_factor = 100)
        imshow(torchvision.utils.make_grid(images))
        
</code></pre>

ptrblck: <p>You could use <code>transforms.Lambda</code> and pass the method (via a <code>partial</code>) to it or write a custom transformation class and use it in the <code>transforms.Compose</code>.</p>
Android install

owen: <p>any ideas for getting android termux to install pytorch.<br>
i have tried for ~about~ a week. still nothing.</p>
<p>android i have arm and aarch64  .both<br>
no conda or anything installs.</p>
<p>so it is all build from source.</p>
<p>the closest i have come is a pytorch-install dir with all kinds of files that do not work.</p>
<p>no =- i do not want to emulate ubunto\u or any pther other gnu/linux.  should go straight through termux.</p>
<p>again . about a week in and i know ~far~ too much about pytorch cmake options …</p>
Parallelize four conv2d layers on GPU

ElToto: <p>I have an input tensor with size [1,3,4,100,100] which corresponds to [batchsize, channels, depth, width, height].<br>
I want to use a 2d convolution for each depth so I need four 2d convolutions. After doing this,<br>
I stack the results again into the depth dimension.<br>
Code:</p>
<pre><code class="lang-auto"> x = torch.ones([1,3,4,100,100], dtype=torch.float32).cuda()
c_1 = nn.Conv2d( in_channels=3, out_channels=100, kernel_size=[3,3],padding=1).cuda() # all conv layers have the same parameters
c_2 = nn.Conv2d(3,100,3,padding=1).cuda()
c_3 = nn.Conv2d(3,100,3,padding=1).cuda()
c_4 = nn.Conv2d(3,100,3,padding=1).cuda()
#### Can i do this in parallel
pred_1 = c_1(x[:,:,0,:])
pred_2 = c_2(x[:,:,1,:])
pred_3 = c_3(x[:,:,2,:])
pred_4 = c_3(x[:,:,3,:])
###
pred = torch.stack([pred_1,pred_2,pred_3,pred_4], dim=2)
# pred has size [1,100,4,100,100]
</code></pre>
<p>This calculates the convolutions after each other.<br>
But each convolution is completely independent of each other. They all take seperate inputs.<br>
Is it possible to calculate all convolutions at the same time on the gpu?</p>

InnovArul: <p>It’s a bit hacky. You could parallelize Conv2d with <code>groups</code> parameter, given that all the convolution layers have same number of input, output channels, kernel_sizes etc.</p>
<p>A small example:</p>
<aside class="onebox githubgist">
  <header class="source">
      <a href="https://gist.github.com/InnovArul/bd8b7ce5ac9e615ec251470293563eb2" target="_blank" rel="noopener">gist.github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://gist.github.com/InnovArul/bd8b7ce5ac9e615ec251470293563eb2" target="_blank" rel="noopener">https://gist.github.com/InnovArul/bd8b7ce5ac9e615ec251470293563eb2</a></h4>
<h5>conv2d_groups_trick.py</h5>
<pre><code class="Python">import torch, torch.nn as nn, torch.nn.functional as F

def perform_non_parallelconv(input, convs):
    outs = []
    for i in range(len(convs)):
        o = convs[i](input[:, i])
        outs.append(o)

    outs = torch.cat(outs, dim=1)
    return outs</code></pre>
This file has been truncated. <a href="https://gist.github.com/InnovArul/bd8b7ce5ac9e615ec251470293563eb2" target="_blank" rel="noopener">show original</a>

<p>
</p>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
DataParallel in pytorch blitz tutorial

souvik3333: <p>I was reading about data parallel in <a href="https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html" rel="noopener nofollow ugc">here</a>. Its mentioned that we can use it by just calling <code>model = nn.DataParallel(model)</code>. Then there is a <a href="https://pytorch.org/tutorials/beginner/former_torchies/parallelism_tutorial.html" rel="noopener nofollow ugc">link</a> to another tutorial at the bottom of the page. There it is mentioned to use DataParallel like this <code>self.block2 = nn.DataParallel(self.block2)</code>.</p>
<p>My question is what is the difference between them? does the first method do what second method does for every block automatically? if yes Which one is best practice if I want that for every block?  Also in second method what happens to the blocks which are not inside DataParallel?</p>

rvarm1: <p>DataParallel can be applied to any <code>nn.Module</code>. For the first example, you are correct in that every <code>nn.Module</code> (i.e. the blocks) will be data parallel as well, since the entire module is.</p>
<p>The second example mostly seeks to show that you can have a regular <code>nn.Module</code>, but have a data parallel module contained within it, and everything should work as expected, and only the data parallel module will be parallelized across the batch dimension.</p>
Pytorch - network with non-differentiable module

cojdex: <p>Hi, I would like to know how to write in pytorch and train a neural network that has a module that is explicitly non-diferentiable, for example a module/code written in another language that I cannot re-write into python (a black box function). In that case, I imagine that the derivatives have to be calculated using finites differences. Could I define this function in a way pytorch can use it? Thanks very much!</p>

tom: <p>Finite differences does not give gradients easily - you would need to have many evaluations (1 base + 1 per input size) of your black box function to compute the derivative. Then, of course, it could work.</p>

cojdex: <p>Hi Tom, thanks for your comment. Could you expand the answer with code? Could you show me how to define a function with those properties? Thanks a lot!</p>

cojdex: <p>Hi Tom, thanks again for your comment. Maybe redefining the autograd function of the black box like this: <a href="https://pytorch.org/tutorials/beginner/examples_autograd/polynomial_custom_function.html" class="inline-onebox" rel="noopener nofollow ugc">PyTorch: Defining New autograd Functions — PyTorch Tutorials 1.7.1 documentation</a> ?</p>
<p>Thanks!</p>
ValueError: ('Cannot warp empty image with dimensions', (257, 0))

anotherone_one: <p>Hello everyone!!</p>
<p>I am getting a weird error on the training phase on random epochs, depending on the batch size, saying ValueError: (‘Cannot warp empty image with dimensions’, (257, 0)). For example, when I have a batch of 5 and I am training the inception v3,  the error only occurs on epoch 32, which doesn’t make sense since the other epoch until this point iterated over the whole dataset too, so I don’t understand this random crash. The error occurs when loading the new batch in the epoch:</p>
<pre><code class="lang-auto">#Now we iterate throught the batches of the dataloader
       for batch in loader:
</code></pre>
<p>Thank you in advance!</p>

ptrblck: <p>I guess this error might be raised by a <code>scipy</code> transformation, if an empty numpy array is passed to it.<br>
You could add print statements to the <code>Dataset.__getitem__</code> method and check, which index (and transformation) raises this error to narrow down, if the previous transformations might have created an empty array.</p>

anotherone_one: <p>I actually don’t use scipy, I only use skimage in the method! I did that debug, I printed all the shaped and all the images of my dataset, however it doesn’t crash <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"> But I know remembered that I didn’t try to iterate the dataloader, I only iterated the datasets. I am going to try that</p>

anotherone_one: <p>My ROI detector had a bug, it was trying to crop an image with width 0.</p>
<p>Thank you !</p>
Summary of our model

swarup: <p>how to get summary of our model?<br>
Like in keras we have model.summary()</p>

snowe: <p>Hi <a class="mention" href="/u/swarup">@swarup</a></p>
<p>You can use:</p>
<pre><code class="lang-auto">from torchsummary import summary
model = Net() # instantiate your model here
summary(model.cuda(), (3,32,32)) # specify the input shape your model expects
</code></pre>