akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Assemble neural networks to improve performance

Gianmarco_Luchetti_S: <p>Hello everybody,<br>
I am approaching the world of Geometric Deep Learning for the first time and I have a question, I hope someone can answer it. I am currently working on models to classify some drugs as highly active, active and inactive (labels: 0, 1, 2) using different neural networks (such as Convolutional Graph Neural Network or Graph Isomorphism Network). The problem is that I can’t get high performance with any model, but I’ve noticed that each network makes different mistakes so I thought I’d put them together and make them work together.<br>
Now the real questions are:</p>
<ol>
<li>do you believe that assembling networks is a good strategy to increase performance?</li>
<li>would it be better to train networks with different datasets or with the same set before assembling them?</li>
</ol>
<p>Could you also recommend some assembly techniques, articles or tutorials? Thank you all!</p>
A fully connect layer which maps a vector to a matrix

zecheng_zhang1: <p>My network input has dimension $[K, n]$ where $K$ is the number of samples, I want to create a layer that will map this vector of dimension $n$ to a matrix $[d, n]$, that is, the output of the network is $[K, d, n]$ does anyone know which nn.layer I can use? Thank you very much!</p>

KFrank: <p>Hi Zecheng!</p>
<aside class="quote no-group" data-username="zecheng_zhang1" data-post="1" data-topic="154509" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zecheng_zhang1/48/50664_2.png" class="avatar"> zecheng_zhang1:</div>
<blockquote>
<p>I want to create a layer that will map this vector of dimension $n$ to a matrix $[d, n]$</p>
</blockquote>
</aside>
<p>You can simply take the output of a <code>Linear</code> that has <code>out_features = d * n</code><br>
and <code>view()</code> (or <code>reshape()</code> or <code>Unflatten</code>) it as a batch of <code>d x n</code> matrices:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.11.0'
&gt;&gt;&gt; K = 2
&gt;&gt;&gt; n = 5
&gt;&gt;&gt; d = 7
&gt;&gt;&gt; input = torch.randn (K, n)
&gt;&gt;&gt; torch.nn.Linear (in_features = n, out_features = d * n) (input).view (-1, d, n).shape
torch.Size([2, 7, 5])
</code></pre>
<p>Because <code>Linear</code> is fully connected, it has the right number degrees of<br>
freedom to generate any affine (i.e., “linear”) transformation that maps<br>
a vector of length <code>n</code> to a matrix of size <code>d x n</code>.  (This transformation then<br>
gets applied sample-wise to a batch of <code>K</code> samples.)</p>
<p>Best.</p>
<p>K. Frank</p>
Will fine-tune with cropped image influence the performance

Jo-w: <p>Hi there,</p>
<p>Currently, I have a well-trained segmentation model on GTA5. I’d like to further fine-tune this trained model on a self-defined cropped gta5 image. Now I have two choices:</p>
<ol>
<li>crop the original image and input a small-sized cropped version into the network.</li>
<li>keep the image as original size and replace all non-crop regions with 0 (non-cropped region will be labeled as 255).</li>
</ol>
<p>I was just wondering if these two operations have significantly different performance.</p>
<p>Thank you!</p>
Problem with pytorch in training stylegan3 - process timeout

Fu_Wenjin: <p>I faced a problem on training stylegan3 where the terminal stuck at "Setting up PyTorch plugin “upfirdn2d_plugin”… ". I have tried all the methods I found, such as reinstall conda, re-download the whole stylegan3 to retrain the whole network, and pip install ninja etc,. but none of them work. I don’t know if anyone met similar situation before, and how do you solve it? I have attached the screenshot of the problem below. Thank you so much!!!</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/4/84866ff37a9dce62f46d241f4a23e4ff26a97923.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/84866ff37a9dce62f46d241f4a23e4ff26a97923" title="stylegan3-problem"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/4/84866ff37a9dce62f46d241f4a23e4ff26a97923_2_150x500.jpeg" alt="stylegan3-problem" data-base62-sha1="iUn3vYsDMARcSBIdcEnxVDmvX7Z" width="150" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/4/84866ff37a9dce62f46d241f4a23e4ff26a97923_2_150x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/8/4/84866ff37a9dce62f46d241f4a23e4ff26a97923_2_225x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/4/84866ff37a9dce62f46d241f4a23e4ff26a97923_2_300x1000.jpeg 2x" data-dominant-color="262626"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">stylegan3-problem</span><span class="informations">1904×6328 637 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
How to bin the values of a pytorch tensor to the nearest maximum and minimum from an array

Vedant_Sanil1: <p>Say I have an array of values <code>w = [w1, w2, w3, ...., wn]</code> and this array is sorted in ascending order, all values being equally spaced.</p>
<p>I have a pytorch tensor of any arbitrary shape. For the sake of this example, lets say that tensor is:</p>
<pre><code class="lang-auto">import torch 

a = torch.rand(2,4)
</code></pre>
<p>assuming <code>w1=torch.min(a)</code> and <code>wn=torch.max(a)</code>, I want to create two separate tensors, <code>amax</code> and <code>amin</code>, both of shape <code>(2,4)</code> such that <code>amax</code> contains values from <code>w</code> that are the nearest maximum value to the elements of <code>a</code>, and vice-versa for <code>amin</code>.</p>
<p>As an example, say:</p>
<pre><code class="lang-auto">a = tensor([[0.7192, 0.6264, 0.5180, 0.8836],
            [0.1067, 0.1216, 0.6250, 0.7356]])

w = [0.0, 0.33, 0.66, 1]
</code></pre>
<p>therefore, I would like <code>amax</code> and <code>amin</code> to be,</p>
<pre><code class="lang-auto">amax = tensor([[1.000, 0.66, 0.66, 1.000],
               [0.33, 0.33, 0.66, 1.00]])

amin = tensor([[0.66, 0.33, 0.33, 0.66],
               [0.00, 0.00, 0.33, 0.66]])
</code></pre>
<p>What is the fastest way to do this?</p>
When and why exactly is NNPack initialized?

NicoAdamo: <p>Hi, I am running PyTorch’s nightly build inside a docker container running Linux on an M1 mac (macOS 12.4). The following code:</p>
<pre><code class="lang-auto">weights = ResNet18_Weights.DEFAULT

img = read_image(f"./test/data/torch_test/dog.jpeg")
  
preprocess = weights.transforms()
batch = preprocess(img).unsqueeze(0)
    
model = resnet18(weights=weights)
model.eval()
prediction = model(batch).squeeze(0).softmax(0)

weights = FasterRCNN_ResNet50_FPN_Weights.DEFAULT
     
model = fasterrcnn_resnet50_fpn(weights=FasterRCNN_ResNet50_FPN_Weights.DEFAULT)
model.eval()
boxes = model(batch)
</code></pre>
<p>Produces the warning <code>Could not initialize NNPACK! Reason: Unsupported hardware</code>, which makes sense considering I am computing convolutions on the M1, but the warning is not raised until the final line. What is different about <code>fastrrcnn_resnet50_fpn</code> from <code>resnet18</code> that torch doesn’t attempt to initialize nnpack until the detection model is called, rather than the classification? It looks like under the hood they’re both normal Conv2ds.</p>
<p>For context, this failure to initialize seems to be causing a massive (&gt;1000x) slowdown in inference for the detection model compared to inference with the classification model (I know the classification model should be faster, but not THAT much faster) and I’m trying to understand how the classification model gets by without initializing NNPack but the detection one needs it.</p>
Keeping the modification inplace after masking a tensor

Thibault712: <p>Let say for the example I have:</p>
<pre><code class="lang-auto">x = torch.randn((10, 20))
mask = torch.bernoulli(torch.full(size=(10,), fill_value=0.50)).type(torch.bool)
i = 0
oh = F.one_hot(1, num_classes=20).type(torch.float)
</code></pre>
<p>When I do</p>
<pre><code class="lang-auto">x[mask][i] = oh
</code></pre>
<p>I’m hoping that one of the line of x is replaced by oh but it’s not the case, the change is not kept why? Do you have an alternative for it to be the case</p>
<p>I found a way to make it work but it’s not pretty:</p>
<pre><code class="lang-auto">i_star = torch.arange(x.size(0))[mask][i]
x[i_star] = oh
</code></pre>
<p>I’m sure there is a more pythonic way</p>

ptrblck: <aside class="quote no-group" data-username="Thibault712" data-post="1" data-topic="154267">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/thibault712/48/36947_2.png" class="avatar"> Thibault712:</div>
<blockquote>
<p>the change is not kept why?</p>
</blockquote>
</aside>
<p>I think because you are indexing an intermediate result by “chaining” the indexing operation.</p>
<p>I’m not sure I fully understand your use case, but it seems you want to index <code>mask</code> with <code>i</code> first so I guess this should also work:</p>
<pre data-code-wrap="python"><code class="lang-python">x[mask.nonzero()[i]] = oh
</code></pre>

InnovArul: <aside class="quote no-group" data-username="Thibault712" data-post="1" data-topic="154267">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/thibault712/48/36947_2.png" class="avatar"> Thibault712:</div>
<blockquote>
<p><code>x[mask]</code></p>
</blockquote>
</aside>
<p>Just adding an additional note to <a class="mention" href="/u/ptrblck">@ptrblck</a>’s answer, I think <code>x[mask]</code> is a type of advanced indexing and according to PyTorch documentation, this indexing returns a <code>copy</code> of the tensor rather than a <code>view</code> of the underlying tensor.</p>
<p>Quoting from <a href="https://pytorch.org/docs/stable/tensor_view.html" class="inline-onebox">Tensor Views — PyTorch 2.1 documentation</a>,</p>
<pre><code class="lang-auto">When accessing the contents of a tensor via indexing, PyTorch follows Numpy behaviors that basic indexing returns views, while advanced indexing returns a copy.
</code></pre>
<p>Thus <code>x[mask][i] = oh</code> replaces the elements in the <code>copy</code> of the tensor rather than the <code>view</code> of the underlying tensor. So, <code>x</code> is unmodified.</p>
How to resize an affine_grid

Zac_Boussaid: <p>Hi guys,<br>
I have a simple question: normally when I want to resize a 4D tensor, I apply the resize method provided by the torchvision.transforms library, and everything works correctly because I pass a 4D tensor with this form: [num_batches, num_channels, Width, Height].</p>
<p>I wanted to try the same function with the output of the affine_grid function and I found that the output has this form: [num_channels, Width, Height, 2]</p>
<p>Could someone give me an idea or example on how to resize the output of F.affine_grid?</p>
<p>Thank you in advance</p>

Zac_Boussaid: <p>just implement the function nn.permute then resize the image using resize then reuse the method permute to set the output of the affine_grid as the original form.<br>
transform2 = T.Resize((28,28))<br>
grid = grid.permute(0, 3, 1, 2)<br>
grid = transform2(grid)<br>
grid = grid.permute(0, 2, 3, 1)</p>
<p><a href="https://pytorch.org/docs/stable/generated/torch.permute.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.permute.html</a></p>
Groupby mean according to two tensor

kavka: <p>Suppose we have the tensor looks this follow, assume that we have 3 different class.</p>
<pre><code class="lang-auto">sample = torch.Tensor([
                     [0.1, 0.1],    #-&gt; batch 1 / class 1
                     [0.2, 0.2],    #-&gt; batch 1 / class 2
                     [0.4, 0.4],    #-&gt; batch 1 / class 2
                     [0.5, 0.5]     #-&gt; batch 1 / class 0
                     [0.7, 0.7]     #-&gt; batch 1 / class 0
                     [0.3, 0.3]     #-&gt; batch 2 / class 1
                     [0.1, 0.1]     #-&gt; batch 2 / class 1
                     [0.8, 0.8]     #-&gt; batch 2 / class 0
              ])

batch_num = torch.Tensor([5, 3], dtype = torch.Long)
node_type = torch.Tensor([1, 2, 2, 0, 0, 1, 1, 0], dtype = torch.Long)
</code></pre>
<p><code>sample</code> is our data.</p>
<p><code>batch_num </code>means how many nodes in each sample in the batch.</p>
<p><code>node_type </code> means type of each node,  <code>len(node_type) == sample .size(0)</code></p>
<p>What I want to compute is that:</p>
<p>for each sample, I want to compute the mean according to the class, so the final result would looks like</p>
<pre><code class="lang-auto">output= torch.Tensor([
                     [0.1, 0.1],    #-&gt; batch 1 / class 1
                     [0.3, 0.3],    #-&gt; batch 1 / class 2
                     [0.6, 0.6]     #-&gt; batch 1 / class 0
                     [0.2, 0.2]     #-&gt; batch 2 / class 1
                     [0.8, 0.8]     #-&gt; batch 2 / class 0
                     [0., 0.]       #-&gt; batch 2 / class 2
              ])
</code></pre>
<p>Note that, for batch 2, there only two classes, for this case, we need to add <code>0</code> tensor to it</p>
<p>I was stuck here for a while, any help would be appreciated.</p>
AttributeError: module 'config' has no attribute 'model_path'

PPB: <p>Although I have assigned a model path in the config.py file, I’m getting the following error.</p>
<pre><code class="lang-auto">if mode == "test":
    # Test data address
    lr_dir = f"./data/"
    sr_dir = f"./results/"
    hr_dir = f"./data/"

    model_path = "./results/baseline/g_best.pth.tar
</code></pre>
<blockquote>
<p>Traceback (most recent call last):<br>
File “./test.py”, line 111, in <br>
main()<br>
File “./test.py”, line 33, in main<br>
checkpoint = torch.load(config.model_path, map_location=lambda storage, loc: storage)<br>
<strong>AttributeError: module ‘config’ has no attribute ‘model_path’</strong></p>
</blockquote>
<p><strong>Pytorch version: 1.10.2</strong><br>
<strong>Python: 3.6.13</strong><br>
Please help me in this regard.</p>
Looking for an efficient way to index 2D tensor by another 2D tensor

Kriss: <p>I have a tensor, say</p>
<pre><code class="lang-auto">A = tensor([
                   [0, 0],
                   [0, 2],
                   [0, 3],
                   [0, 4],
                   [0, 5],
                   [0, 6],
                   [1, 0],
                   [1, 1],
                   [1, 4],
                   [1, 5],
                   [1, 6]
       ])
</code></pre>
<p>and the other tensor</p>
<pre><code class="lang-auto">b = tensor([[0, 2], [1, 2]])
</code></pre>
<p>I would like to find an efficient way to index into A by b such that the result is</p>
<pre><code class="lang-auto">result = tensor([[0, 3], [1, 4]])
</code></pre>
<p>That is, match A’s first column of last dim (i.e. [0,…,1…]) with b’s first column of the last dim (i.e. [0,1]) by their values and then use b’s second column (i.e. [2, 2]) to index A’s second column.</p>
<p>Thanks</p>

harpone: <p>Can’t give you a straight answer to this, but it sounds like you might want to separate the two steps… first do the 0:th column matching, then use e.g. <code>torch.gather</code> to gather values from <code>A[:, 1]</code> by indexing with <code>b[:, 1]</code>.</p>

Kriss: <p>Thanks for your suggestion.<br>
I convert it into one dimensional problem with torch.nonzero and offset by mask sum, and work out a solution.<br>
Instead of the original A, get a flatten version, like</p>
<pre><code class="lang-auto">A = tensor([[ 0],
			[ 2],
			[ 3],
			[ 4],
			[ 5],
			[ 7],
			[ 8],
			[11],
			[12]])
</code></pre>
<p>and also calculate the offsets along batch,</p>
<pre><code class="lang-auto">offset = tensor([[0],
				 [5],
				 [4]])
</code></pre>
<p>Similarly, get b</p>
<pre><code class="lang-auto">b = tensor([2, 2])
</code></pre>
<p>and</p>
<pre><code class="lang-auto">offset_b = b+offset.reshape(-1)[:-1]
</code></pre>
<p>Then</p>
<pre><code class="lang-auto">indices=A.reshape(-1)[offset_b]
</code></pre>
How to compare to models created by nn.module

hitbuyi: <p>suppose that  I have two models, which are created both by nn.module</p>
<p>model1  = Sequential(…)<br>
model2 = Sequential(…)</p>
<p>model1 and modle2 looks like each other</p>
<p>so ,how to compare model1 and model 2 exactly? does pytorch provide API to compare them?</p>
<p>Thanks</p>

ptrblck: <p>Could you explain what “compare” means in your use case?<br>
If you want to check the parameters of both models and see if they are equal, you could check both <code>state_dict</code>s and use e.g <code>torch.equal</code> for each parameter/buffer.<br>
On the other hand, if you want to verify the same forward pass logic (additionally to the <code>state_dict</code>s), you could make sure the results are deterministic by following <a href="https://pytorch.org/docs/stable/notes/randomness.html">these docs</a> and compare both outputs.</p>

hitbuyi: <p>I wanna to compare both parameters network architecture</p>
<ol>
<li>whether the net architectures are same or not</li>
<li>if net architectures are same, I wanna to compare if the parameters are same</li>
</ol>
How to store backprop gradient properly to avoid CUDA out-of-memory error

promach: <p>Hi, I am having issue with <a href="https://www.diffchecker.com/qXb0RQBo" rel="noopener nofollow ugc">backprop gradient</a> that <a href="https://colab.research.google.com/drive/1v5wY22CkyvKPz21tdwSMPv0T3fsIro0D?usp=sharing" rel="noopener nofollow ugc">eats up a lot of GPU memory</a>.</p>
<p>Note: I tried to decrease <code>BATCH_SIZE</code> , but it does not help.</p>

promach: <p>Someone told me to use deepspeed zero offload.</p>
<p>However, my code is quite similar to some GNN structure : <a href="https://gist.github.com/buttercutter/b6f526c56e20f029d68e6f9041c3f5c0/3d1a3e6844680545fb8b75225267ea625ba9df5b#file-gdas-py-L665" rel="noopener nofollow ugc">NN_output = graph.forward(NN_input, types=“f”)</a></p>
<p>So, <a href="https://github.com/microsoft/DeepSpeedExamples/blob/36212dd59cb3eb342c39bc8965aaba04d5491933/cifar/cifar10_deepspeed.py#L281" rel="noopener nofollow ugc">outputs = model_engine(inputs)</a> seems does not really fit in my case ? <code>args</code> also does not follow such code styling.</p>
<p>Any idea ?</p>
Torchinfo issue

Xanthan: <p>So, I’m working on my research and I have build a model and I’m facing a problem while printing model’s summary.</p>
<p>The model takes a (3,224,224) shape image with batch size = 2. When I pass a batch through the model, it passes without any problem. I wanted to see the model summary so, I used <code>torchinfo</code>'s <code>summary</code> and I face this error:</p>
<pre><code class="lang-auto">TypeError: Model contains a layer with an unsupported input or output type: 0, type: &lt;class 'int'&gt;
</code></pre>
<p>I can show the last traceback call which led to this error:</p>
<pre><code class="lang-auto">
D:\Anaconda\envs\ffconvolution\lib\site-packages\torchinfo\torchinfo.py in hook(***failed resolving arguments***)
    507         info.input_size = info.calculate_size(inputs, batch_dim)
--&gt; 508         info.output_size = info.calculate_size(outputs, batch_dim)
    509         info.executed = True

D:\Anaconda\envs\ffconvolution\lib\site-packages\torchinfo\layer_info.py in calculate_size(inputs, batch_dim)
    101         else:
--&gt; 102             raise TypeError(
    103                 "Model contains a layer with an unsupported input or output type: "
</code></pre>
<p>I’m not sure what’s going wrong. Give suggestions.</p>

TylerYep: <p>Could you post your complete code? What is the model you are using?</p>
Non-deterministic behavior in Custom CUDA extension

Matthias_WB: <p>Hey there!</p>
<p>Found a strange reproducibility issue in a custom CUDA extension.<br>
[The forward pass as well as backward pass work; tested against a numerical (slow) torch implementation of our custom layer. ]<br>
The issues are:</p>
<ul>
<li>
<p>The backward pass for the weights only gives the correct results for batch_size =1. Else-wise, it randomly produces the correct numerical results.</p>
</li>
<li>
<p>The error to the correct results is a multiple of the previous result for batch_size=1. My guess: the CUDA Kernel is accessing memory pointers multiple times. Is there an issue with my thread/block indexing?</p>
</li>
<li>
<p><strong>Most notable the non-deterministic behavior</strong>: when I execute the code repeatedly  the results change for the same inputs. Again the shifts are multiples of the results for batch_size =1 for the given index of d_weights respectively.</p>
</li>
</ul>
<p>Here is the relevant backward pass code. The entire code is more involved in particular for the backward pass for the inputs. However, the d_weights returned for this function do not get modified anywhere else.</p>
<p>//batch index<br>
const int n = blockIdx.y;<br>
// column index<br>
const int c = blockIdx.x * blockDim.x + threadIdx.x;<br>
const int state_size = input.size(1);<br>
const int batch_size = input.size(0);<br>
if (c &lt; state_size){<br>
if (n &lt; batch_size){ for (int i = 3<em>c</em>state_size; i &lt; 3*(c+1)*state_size; i += 1){<br>
d_weights[i] += (input[n][idxs[0][i]]-input[n][idxs[1][i]])*grad_out[n][c];<br>
…}</p>
<p><strong>The workaround I can think of rely on extending the tensors by another dimensions of length = state_size; Which is however what I want to prevent was it will cause memory shortages.</strong></p>

Matthias_WB: <p>The solution is to use atomic_add:</p>
<p>atomicAdd(&amp;d_weights[i], (input[n][idxs[0][i]]-input[n][idxs[1][i]])*grad_out[n][c]);</p>
How can I get the gradient of quantized tensor?

Rahul_Xie: <p>I quantized my model with post-training quantization in PyTorch. Then I want to compare the gradient of input ‘x’ before and after quantization. I first set <code>requires_grad=True</code>. After <code>self.quant(x)</code>, <code>x.requires_grad</code> became False. When I set it to True, it reported <code>RuntimeError: only Tensors of floating point dtype can require gradients</code>. How can I get the gradient of quantized tensor?</p>
<pre><code class="lang-auto">x = self.preprocess(x)
x = self.quant(x)
# x.requires_grad_(True)
x = self.features(x)
</code></pre>

andrewor: <p>Hi Rahul,</p>
<p>As the error message suggests, I don’t think it is possible to get the gradient of a quantized tensor. The only quantization mode in which we need gradients is QAT, and in this mode all weights and activations are only “fake quantized” and not actually cast to int8 values. Please see <a href="https://pytorch.org/blog/introduction-to-quantization-on-pytorch/" rel="noopener nofollow ugc">this blog post</a> for more information.</p>
<p>Best,<br>
-Andrew</p>

Rahul_Xie: <p>Thanks for your reply.</p>
Content of dataloader?

arcoyk: <p>Hi could someone explain the content of dataloader?</p>
<p>Example from <a href="https://colab.research.google.com/github/smartgeometry-ucl/dl4g/blob/master/autoencoder.ipynb#scrollTo=k29VhdUIuSPn" rel="noopener nofollow ugc">Google Colab notebook</a>, they use</p>
<pre><code class="lang-auto">for image_batch, _ in train_dataloader:
</code></pre>
<p>What is “_” here? If I print it, it shows</p>
<pre><code class="lang-auto">tensor([4, 5, 3, 9, 0, 4, 4, 1, 1, 3, 8, 7, 4, 9, 4, 1, 0, 0, 0, 0, 7, 1, 1, 6,
        4, 9, 1, 6, 6, 2, 3, 3, 8, 4, 6, 3, 1, 8, 1, 7, 7, 2, 4, 5, 3, 4, 9, 7,
        1, 0, 1, 4, 7, 5, 1, 0, 5, 0, 7, 9, 5, 4, 0, 8, 5, 6, 1, 1, 3, 1, 7, 2,
        4, 4, 0, 2, 6, 8, 4, 6, 2, 4, 5, 5, 3, 7, 7, 6, 2, 7, 4, 6, 6, 0, 6, 0,
        0, 7, 1, 9, 7, 6, 3, 8, 2, 0, 5, 9, 3, 5, 7, 5, 1, 6, 7, 8, 2, 7, 2, 5,
        9, 8, 8, 6, 6, 2, 1, 2])
</code></pre>

fpozzi: <p>Hi <a class="mention" href="/u/arcoyk">@arcoyk</a>,</p>
<p>the <code>train_dataloader</code> iterates over the <a href="https://pytorch.org/vision/stable/generated/torchvision.datasets.MNIST.html#torchvision.datasets.MNIST" rel="noopener nofollow ugc">MNIST</a> dataset, so it returns pairs composed of <code>image_batch</code> and <code>label_batch</code>.</p>
<p>For further details, look <a href="https://github.com/pytorch/vision/blob/12bb88738a5b6b96767c4165282f644495780a80/torchvision/datasets/mnist.py#L150" rel="noopener nofollow ugc">here</a>.</p>

arcoyk: <p><a class="mention" href="/u/fpozzi">@fpozzi</a> Thank you for quick rep!</p>

arcoyk: <p>Follow up memo to someone who wondered this line.</p>
<p>Why</p>
<pre><code class="lang-auto">train_dataloader[0]
</code></pre>
<p>isn’t allowed though</p>
<pre><code class="lang-auto">for image_batch, _ in train_dataloader
</code></pre>
<p>is allowed.</p>
<p>This is because train_dataloader uses custom iterator <code>torch.utils.data.dataloader._SingleProcessDataLoaderIter</code>.</p>
<p>You can get an element (though not recommended)</p>
<pre><code class="lang-auto">t = train_dataloader.__iter__()
a, b = t.next()
</code></pre>
New_boxes = boxes + torch.FloatTensor([left, top, left, top]).unsqueeze( RuntimeError: The size of tensor a (0) must match the size of tensor b (4) at non-singleton dimension 1

gesm: <p>I faced this error when I tried to train model with large dataset (as I tested it with a few subset and works fine). I added “drop_last=True” to avoid the division problem by the batch size.</p>
<p>Any idea about the error I faced :     “The size of tensor a (0) must match the size of tensor b (4) at non-singleton dimension 1”</p>
<p>Thanks</p>

ptrblck: <p>Based on the error message it seems <code>boxes</code> is an empty tensor:</p>
<pre><code class="lang-python">boxes = torch.tensor([])
boxes + torch.tensor([0., 1., 2., 3.])
# RuntimeError: The size of tensor a (0) must match the size of tensor b (4) at non-singleton dimension 0
</code></pre>
<p>so make sure this tensor is valid before trying to add another tensor to it.</p>
Export to ONNX failure with mismatched devices

Junyuan: <p>hi PyTorch, I am trying to do ONNX conversion for a module and encountered following error:</p>
<pre><code class="lang-auto">from torch.onnx.symbolic_helper import _constant_folding_opset_versions
        if do_constant_folding and _export_onnx_opset_version in _constant_folding_opset_versions:
            params_dict = torch._C._jit_pass_onnx_constant_fold(graph, params_dict,
&gt;                                                               _export_onnx_opset_version)
E           RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
</code></pre>
<p>This issue happens for PyTorch v1.11.<br>
I explicitly checked, on the pytorch side, the tensors are all on GPU.<br>
But when I trace into <code>constant_fold.cpp</code> , failure was triggered for an <code>onnx:mul() </code> OP, two input values are on different devices, CPU and GPU.</p>
<p>Can some one help on this?</p>
<p>Thanks in advance!</p>

ptrblck: <p>Cross-posting from Slack:</p>
<blockquote>
<p>Could you also check if tracing the model would already fail (i.e. via <code>torch.jit.trace</code> )?</p>
</blockquote>
Sort tensors inplace

eslam_bakr: <p>Hello Everyone,</p>
<p>I encounter Cuda-out-of-memory error during the sorting operation due to a huge tensor.<br>
The tensor shape is [40000, 2048] thus I think If we can perform the sorting operation in an inplace manner it would be very nice feature to become more efficient.<br>
Also, is there any workaround to make it inplace by myself without changing the torch version, i.e., 1.5?</p>
<p>Best regards.</p>
Does `torch::deploy` isolate different interpreters?

reyoung: <p>I just read <code>torch::deploy</code> code and it uses <code>dlopen</code> multiple python interpreters to avoid GIL of python. However, as I remember, the global memory addresses across multiple dlopened libraries are same.</p>
<p>So if a python interpreter imports a C++ library containing a global symbol <code>A</code>, and another one imports a C++ library also containing a global symbol <code>A</code>, will it conflict? Are global variables in native modules shared across interpreters?</p>
<p>Also, what if a python interpreter just <code>sigsegv</code>? Will other python interpreters keep running or the process will be panic?</p>

d4l3k: <h2>
<a name="interpreter-c-loading-1" class="anchor" href="#interpreter-c-loading-1"></a>Interpreter C Loading</h2>
<p>When an interpreter loads a custom Python C extension we isolate those symbols with the copy of the interpreter. Since C extensions depend on “global” symbols provided by Python we need to do this otherwise you run into issues. What we don’t do currently do is isolate the symbols of dependencies of those C extensions. That’s basically how we make torch work by loading the Python specific parts in the isolated symbol namespace and then libtorch is shared across all processes.</p>
<p>For most Python C extensions they’re self contained so they’re isolated and shouldn’t have any shared state.</p>
<p>I.e. for typical cases:</p>
<p>Interpreter → <code>import foo._C</code> → loads foo/_C.so (interpreter specific, isolated)</p>
<p>if _C depends on “libfoo.so”</p>
<p>Interpreter → <code>import foo._C</code> → loads foo/_C.so (interpreter specific, isolated) → libfoo.so (shared)</p>
<h2>
<a name="sigsev-2" class="anchor" href="#sigsev-2"></a>SIGSEV</h2>
<p>Generally speaking sigsevs are not recoverable in a process. Since Deploy does have some shared state (i.e. libtorch) if it sigsevs it’s not guaranteed to be recoverable so we just crash instead. If you need extra protection from sigsev etc in a shared environment I’d recommend minimizing the number of C extensions to well tested ones when using deploy or running separate processes per model.</p>
<p>Since libdeploy is a library you could add in a custom signal handler to “gracefully” stop an interpreter and handle certain cases of failures but there’s no guarantees of correctness</p>
TransformerEncoderLayer Neural Network model accuracy changes based on batch size

samuel_fipps: <p>I have trained a Neural Network model with a TransformerEncoderLayer in it. I have the model saved, but when I go to load the model and evaluate the model, the accuracy changes based on the batch size. Why is this?</p>

Andrei_Cristea: <p>Can you show a simple example (with some reproducible code) that illustrates this?</p>

samuel_fipps: <p>It’s just any time I use nn.TransformerEncoderLayer in anyway with a saved model if the data is in a different order i get different results. Is there a way to save the Encode table, this would be in the MultiheadAttention part of the TransformerEncoderLayer right?</p>
<p>I’m some what new to Transformers.</p>
<p>edit:<br>
just using TransformerEncoderLayer and save the model and then use np.random.permutation() to shuffle the input data. This always gives me different results unless I use the same order every time.</p>

samuel_fipps: <p>i have this layer in my model like this self.transformer = nn.TransformerEncoderLayer()<br>
and save the model like so<br>
torch.save(model, path)<br>
does this not save the nn.TransformerEncoderLayer() or something?</p>

Andrei_Cristea: <p>Typically one saves the state_dict, per <a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html" rel="noopener nofollow ugc">the instructions</a>:</p>
<pre><code class="lang-auto">torch.save(model.state_dict(), path)
</code></pre>

samuel_fipps: <p>They both do the same thing, my way just allows you not to have to redefine the model. Sorry for the late response I got sick.</p>

samuel_fipps: <p>It has something to do with the nn.TransformerEncoderLayer() because it works fine if i take this layer out</p>

Andrei_Cristea: <aside class="quote no-group" data-username="samuel_fipps" data-post="3" data-topic="154131">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/samuel_fipps/48/48179_2.png" class="avatar"> samuel_fipps:</div>
<blockquote>
<p>TransformerEncoderLayer</p>
</blockquote>
</aside>
<p>Got it, what is the shape of your input, and are you using <code>batch_first=True</code> or not? Basically one thing to just make sure of is that you don’t have your batch and the sequence dimensions mixed up in your implementation.</p>
<pre><code class="lang-auto">batch_first – If True, then the input and output tensors are provided as (batch, seq, feature). Default: False (seq, batch, feature).
</code></pre>

samuel_fipps: <p>Thank you! this might be my problem</p>
Overriding Device Transfer Behavior

Turakar: <p>I have a model which consists of a shared body and multiple task-specific heads. For each batch, I forward- and backward-pass through the shared body and one of the task-specific heads.</p>
<p>Keeping all task-specific heads in the CUDA memory is not possible on the GPUs available to me. Therefore, I want to dynamically load the required task-specific head for each batch into the GPU memory and unload the previous one. Therefore, I plan two modifications to my existing (rather long) code:</p>
<ol>
<li>Override <code>to()</code> such that it only transfers the shared body to GPU memory.</li>
<li>Add some <code>to()</code> calls in <code>forward()</code> to load the required task-specific head to a local variable temporarily, such that it is garbage collected at the end of the optimizer step.</li>
</ol>
<p>My question is: Is it sufficient to override <code>to()</code> or do I have to also override <code>cuda()</code> and <code>cpu()</code>? I noticed that there is some logic hardcoded in <code>Module.to()</code>, so I think I need to copy that, right?</p>
<p>Thank you for your support!</p>
RuntimeError: index x is out of bounds for dimension 0 with size x in GCNCONV()

Alex_Carlo: <p>Hi everyone.  I am trying to work with GCNCONV() from pytorch-geometric with some artificial data produced by the following code for node classification.  Nodes can be of two class labels (0,1) and the length of the node feature is 100.While training the model, I am getting an error at the end of the code.<br>
Could you please help me to solve the problem?</p>
<pre><code class="lang-auto">import torch
from torch_geometric.data import Data
from torch_geometric.loader import DataLoader
import random
import torch.nn.functional as F
from torch_geometric.nn import GCNConv
from tqdm import tqdm
</code></pre>
<pre><code class="lang-auto">def createGraph():
    N_Nodes = random.randint(10,20)
    N_edges = random.randint(10,N_Nodes*(N_Nodes-1)/2)
    x = torch.tensor([[random.random() for i in range(100)] for j in range(N_Nodes)],dtype=torch.float)
    e = [(random.randint(0,N_Nodes),random.randint(0,N_Nodes)) for i in range(N_edges)]
    r = []
    c = []
    for i in e:
        r.append(i[0])
        c.append(i[1])
    edge_index = torch.tensor([r, c], dtype = torch.long)
    y = torch.tensor([[random.randint(0,1)] for i in range(N_Nodes)],dtype=torch.long)
    
    return x, edge_index, y
</code></pre>
<pre><code class="lang-auto">def createDataSet(n =1000):
    L =[]
    for i in range(n):
        
        x, edge_index, y = createGraph()
        
        L.append(Data(x=x,edge_index=edge_index,y=y))
    return L
DS = createDataSet(n = 1000)
trainLoader = DataLoader(DS,batch_size=32,shuffle=False,)
</code></pre>
<pre><code class="lang-auto">class GCN(torch.nn.Module):
    def __init__(self,num_node_features,num_classes):
        super().__init__()
        self.conv1 = GCNConv(num_node_features, 16)
        self.conv2 = GCNConv(16, num_classes)

    def forward(self, data):
        x, edge_index = data.x, data.edge_index

        x = self.conv1(x, edge_index)
        x = F.relu(x)
        x = F.dropout(x, training=self.training)
        x = self.c
        onv2(x, edge_index)

        return F.log_softmax(x, dim=1)
model = GCN(100,2)
optimizer = torch.optim.Adam(model.parameters(), lr=0.01, weight_decay=5e-4)
</code></pre>
<pre><code class="lang-auto">model.train()
for epoch in tqdm(range(50)):
    for data in trainLoader:
        optimizer.zero_grad()
        out = model(data)
        loss = F.nll_loss(out, data.y)
        loss.backward()
        optimizer.step()
</code></pre>
<pre><code class="lang-auto">1, in scatter_sum(src, index, dim, out, dim_size)
     19         size[dim] = int(index.max()) + 1
     20     out = torch.zeros(size, dtype=src.dtype, device=src.device)
---&gt; 21     return out.scatter_add_(dim, index, src)
     22 else:
     23     return out.scatter_add_(dim, index, src)

RuntimeError: index 15 is out of bounds for dimension 0 with size 15
</code></pre>
How to train multi-branch output network?

MinsooKim: <p>Hello, everyone!</p>
<p>I am working to build multi-branch network.<br>
Here is what I am working on.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/5/8550c512c607ba20636a78bc52128229d6bdc3b6.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/8550c512c607ba20636a78bc52128229d6bdc3b6" title="File (4)"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/5/8550c512c607ba20636a78bc52128229d6bdc3b6_2_690x324.jpeg" alt="File (4)" data-base62-sha1="j1myeikWJAT4g02bjvAgbKWuTps" width="690" height="324" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/5/8550c512c607ba20636a78bc52128229d6bdc3b6_2_690x324.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/8/5/8550c512c607ba20636a78bc52128229d6bdc3b6_2_1035x486.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/8/5/8550c512c607ba20636a78bc52128229d6bdc3b6.jpeg 2x" data-dominant-color="E3E2D2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">File (4)</span><span class="informations">1234×581 66.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Each data sample in the batch has its own class, so each sample goes into a decoder of the same class as itself.<br>
<strong>How to train this kind of network?</strong></p>
<p>Here is a training step in my current code.</p>
<pre><code class="lang-auto">params = list(Encoder.parameters()) + list(Decoder1.parameters()) \
                    + list(Decoder2.parameters()) + list(Decoder3.parameters())
optim = torch.optim.Adadelta(params, learning_rate)

encoder_output = Encoder(input)
optim.zero_grad()
loss_1, loss_2, loss_3 = 0, 0, 0
# split batch
if encoder_output[1_batch_ind].shape[0] != 0:
     output1 = Decoder1(encoder_output[1_batch_ind])
     loss_1 = loss_fn(output1, ground_truth[1_batch_ind])
                
if encoder_output[2_batch_ind].shape[0] != 0:
     output2 = Decoder2(encoder_output[2_batch_ind])
     loss_2 = loss_fn(output2, ground_truth[2_batch_ind])
                
if encoder_output[3_batch_ind].shape[0] != 0:
     output3 = Decoder3(encoder_output[3_batch_ind])
     loss_3 = loss_fn(output3, ground_truth[3_batch_ind])
loss = loss_1 + loss_2 + loss_3
loss.backward()
optim.step()             
                
</code></pre>
<p>What I have expected is each loss_# is backpropagated through each Decoder#, only and Encoder is trained by a total sum of loss, which means the backpropagation happens like<br>
loss_# → Decoder#,<br>
total_loss → Encoder.</p>
<p>Does the code above run as I have expected??</p>

ptrblck: <p>Yes, that should work as expected.<br>
To verify it, you could zero out some losses and check the gradients in each module separately.</p>
Save model when the hooks are used

Vishak_Raj: <p>Hello,</p>
<p>I created a simple model, and registered the hooks to get the intermediate layer outputs,</p>
<pre><code class="lang-auto">    activation2 = OrderedDict()
    def get_activation2(name):
        def hook2(model, input, output):
            activation2[name] = output.detach()
        return hook2

    for name, layer in model.named_modules():
        layer.register_forward_hook(get_activation2(name))
</code></pre>
<p>everthing is fine. When saving the model, using <code>torch.save(model, 'model.pth')</code>, I got this error -</p>
<blockquote>
<p>AttributeError: Can’t pickle local object ‘get_activation2..hook2’</p>
</blockquote>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/f/ff3f656149c5f046c77da915e0f46c257d1838f9.png" data-download-href="https://discuss.pytorch.org/uploads/default/ff3f656149c5f046c77da915e0f46c257d1838f9" title="Screenshot from 2022-06-16 17-24-30"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/f/ff3f656149c5f046c77da915e0f46c257d1838f9_2_690x160.png" alt="Screenshot from 2022-06-16 17-24-30" data-base62-sha1="Aq1zYGaRkqq7J6ARwaHQI2V9GhH" width="690" height="160" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/f/ff3f656149c5f046c77da915e0f46c257d1838f9_2_690x160.png, https://discuss.pytorch.org/uploads/default/original/3X/f/f/ff3f656149c5f046c77da915e0f46c257d1838f9.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/f/ff3f656149c5f046c77da915e0f46c257d1838f9.png 2x" data-dominant-color="F4E9EC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2022-06-16 17-24-30</span><span class="informations">879×204 30.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>How to solve this, thanks</p>

fralupo: <p>Same for me - any news?</p>
Could I freeze a model before loading pre-trained weights into it?

Hawk: <p>Hello, everyone!<br>
If I firstly freeze a model like this:</p>
<pre><code class="lang-auto">for param in model.parameters(recurse=True):
    param.requires_grad_(False)
</code></pre>
<p>and then load a set of pre-trained weights into this model, will this model remains frozen? In other words, will the loaded weights refresh the “requires_grad” flag of the model params?<br>
Thanks! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

Andrei_Cristea: <p>Checking this the old fashion way suggests the model remains frozen <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<pre><code class="lang-auto"># load a model and unfreeze its weights, for good measure
net = torch.hub.load('pytorch/vision:v0.10.0', 'resnet18', pretrained=True)
for param in net.parameters():
    param.requires_grad = True

# save the weights to disk
torch.save(net.state_dict(), "resnet18.pt")

# freeze the weights of the model
for param in net.parameters():
    param.requires_grad = False
    
# load the weights from disk into model
net.load_state_dict(torch.load("resnet18.pt"))

# count number of unfrozen weights
n_unfrozen = 0
for param in net.parameters():
    if param.requires_grad:
        n_unfrozen += 1
        
print(n_unfrozen)

Output:
0
</code></pre>
<p>EDIT: However, as a practical matter of code readability by a later you who may not remember the answer here for sure, it may be worth it just redundantly going through the weights after loading and asserting that they are frozen, just so it’s clear to the reader that they’re frozen.</p>
Find idx between two unique values tensors

huix_cheng: <p>Hi. As shown in the minimal example below, I would like to know how to find the index between two tensors of Bx3xN. Just like a[idx] == b.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/0/201848277aa923f3e5beeedf30496d630e512f05.png" data-download-href="https://discuss.pytorch.org/uploads/default/201848277aa923f3e5beeedf30496d630e512f05" title="QQ截图20220617202329"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/0/201848277aa923f3e5beeedf30496d630e512f05_2_667x500.png" alt="QQ截图20220617202329" data-base62-sha1="4zViLvSSFMQiFGEC7bCFf2WMxGR" width="667" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/0/201848277aa923f3e5beeedf30496d630e512f05_2_667x500.png, https://discuss.pytorch.org/uploads/default/original/3X/2/0/201848277aa923f3e5beeedf30496d630e512f05.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/0/201848277aa923f3e5beeedf30496d630e512f05.png 2x" data-dominant-color="303031"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">QQ截图20220617202329</span><span class="informations">834×625 44.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I tried to use the <a href="https://discuss.pytorch.org/t/for-every-element-in-one-tensor-find-the-index-in-another-tensor/59875">method</a> under that problem, but it suggests excessive memory requirements. Because I have a large N value, close to 50k</p>
Memcpy HtoD in forward is much more slower than DtoH in backward

zxgx: <p>I’m building a model which puts Embedding table in CPU while computes activations in GPU. So I move the embedding vectors to GPU in the forward pass, and as far as I know, the autograd engine would put the gradients w.r.t embedding vectors back to CPU. The transferred activations and gradients should be of the same size.</p>
<p>When I profiled the training process, I found that the Memcpy HtoD, which corresponds to moving activations to GPU in the forward pass, was much more faster than Memcpy DtoH (copying gradients back to CPU) in the backward pass.<br>
Here this is the screenshot of the tensorboard profile:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/4/7468aab23663dc9a9fc5b9181181c287672a6815.png" data-download-href="https://discuss.pytorch.org/uploads/default/7468aab23663dc9a9fc5b9181181c287672a6815" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/4/7468aab23663dc9a9fc5b9181181c287672a6815_2_690x308.png" alt="image" data-base62-sha1="gBNDjy31UAjhZpX20sopACwCOrP" width="690" height="308" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/4/7468aab23663dc9a9fc5b9181181c287672a6815_2_690x308.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/4/7468aab23663dc9a9fc5b9181181c287672a6815_2_1035x462.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/4/7468aab23663dc9a9fc5b9181181c287672a6815_2_1380x616.png 2x" data-dominant-color="DDE6E4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1881×840 77.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Does anyone have any idea why Memcpy DtoH was slower than HtoD? How can I avoid this problem and improve the speed of DtoH in the backward pass?</p>