akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Importing torch/script.h shows errors after building the project

krshbnv: <p>I am using VS2019 to create a C++ DLL project that will use libtorch. I want to create two functions:</p>
<ul>
<li>loadModel - that will load a .pt file</li>
<li>predictImage - that will evaluate the output for a given image<br>
I want to pack these two functions into a DLL for a further use in another programs.</li>
</ul>
<p>I have set up my DLL project as follows:</p>
<ul>
<li>I am using “libtorch-win-shared-with-deps-debug-1.6.0+cpu”</li>
<li>I only use one configuration - DEBUG CPU x64</li>
<li>In the property pages I have set-up the following changes:<br>
---- Configuration Properties &gt; General &gt; C++ Language standard : ISO C++ 14<br>
---- Configuration Properties &gt; C/C++ &gt; General &gt; Additional Include Libraries:<br>
------- Absolute path to libtorch\include<br>
------- Aboslute path to libtorch\include\torch\csrc\api\include<br>
---- Configuration Properties &gt; C/C++ &gt; General &gt; SDL cheks: No<br>
---- Configuration Properties &gt; C/C++ &gt; Language &gt; Conformance Mode: No<br>
---- Configuration Properties &gt; Linker &gt; General &gt; Additional Library Directories:<br>
------- Absolute path to libtorch\lib<br>
---- Configuration Properties &gt; Linker &gt; Input &gt; Additional Dependecies:<br>
------- torch_cpu.lib<br>
------- c10.lib<br>
------- torch.lib<br>
---- Configuration Properties &gt; Build Events &gt; Post-Build Event:<br>
------- xcopy Absolute path to libtorch\lib*.dll $(SolutionDir)$(Platform)$(Configuration)\ /c /y</li>
</ul>
<p>After setting this up, I can import the header files as normal, but building the project fails.<br>
Just after including torch/script.h and buidling the project, I get a bunch of errors and warnings.</p>
<p>Another thing I have noticed is, that the IntelliSense will find the imported modules, and give me suggestions. For example, it allowed me to create a Module object my typing:<br>
torch::jit::Module model;<br>
Notice that there is no “script” between “jit” and “Module”, and the IntelliSense would show error.</p>
<p>When using torch::jit::load, the IntelliSense would give me a suggestion for the load function, but as soon as I type it, it will show me an error that “jit has no method called load”.</p>
<p>Again I cannot see if the declaration of the model variable or loading of the model works, since I cannot build the solution.</p>
<p>I am doing something wrong with setting up the project? I appreciate all the help. Thanks!</p>
Calculating Jacobian using functorch

Vendrick17: <p>Hello, I am trying to calculate the Jacobian matrix of a GAN using <code>functorch</code>, so I need the derivative of each intermediate layer with respect to the input (z), I am using the generator in this <a href="https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html" rel="noopener nofollow ugc">tutorial</a>, for example, I need to calculate the derivative of the 5th layer of the generator with respect to <code>z</code>, to do this I am using the chain rule in the following loop:</p>
<pre><code class="lang-auto">grads = []
z = sample_batch(20) #Generates a random vector of size (20,100)
inp = z[:,:,None,None] #Reshaping to match the Convolution layer
index = 5
grad = 1

for i in range(index):
  jac_func = lambda x: G.main[i](x)
  if i != 0 :
    inp = Get_Hidden_Output(z, i-1)  #Gets the output of the i-1 layer when the input of the model is z
  grad = vmap(jac_func)(inp)*grad
  grads.append(grad)
</code></pre>
<p>However, while computing, I get an error when reaching to the BatchNorm layers of the model, so I would like to know if it is possible to calculate the such operation directly without using the chain rule, any help would be appreciated, just in case the error I get for the BatchNorm layer is:</p>
<pre><code class="lang-auto">ValueError: expected 4D input (got 3D input)
</code></pre>

clerkie: <p>Do you get the same error when you use a 4D input?</p>

Vendrick17: <p>No, I used <code>z.unsqueeze(1)</code> to avoid the error, but I am not convinced with the solution.</p>
Using same pytorch dataloader twice

Tamal_Chowdhury: <p>I have created one pytorch dataloader called <strong>“al_dataloader”</strong> for training my custom deep learning model. After the completion of model training I want to extract the intermediate feature map from the trained model on the same dataset that was used during training time. Can I use the same <strong>al_dataloader</strong> for the feature extraction phase also, or should I create a new dataloader object although it has already iterated entirely during model training?</p>
Avoid onnx:NonZero operation when filltering pytorch

Kracozebr: <p>I’m trying to convert my PyTorch model to ONNX and after that to TensorRT and found out that there is no NonZero op for TensorRT<br>
I found the part in my code that produces this operation and it is filtering tensors:</p>
<pre><code class="lang-auto">keep = (conf_scores &gt; conf_thresh)
sc = conf_scores[keep]
</code></pre>
<p>When I apply my mask keep (it contains 1 and 0) to my tensor this operation produces NonZero tensor when I execute <code> torch.onnx.export</code><br>
I tried to a little bit another code:</p>
<pre><code class="lang-auto">mask = conf_scores &gt; conf_thresh
indices = torch.nonzero(mask)
sc = conf_scores[indices]
</code></pre>
<p>The result is the same.<br>
How to filter tensor and not to produce NonZero operation when converting in onnx?</p>
<p>UPD: tried the other code:</p>
<pre><code class="lang-auto">mask = conf_scores &gt; conf_thresh
sc = torch.masked_select(conf_scores, mask)
</code></pre>
<p>and the other:</p>
<pre><code class="lang-auto">sc = conf_scores[conf_scores &gt; conf_thresh]
</code></pre>
<p>and again there is NonZero op in ONNX</p>

lk137095576: <p>I met the same question, have you solved it?</p>

Kracozebr: <p>No, whatever I do it always come with NonZero op.<br>
I had to convert pythorch model to tensorrt and such code was in postprocessing (nms and so on), so I decided to remove postprocess from tensorrt and write it on pytorch. If this is the case I advice you to do the same</p>

lk137095576: <p>Thanks, I also met it in nms, but I found the latest onnx-tensorrt add NonZero op(just this mounth, lucky <img src="https://discuss.pytorch.org/images/emoji/apple/grin.png?v=12" title=":grin:" class="emoji" alt=":grin:" loading="lazy" width="20" height="20">), but I haven’t try it, I met some questions when build, if it works, I will reply you.</p>

Kracozebr: <p>Yes, it’s a long time issue from 2020 (<a href="https://github.com/onnx/onnx-tensorrt/issues/401#issuecomment-1261247344" class="inline-onebox" rel="noopener nofollow ugc">No importer registered for op: NonZero · Issue #401 · onnx/onnx-tensorrt · GitHub</a>)<br>
If it works, let me know)</p>
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.LongTensor [256, 10, 128]] is at version 47; expected version 46 instead

winchest: <p>The full error message is down below:</p>
<pre><code class="lang-auto">File "F:\Anaconda3\envs\Mypycharm\lib\site-packages\torch\_tensor.py", line 396, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "F:\Anaconda3\envs\Mypycharm\lib\site-packages\torch\autograd\__init__.py", line 173, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.LongTensor [256, 10, 128]] is at version 47; expected version 46 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
</code></pre>
<p>my model’s forward function is down below:<br>
def forward(self, input, return_pi=False):</p>
<pre><code>    if self.checkpoint_encoder and self.training:  # Only checkpoint if we need gradients
        embeddings, _ = checkpoint(self.embedder, self._init_embed(input))
    else:
        embeddings, _ = self.embedder(self._init_embed(input))

    _log_p, pi, cost = self._inner(input, embeddings)

    init_lengths, mask = self.problem.get_costs(input, pi)
    init_lengths = init_lengths[:,None]
    final_lengths = cost + init_lengths

    ll = self._calc_log_likelihood(_log_p, pi, mask)
    if return_pi:
        return final_lengths.squeeze(), ll, pi

    return final_lengths.squeeze(), ll
</code></pre>
No such operator image::read_file on google colab

itorch: <p>I have created a NN to train a custom dataset. The code was running so far until today when I am seeing the error. Most solutions found online point to incompatibility issues and suggest uninstalling and reinstalling. However, I am working on google colab, and I am not sure what to do here.<br>
If this issue is less likely due to incompatibility than the class method I wrote, it should not have been working before today as it had been. But, I am confused why the incompatibility is suddenly an issue on the exact same script.</p>
<pre><code class="lang-auto">for class, value in enumerate(train_dataloader, 0):
    "STATEMENT"
</code></pre>
<p>Any operation on my train loader like the one above is causing the following error.</p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/torch/_ops.py in __getattr__(self, op_name)
    197         try:
--&gt; 198             op, overload_names = torch._C._jit_get_operation(qualified_op_name)
    199         except RuntimeError as e:

RuntimeError: No such operator image::read_file

The above exception was the direct cause of the following exception:

AttributeError                            Traceback (most recent call last)
9 frames
/usr/local/lib/python3.7/dist-packages/torch/_ops.py in __getattr__(self, op_name)
    200             # Turn this into AttributeError so getattr(obj, key, default)
    201             # works (this is called by TorchScript with __origin__)
--&gt; 202             raise AttributeError(f"'_OpNamespace' object has no attribute '{op_name}'") from e
    203 
    204         # let the script frontend know that op is identical to the builtin op

AttributeError: '_OpNamespace' object has no attribute 'read_file'
</code></pre>

clerkie: <p>Check the ops that are supported: <a href="https://pytorch.org/docs/stable/jit.html#supported-operators" class="inline-onebox" rel="noopener nofollow ugc">TorchScript — PyTorch 1.13 documentation</a></p>
Batch indexing to avoid the loop

hwang: <p>Hi, my post is closed, so I upload this again.<br>
I want to set values inside bounding boxes to zero.<br>
batch size is 256<br>
Image size is 32,32,3<br>
box components are [ymin,ymax,xmin,xmax]</p>
<p>My original code is below<br>
img=torch.randn((256,3,32,32))<br>
B=torch.rand((256,4))*32<br>
B=B.type(torch.int32)<br>
for i in range(256):<br>
img[i,:,B[i,0]:B[i,1],B[i,2]:B[i,3]]=0</p>
<p>Is there any way to avoid for-loop?<br>
Thanks</p>

KFrank: <p>Hi Hwang!</p>
<aside class="quote no-group quote-modified" data-username="hwang" data-post="1" data-topic="166111" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/h/ba8739/48.png" class="avatar"> hwang:</div>
<blockquote>
<p>I want to set values inside bounding boxes to zero.<br>
…<br>
Is there any way to avoid for-loop?</p>
</blockquote>
</aside>
<p>Yes, you can use your bounding boxes, <code>B</code>, to build a <code>mask</code> tensor that<br>
you then multiply onto <code>img</code> to zero out the values in the bounding boxes:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; print (torch.__version__)
1.13.0
&gt;&gt;&gt;
&gt;&gt;&gt; _ = torch.manual_seed (2022)
&gt;&gt;&gt;
&gt;&gt;&gt; nBatch = 256
&gt;&gt;&gt; nChannels = 3
&gt;&gt;&gt; h = 32
&gt;&gt;&gt; w = h
&gt;&gt;&gt;
&gt;&gt;&gt; img = torch.randn (nBatch, nChannels, h, w)
&gt;&gt;&gt; B = torch.rand (nBatch, 4) * h
&gt;&gt;&gt; B = B.type (torch.int32)
&gt;&gt;&gt;
&gt;&gt;&gt; imgB = img.clone()
&gt;&gt;&gt;
&gt;&gt;&gt; # loop version
&gt;&gt;&gt; for  i in range (nBatch):
...     img[i, :, B[i, 0]:B[i, 1], B[i, 2]:B[i, 3]] = 0
...
&gt;&gt;&gt; # loop-free mask version
&gt;&gt;&gt; hMask = torch.logical_or (torch.arange (h).unsqueeze (0) &lt; B[:, 0, None], torch.arange (h).unsqueeze (0) &gt;= B[:, 1, None])
&gt;&gt;&gt; wMask = torch.logical_or (torch.arange (w).unsqueeze (0) &lt; B[:, 2, None], torch.arange (w).unsqueeze (0) &gt;= B[:, 3, None])
&gt;&gt;&gt; mask = torch.logical_or (hMask.unsqueeze (2), wMask.unsqueeze (1))
&gt;&gt;&gt; imgB *= mask.unsqueeze (1)
&gt;&gt;&gt;
&gt;&gt;&gt; torch.equal (img, imgB)
True
</code></pre>
<p>Best.</p>
<p>K. Frank</p>
Uncaught exception of type NSException

Aryan_Philip: <p>I got this error below while training a VQGAN model using mps acceleration and it threw this error while running the training script</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/0/2084d786a29261a0833bbacba3b24f417f89e3ee.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/2084d786a29261a0833bbacba3b24f417f89e3ee" title="Screenshot 2022-11-18 at 12.54.26 AM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/0/2084d786a29261a0833bbacba3b24f417f89e3ee_2_690x379.jpeg" alt="Screenshot 2022-11-18 at 12.54.26 AM" data-base62-sha1="4DFTgXqFXE9LaXMDcmR4TYkZnZQ" width="690" height="379" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/0/2084d786a29261a0833bbacba3b24f417f89e3ee_2_690x379.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/2/0/2084d786a29261a0833bbacba3b24f417f89e3ee_2_1035x568.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/0/2084d786a29261a0833bbacba3b24f417f89e3ee_2_1380x758.jpeg 2x" data-dominant-color="37303A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2022-11-18 at 12.54.26 AM</span><span class="informations">1920×1057 215 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Difference between BCEWithLogits and CrossEntropy

Yoo: <p>I thought the following code would output the same result, but it did not.<br>
What is the difference between BCEWithLogits and CrossEntropy loss used for classification?</p>
<pre><code class="lang-auto">loss1 = nn.BCEWithLogitsLoss()
loss2 = nn.CrossEntropyLoss()

input = torch.randn(11,3, requires_grad=True)
target = torch.empty(11,3).random_(2)
output1 = loss1(input, target)
output2 = loss2(input, target)
</code></pre>

srishti-git1110: <p>Hi,<br>
<code>CrossEntropyLoss</code> is used in multi-class (single-label) problems where one and only one class out of the possible classes is the right answer.</p>
<p><code>BCEWithLogitsLoss</code> is used in multi-class multi-label problems where a data point could belong to one or more than one classes.</p>
Using torch data prallel. Invalid device string

spacekcz: <p>I’m trying to use the torch dataparallel class with specified GPU devices. When I set the torch.device as torch.device(“cuda:0,1”). Pytorch gives me the following error: RuntimeError: Invalid device string: ‘cuda:0,1’.</p>
<p>I have two GPU devices on my machine. Using Windows 10 OS.</p>
<p>I will look forward to any suggestions.</p>

ptrblck: <p><code>torch.device</code> does not accept multiple devices. Check the <a href="https://pytorch.org/docs/stable/generated/torch.nn.DataParallel.html#torch.nn.DataParallel">docs</a> and specify the GPU ids in the <code>DataParallel</code> initialization (or better: use <code>DistributedDataParallel</code> for better performance).</p>
Mapminmax function

John4: <p>Hello. I should translate the following matlab code to pytorch:</p>
<pre><code>var = mapminmax(X, 0, 1); 
</code></pre>
<p>I used:</p>
<pre><code>from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler(feature_range=(0, 1))
var = = scaler.fit_transform(X)
</code></pre>
<p>but the values I get in pytorch are quite different to that of matlab. Do you know if there is an other way to perform it?</p>

ptrblck: <p>From the MATLAB docs:</p>
<blockquote>
<p>Process matrices by mapping row minimum and maximum values to [<code>-1 1</code>]</p>
</blockquote>
<p>I guess your scikit-learn approach is normalizing the entire array, so you might need to normalize the rows separately.<br>
The question is also unrelated to PyTorch so you might get a faster and better answer on e.g. StackOverflow.</p>
Shared memory leak while using multiprocessing

remiliani93: <p>Hi I have set the torch multiprocessing strategy to <code>file_system</code> for a multiprocessing DataLoader. After a certain number of epochs (not the same every time I run) I get one of the workers fails because of insufficient shared memory. While monitoring <code>/dev/shm</code> I notice an increase at the end of each epoch, hence there are some tensors that are not freed. Is there a known issue about this? Or a way to identify where the memory leak could be happening?</p>

robotrapta: <p>I haven’t noticed a problem with leaks during a run.  But I definitely see my machine leaking <code>/dev/shm</code> between runs.  Hundreds of gigabytes accumulate after a while and clog up the machine.</p>
<p><code>rm /dev/shm/*</code></p>
<p>will flush it out, but only do this when nothing is running because it will probably mess up jobs-in-progress.</p>
Deduce input shape from torch.jit.ScriptModule

dshin: <p>The <code>torch.jit.trace()</code> python function accepts a <code>nn.Module</code> and an example input, and outputs a <code>torch.jit.ScriptModule</code>.</p>
<p>Is there a way to extract out the expected input shape from the <code>torch.jit.ScriptModule</code>? In other words, if the <code>trace()</code> call was passed as its second argument a Tensor of shape <code>(1, 2, 3, 4)</code>, is there a way to recover <code>(_, 2, 3, 4)</code> from the <code>torch.jit.ScriptModule</code> outputted by <code>trace()</code>?</p>

davidberard98: <p>If you do something like this</p>
<pre><code class="lang-python">import torch

def fn(x):
    return x.relu()

x = torch.rand((4, 4))
fn_t = torch.jit.trace(fn, (x,))
print(fn_t.graph)
</code></pre>
<p>Then I believe the printed graph will show shape info. You can also programmatically access the input types, see some minimal examples here: <a href="https://github.com/pytorch/pytorch/blob/master/test/jit/test_python_bindings.py" class="inline-onebox" rel="noopener nofollow ugc">pytorch/test_python_bindings.py at master · pytorch/pytorch · GitHub</a></p>
<p>However, <strong>this is an implementation detail</strong>. The shapes shown in the graph are just an artifact of the tracing process which could change in the future.</p>

dshin: <p>Thanks. If this is the best way to get the input shape in python, I am guessing that the c++ <code>torch::jit::script</code> module won’t offer much better options. In my context, I was hoping to save the python <code>torch.jit.ScriptModule</code> to disk, load it in c++, and infer the input shape at c++ runtime.</p>
How to flatten parameters of LSTM layers with spectral normed weights

erogol: <p>Looks like directly running <code>layer.flatten_parameters()</code> does not work for LSTM layers if the weights are passed over the spectral norm.</p>
<p>I’ve tried first removing the spectal norm then flattening but it also didn’t work.</p>
<pre><code class="lang-auto">self.bilstm = nn.LSTM(in_dim, lstm_channels, 1,
                                    batch_first=True, bidirectional=True)

self.bilstm = spectral_norm(self.bilstm, 'weight_hh_l0')
self.bilstm = spectral_norm(self.bilstm, 'weight_hh_l0_reverse')

...

self.bilstm = remove_spectral_norm(self.bilstm, name="weight_hh_l0")
self.bilstm = remove_spectral_norm(self.bilstm, name="weight_hh_l0_reverse")
self.bilstm.flatten_parameters()

...

self.bilstm.flatten_parameters()
context = self.bilstm(context)[0]
</code></pre>
<p>I still receive the warning.</p>
<pre><code class="lang-auto">UserWarning: RNN module weights are not part of single contiguous chunk of memory. This means they need to be compacted at every call, possibly greatly increasing memory usage. To compact weights again call flatten_parameters(). (Triggered internally at /opt/conda/conda-bld/pytorch_1666642975993/work/aten/src/ATen/native/cudnn/RNN.cpp:968.)
</code></pre>
[PyTorch1.13] [CUDA 11.7] Could not run 'aten::empty_strided' with arguments from the 'CUDA' backend

benlag: <p>Hi All,</p>
<p>I have a issu to execute an inférence on torch under Cuda 11.7.<br>
Previously  I use PyTorch with cuda 10.2 on GPU on Turing and Pascal Nvidia Card.<br>
To use PyTorch endvironnment in Ampere génération, I must install Cuda 11.7</p>
<p>To Load the model :<br>
torch:jit:Load('yolo.pt")</p>
<p>with the error : Could not run ‘aten::empty_strided’ with arguments from the ‘CUDA’ backend.</p>
<p>Could you please give a piece of advise ?</p>
<p>3080 TI<br>
Visual Studio 2017<br>
Cuda 11.7<br>
Pytorch 1.13</p>

ptrblck: <p>Could you post a minimal and executable code snippet to reproduce the issue, please?</p>

benlag: <p>Hi All,</p>
<p>I have the following error with the current code</p>
<p>It’s located on load of my classifier…</p>
<p>Thanks in adavance.</p>
<p>Ben<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/0/6019e5cbbe8e79de9fb40e4b1aeef54de3ef91fd.png" data-download-href="https://discuss.pytorch.org/uploads/default/6019e5cbbe8e79de9fb40e4b1aeef54de3ef91fd" title="code"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/0/6019e5cbbe8e79de9fb40e4b1aeef54de3ef91fd.png" alt="code" data-base62-sha1="dI9k04TZc6C2lFtV0zzEyRPa5tz" width="533" height="500" data-dominant-color="4C4C4C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">code</span><span class="informations">1154×1081 54.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Is there an implementation of edit distance to calculate wer?

josipovic: <p>Is there edit_distance function in pytorch to calculate wer/ler? Its levenshtein distance. There is a pip package <a href="https://pypi.org/project/torch-edit-distance/" rel="nofollow noopener">https://pypi.org/project/torch-edit-distance/</a> but its a bit buggy. If there is none in pytorch, why?</p>
<p>cheers</p>

30stomercury: <p>Hi, why it is buggy? It works fine for me.</p>

kinwai_cheuk: <p>It does’t work when <code>device=cuda:1</code> or above. The example code assume that we use only <code>cuda:0</code> (<code>.cuda()</code>), which is not true in general especially when you are training your models in a server.</p>

ShunLu: <p>I used cuda10.2 and successfully installed this package. However, I still met this problem though I tried cuda:0(.cuda()).<br>
Have you solved this problem and could you please give me some help? <img src="https://discuss.pytorch.org/images/emoji/apple/smiling_face_with_tear.png?v=12" title=":smiling_face_with_tear:" class="emoji" alt=":smiling_face_with_tear:" loading="lazy" width="20" height="20"></p>
PyTorch Android documentation

vjayd: <p>Where can I find PyTorch Android API documentation?</p>

cccclai: <p><a href="https://pytorch.org/mobile/android/" class="inline-onebox" rel="noopener nofollow ugc">Android | PyTorch</a> and <a href="https://pytorch.org/javadoc/1.9.0/" class="inline-onebox" rel="noopener nofollow ugc">pytorch_android 1.9.0 API</a> can be good reference.</p>

cerebruminfotech: <p>Check out Mobile Performance Recipes, which explain how to improve your model and determine whether improvements were successful using benchmarking.<br>
To understand how to create native Android applications using PyTorch prebuilt libraries, see this recipe.<br>
need any help in project contact a <a href="https://www.cerebruminfotech.com" rel="noopener nofollow ugc">mobile app development company</a>,<br>
The Javadoc contains more information about the PyTorch Android API <a href="https://pytorch.org/javadoc/1.9.0/" rel="noopener nofollow ugc">click here</a>.</p>

cerebruminfotech: <p>PyTorch Android prebuilt libraries from gradle dependencies (which is aar files). PyTorch supports an end-to-end workflow from Python to deployment on iOS and Android.It support mobile platform.It helps you accelerate machine learning research and development with a highly optimized implementation on mobile devices.<br>
contact me if u want any kind of help:<a href="https://www.cerebruminfotech.com/" rel="noopener nofollow ugc">Development Company</a></p>
Why Anchors generate in FCOS?

ali_tavakolpoor: <p>Hi all teams of Pytorch Community.<br>
I have a enquiry regarding FCOS (Fully Convolutional One-Stage Object Detection) code in PyTorch. I have checked FCOS Pytorch subcode in Github, the author uses “Anchorgenerator” like Retinanet and other anchor-based detectors. while the FCOS net is the Anchor-free detector according to this paper.<br>
do anyone know that why the programmer uses Anchor-generator in this code?</p>
Examples for training with JIT

quornmd: <p>TorchScript seems to be focused on low latency inference. Can it also be used for model training? If so, are there some examples out there, e.g. training a ResNet on MNIST?</p>
<p>Recently, I tried JAX to train a simple MLP model. I saw 100x speed-up through JIT, but I do prefer pytorch in general.</p>
Can't saving model for inference using torch.save

Tan_Vang_Ismael: <p>So I can not saving model after training. Can anyone help me? Thank<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/6/9611c32d8545e365d8f51b9cafcb9ffb8ba7c791.png" data-download-href="https://discuss.pytorch.org/uploads/default/9611c32d8545e365d8f51b9cafcb9ffb8ba7c791" title="model"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/6/9611c32d8545e365d8f51b9cafcb9ffb8ba7c791_2_690x426.png" alt="model" data-base62-sha1="lpzFVW6swStDH8s0XuwGly05lCN" width="690" height="426" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/6/9611c32d8545e365d8f51b9cafcb9ffb8ba7c791_2_690x426.png, https://discuss.pytorch.org/uploads/default/optimized/3X/9/6/9611c32d8545e365d8f51b9cafcb9ffb8ba7c791_2_1035x639.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/9/6/9611c32d8545e365d8f51b9cafcb9ffb8ba7c791_2_1380x852.png 2x" data-dominant-color="F4E5E5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">model</span><span class="informations">1459×902 60.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>I’m not familiar enough with <code>torch2trt</code> so would recommend to either use <a href="https://github.com/pytorch/TensorRT">TorchTRT</a> or create an issue in the <code>torch2trt</code> GitHub repository so that the code owners could take a look.</p>
Torch.load(); meaning map_location=lambda storage, loc: storage

Jihun_Yoon: <p>I understand that an usage of <code>map_location</code> is to make sure a model to be loaded on the device where it was trained. And I fount that <code>map_location=labmda storage, loc: storage</code> is a way of load model on CPU. But what’s the meaning of this lambda function? What are storage and loc here? How does it work?</p>

shivam13juna: <p>For Posterity, <a href="https://pytorch.org/docs/stable/generated/torch.load.html" class="inline-onebox" rel="noopener nofollow ugc">torch.load — PyTorch 1.13 documentation</a></p>
Loading a pre-trained model on CPU while had been trained on multiple GPUS

hamedB: <p>Hi</p>
<p>I load a model and pass it into CPU. Some modules of this model have been wrapped by data parallel on 4 GPUs during the training phase. Now, in the test phase when I pass both model and input into the CPU and when I feed the model with the input, I get the following error.</p>
<p>How can I work with a model in CPU when it has been trained on 4 GPUs during the training phase?</p>
<p>Thanks</p>
<blockquote>
<p>RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cpu</p>
</blockquote>

ptrblck: <p>You could store the <code>state_dict</code> of the internal <code>.module</code> as described <a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-torch-nn-dataparallel-models">here</a> and then use <code>map_location</code> to load it onto the CPU as described <a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html#save-on-gpu-load-on-cpu">here</a>.</p>

hamedB: <p>Thank you <a class="mention" href="/u/ptrblck">@ptrblck</a>  for your reply. Is there any way to load on CPU for the model already trained? because the link you have shared mean that I should run my program again to be trained. Am I right?</p>
Tensor.copy_ to some index do not work

tsp_t: <p>You want to copy_ directly into the index of the dst tensor. In particular, since the dtypes are different, it is not possible to directly map them. (Y[…] = X)<br>
So I used copy, but the copy to the indexed (Y[:, index]) doesn’t work as expected.<br>
Can anyone confirm if I’m using it wrong?</p>
<pre><code class="lang-auto">import torch

X = torch.randn([2, 360, 5000], device="cuda", dtype = torch.float32)
Y = torch.zeros([64, 2, 360, 5000], device="cuda", dtype = torch.float16)
index = torch.randint(0, 2, [2, 360, 5000], device="cuda") == 1
Y[:, index].copy_(X[None, index], non_blocking=True)
print(torch.allclose(Y[0, index].float(), X[index])) # False
</code></pre>
Trying to understand significance of same convolution in CenterNet heatmap generation

Debojyoti_Biswas: <p>Hello altruists,<br>
I am new to this domain and trying to understand a block of model. I am looking into the code for a paper named  “CenterNet: Objects as Points”. Let’s say we have feature from the backbone and the shape of feature = [12, 256, 100, 100]. Now, this feature is fed into the below block:</p>
<pre><code class="lang-auto">bbox_tower= Sequential(
        Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        GroupNorm(32, 256, eps=1e-05, affine=True)
        (ReLU()
        Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        GroupNorm(32, 256, eps=1e-05, affine=True)
         ReLU()
         Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
         GroupNorm(32, 256, eps=1e-05, affine=True)
        ReLU()
        Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        (GroupNorm(32, 256, eps=1e-05, affine=True)
        ReLU()
      )

</code></pre>
<p>So, in this block we are doing same convolution 4 times with Group Normalization. I am trying to understand what we are expecting after doing the same convolutions on the feature.</p>
<p>Then the output from the <strong>bbox_tower</strong> is passed to the <strong>below block</strong>:</p>
<pre><code class="lang-auto">agn_hm= Conv2d(256, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))

</code></pre>
<p>I am assuming in the above block class agnostic heatmap is generated from the <strong>bbox_tower</strong> output. Am I right?</p>
<p>Could anyone please give me a brief idea about the working mechanism of the above code snippet and guide me how exactly we are achieving class agnostic heatmap from the above two module.</p>
<p>Thanks in advance!!!</p>
Ray.get(.) is too slow

periklis17: <p>I’m working in the federated learning setting, so I want to parallelize my code using ray, with respect to the users. So, when I use ray.get(.) , where inside get(.) I call each user, I noticed that the training is extremely slower that running the same code sequentially.<br>
I use 3 gpus.<br>
Any ideas??<br>
Thanks!</p>
PyTorch vs using cuDNN (in C++)

rinkujadhav2013: <p>I’m trying use cuDNN directly in C++ (for various reasons). My code works just fine as in it compiles, loss goes down, accuracy goes on MNIST as expected but it is awfully slower than using PyTorch in Python. PyTorch is somehow WAY faster than directly using cuDNN in C++ even when I load the whole MNIST dataset into GPU to avoid host&lt;–&gt;memory transfers and set convolution type to <code>CUDNN_TENSOR_OP_MATH_ALLOW_CONVERSION</code>.<br>
My question is what extra stuff is PyTorch doing that makes it so much faster? Is it a a whole bunch of small stuff that adds up to a gigantic speed up or do the 80% gains come from some small 20% things? Is there a place where I can look for some of these things?</p>

ptrblck: <p>You can take a look at the <a href="https://github.com/pytorch/pytorch/tree/61801799a0a6a2fe0b577450c1fdd55af6063664/aten/src/ATen/native/cudnn">cudnn bindings</a> and in particular into <a href="https://github.com/pytorch/pytorch/blob/61801799a0a6a2fe0b577450c1fdd55af6063664/aten/src/ATen/native/cudnn/Conv_v7.cpp"><code>Conv_v7.cpp</code></a> for the legacy API and <a href="https://github.com/pytorch/pytorch/blob/61801799a0a6a2fe0b577450c1fdd55af6063664/aten/src/ATen/native/cudnn/Conv_v8.cpp"><code>Conv_v8.cpp</code></a> for the new frontend API usage.</p>

rinkujadhav2013: <p>Thanks for the link! They both look very different and <code>Conv_v7.cpp</code> seems to have more stuff going on in it.<br>
Anyway, the main thing I got out of <code>Conv_v7.cpp</code> is that PyTorch is doing at least these 2 things:</p>
<ol>
<li>Some kind of 32bit split across the batch dimension. I didn’t get what it is but since I’m working with giga-pixeled images (and hence my own cuDNN code), my batch size is one so I guess I can ignore it.</li>
<li>Instead of setting <code>cudnnSetConvolutionMathType</code> to something and then calling <code>cudnnFindConvolutionForwardAlgorithm</code>, PyTorch tries all possible MathType with <code>cudnnFindConvolutionForwardAlgorithmEx</code> (and not <code>cudnnFindConvolutionForwardAlgorithm</code>) to find the fastest combination.<br>
Is that correct? And did I miss something that’s very important for performance?</li>
</ol>

ptrblck: <ol>
<li>
<p>This is used to work around the 32bit limitation of cuDNN for large inputs and we are splitting the workload in the batch size if possible.</p>
</li>
<li>
<p>cudnnFind is used when <code>torch.backends.cudnn.benchmark = True</code> is used.</p>
</li>
</ol>
Straight Through Estimator like

Daniel_Richards: <p>Hello,<br>
I am trying to implement the Straight through Estimator method mentioned in this paper: <a href="https://arxiv.org/pdf/2209.04836.pdf" rel="noopener nofollow ugc">https://arxiv.org/pdf/2209.04836.pdf</a>. In simple words, there are two models of the same architecture and I am trying to build a third model that is the average of model1 and the non-differentiable operation of model2 (so I would average corresponding weight matrices). However when I back pass I would like to do it only on model2.</p>
<p>I tried the following:<br>
model3 = type(model1)()<br>
for p1, p2_orig, p2, p3 in zip(model1.parameters(), model2.parameters() non_differentiable_func(model2).parameters(), model3.parameters()):<br>
ps.copy_(p1) = 0.5*(p1.requires_grad_(False) + p2.requires_grad_(False) +p2_orig - p2_orig.requires_grad_(False)</p>
<p>However, this throws an error stating that values can’t be modified in leaf tensors or something like that, but as far as I understood I can’t modify the contents as such.</p>
<p>Any help on this matter would be appreciated</p>
How can I reduce the dimension of 3d tensor to 2d using convolutional filters?

juanitapuentes: <p>I have one question about the dimension of a tensor</p>
<ol>
<li>How do I convert my model output torch.Size([2560, 128, 128]) to torch.Size([2560,128]) using convolutions?</li>
</ol>

ptrblck: <p>You could use an <code>nn.Conv1d</code> layer with a kernel size of <code>128</code> and squeeze the output tensor in <code>dim2</code> afterwards:</p>
<pre><code class="lang-python">conv = nn.Conv1d(in_channels=128, out_channels=128, kernel_size=128)
x = torch.randn(2560, 128, 128)

out = conv(x)
out = out.squeeze(2)
print(out.shape)
# torch.Size([2560, 128])
</code></pre>
Libtorch include files/library generation

christianperone: <p>When I get the <code>https://download.pytorch.org/libtorch/nightly/cpu/libtorch-shared-with-deps-latest.zip</code> file from the server I can see that there is all the include headers and libraries. How is this package generated ? Is there a script to generate the contents of this package ? I’m asking this because the <code>build_libtoch.py</code> alone doesn’t copy the include headers into the build folder, what am I missing ? Thank you !</p>
<p>To clarify: what I need is to generate the includes + share (cmake files) folder after building libtorch, I guess a script does that in CI but I wasn’t able to find it.</p>

ptrblck: <p>The <a href="https://github.com/pytorch/builder"><code>pytorch/builder</code></a> repository should contain all build scripts. I don’t know where the packaging steps are executed right now, so let me know if you get stuck and I can take another look at it.</p>

christianperone: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a>, Sam Estep helped me with it, the solution was to use makefile install targets, it created a very similar directory tree (w/ libs, includes and the cmake files required to find libs) to the libtorch one that is distributed in the website.</p>
How to update DataLoader in between epochs?

agneet: <p>So, I am trying out curriculum learning, where the idea is to update the dataloader, after a certain number of epochs.<br>
My current code looks like this -</p>
<pre><code class="lang-auto">for epoch in range(config.max_epochs):  
    print(f"Epoch: {epoch}")
    progress = tqdm.tqdm(total=train_batch_num,
                             ncols=75,
                             desc=f'Train {epoch}')
    model.train()
    total_train_accuracy=0
    total_train_loss=0
   ''' UPDATE TRAIN DATALOADER HERE'''
    for batch in train_dataloader:
      batch = {k:v.to(device) for k,v in batch.items()}
      optimizer.zero_grad()
      outputs = model(**batch)
</code></pre>
<p>I want to update the train_dataloader, as mentioned above.<br>
Can I use something like <code>train_dataloader = DataLoader(train_dataset, collate_fn=collate_fn, batch_size=config.batch_size, shuffle=True)</code> to update the train_dataloader after a given number of epochs? I am currently not using multiple workers for the dataloader however this will be trained on a GPU. My doubts are : Will this update be reflected on the data loader during training?</p>

nivek: <p>Based on your code snippet, it should work. As long as you don’t modify the <code>DataLoader</code> <strong>while</strong> it is yielding batches (and expect batches to change halfway through), it should be fine.</p>
Add a category to a pretrained model for classification in object detection

ilanit: <p>Hi all,</p>
<p>I ran the transfer learning on my datasets, achieved n classes on the top of the network, but my purpose is simply <strong>expanding</strong> the categories of a pretrained model (such as resnet18 which trained on ImageNet and can predict thousand categories) and add a new category. The advantage of this approach is avoiding a situation in which the input is X1 (categorized by the pretrained categories) and my network mistakenly predict one of my trained classes (of the n classes that I trained).</p>

Junwu_Weng: <p>You just need to simply remove the the fc layer and replace it with a Linear(512,n+1) layer. Then the thing you need to do is to use your own dataset to finetune the new model.</p>

ilanit: <p>This is exactly what I DON’T want to do because I want to leave the 1000 categories as they are and simply add another category and retrain the whole net. So the number of classes would be 1001 after this process.</p>

Killer_Glass: <p>Hello, I have the same problem, I want to add a new class to a pre-trained model from yolov7. Did you happen to find a solution? If yes, could you share?</p>
Can I load *.ptl model file in Python?

SungmanHong: <p>I converted the pytorch model(<em>.pt) to pytorch mobile (</em>.ptl)</p>
<p>but I want to load *.ptl file and run on python.</p>
<p>It is possible?</p>
<p>thx.</p>

yakupakkaya: <p>Did you achieve to find any solution?</p>
Delete nodes (or layers) in MLP

Hayden_Johnson: <p>I have trained a simple MLP with an input layer, one hidden layer, and an output layer of 18 nodes. I would like to be able to selectively delete 17 output nodes and their associated weights/biases, leaving just one output node. I am seeking either a way to completely remove 17 output nodes, or a possible alternative of deleting the final layer and replacing it with a 1 node output layer corresponding to the 1 node that would have been leftover after “pruning”.</p>
<p>Below is my model definition:<br>
class NMR_Model(nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
self.lin1 = nn.Linear(14000, 200)<br>
self.relu1 = nn.ReLU()<br>
self.lin2 = nn.Linear(200, 18)<br>
def forward(self, input):<br>
return (self.lin2(self.relu1(self.lin1(input))))</p>
<p>I have been able to change 17 out of the18 weights and biases to 0 using the following code, but for my purposes I need them fully removed:<br>
p = prune.L1Unstructured(amount=17/18)<br>
pruned = p.prune(model2.lin2.bias)<br>
pruned2 = p.prune(model2.lin2.weight)<br>
model2.lin2.bias = torch.nn.Parameter(pruned)<br>
model2.lin2.weight = torch.nn.Parameter(pruned2)</p>

Hayden_Johnson: <p>I think I may be on the right track with deleting the final layer and appending a new layer of only 1 node. Below is the code I used to delete the final layer, as well as to define the weights and bias I want to append to the end of the MLP:</p>
<p>newmodel = torch.nn.Sequential(*(list(model.children())[:-1]))<br>
w1 = model.lin2.weight[0]<br>
b1 = model.lin2.bias[0]</p>
<p>Now if I can determine how to add the “w1” and “b1” tensors to the end of “newmodel” then I believe that will resolve this issue.</p>

Hayden_Johnson: <p>I believe I have found a solution by defining a second model with 1 output node, and then specifying the parameters from the first trained model:</p>
<p>class NMR_Model2(nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
self.lin1 = nn.Linear(14000, 200)<br>
self.relu1 = nn.ReLU()<br>
self.lin2 = nn.Linear(200, 1)<br>
def forward(self, input):<br>
return (self.lin2(self.relu1(self.lin1(input))))</p>
<p>model2 = NMR_Model2()<br>
model2.lin1.weight.data = model.lin1.weight.data<br>
model2.lin1.bias.data = model.lin1.bias.data<br>
model2.lin2.weight.data = model.lin2.weight.data[0]<br>
model2.lin2.bias.data = model.lin2.bias.data[0]</p>
Model parameters are becoming nan after some batch backpropagation

Vaijenath_Biradar: <p>my model contains lstm with batch. model is so simple, But after running some batches not even completed one epoch the model parameters are becoming nan. I tried with learning rate as 0.0 even though they are becoming zero. may i know why? My code is at <a href="https://bitbucket.org/Vsbiradar/siamese-network/src/b5d6428c045cdcc1900fa216bece08d0012dab9b/siamese-network/siamese.py?at=master&amp;fileviewer=file-view-default" rel="nofollow noopener">https://bitbucket.org/Vsbiradar/siamese-network/src/b5d6428c045cdcc1900fa216bece08d0012dab9b/siamese-network/siamese.py?at=master&amp;fileviewer=file-view-default</a></p>

SpandanMadan: <p>One obvious possible reason is because your training rate is too large and gradients blow up.</p>
<p>Look at the gradients of your model layers. Also, try gradient clipping.</p>

Vaijenath_Biradar: <p>But my learning rate is 0.And algorithm is SGD. I dont know why the model parameters are even changing with 0 learning rate</p>

QuantScientist: <p>Did you try using a different optimizer? Adam for instance?</p>

Vaijenath_Biradar: <p>Yes. Adam and Adadelta. any of these are not working</p>

Antlar-lu: <p>Did you solve it ? And which version do you apply?</p>

Wenyao_Zhang: <p>Did you solve it? I have the same problem. Can you give me some advice?</p>