akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Can NLP augmentation affect the performance of certain categories of data?

theorem_on: <p>There are some few different kinds of subcategories in my NLP data.<br>
My NLP Model performance is not that high because some subcategories.<br>
I expect to increasing model performance via increasing subcategories data by NLP augmentation. Does it make sense?<br>
Just worried about thing is even if distribution of subcategories hasn’t unbalanced, NLP augmentation could be helpful?<br>
Is Increasing the subcategories data good solution ignoring whole data balance?</p>

vdw: <p>Data augmentation can be a suitable way to address imbalanced datasets. However, the question is: How exactly would you do this? I would argue that this for from trivial form NLP-related task.</p>
<p>In case of images, you can: rotate, crop, squeeze, change color depth, decrease/increase hue, etc. Assuming your inputs are sentences, how would you “tweak” this sentences to get additional and meaningful input sentences?</p>

theorem_on: <p>Thank you for repling.<br>
I’m trying to apply ContextualWordEmbsAug(word insert or substitute) as word augmenter based on BERT using nlpaug <code>https://github.com/makcedward/nlpaug</code></p>

vdw: <p>Thanks for the link. Looking at the example from their Github page kind of confirms my concerns:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/c/cc79cca76c92d2ff2a1c06e227f53a9103e13a8e.png" data-download-href="https://discuss.pytorch.org/uploads/default/cc79cca76c92d2ff2a1c06e227f53a9103e13a8e" title="textual_example"><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/c/cc79cca76c92d2ff2a1c06e227f53a9103e13a8e.png" alt="textual_example" data-base62-sha1="taSmckr5cNG3Wm7cUuY1ymzcsJE" width="690" height="182" data-dominant-color="E7E5E5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">textual_example</span><span class="informations">966×256 14.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Data augmentation for sentences is not easy when the goal is the get semantically meaningful outputs. Even reasonably straightforward synonym replacement is not foolproof. In the table above, 2nd row, it works well enough. But for example, let’s say the original sentence is “the horse jumps over the fence” and the augmented output is “the horse climbs over the fence”. It doesn’t really seem semantically meaningful. Not sure if I would want to train a language model based on this.</p>
<p>That being said, those are more my own gut feelings. I assume the authors have quantitative result to show the benefits of their approaches.</p>
Installation always got killed

consh: <p>Hello everyone,</p>
<p>I was trying using the <a href="http://pytorch.org" rel="noopener nofollow ugc">pytorch.org</a> default command below to install:</p>
<blockquote>
<p>pip3 install torch torchvision torchaudio</p>
</blockquote>
<p>The download process into ubuntu machine started, but then after reaching the end downloading of data byte…the process got killed.</p>
<p>Anyone can help me to resolve this issue?</p>

ptrblck: <p>Check if the OS killed the process if it was running OOM. <code>dmesg</code> should confirm it and show a corresponding log in such a case.</p>

consh: <p>Thanks for your reply.</p>
<p>This issue is resolved by choosing the proper compute platform provided on <a href="http://pytorch.org" rel="noopener nofollow ugc">pytorch.org</a> homepage which is by adding <code>--index-url</code> option</p>
Multi-threaded, Single GPU libtorch usage

jxtps: <p>I’m running multi-threaded inference using libtorch using something like (pseudocode for simplicity):</p>
<pre><code class="lang-auto">in_tensor = in_tensor.to(cuda, dtype, /*non_blocking=*/ false, /*copy=*/ false, memory_format); 
semaphore.acquire(); 
out_tensor = model.forward(in_tensor); // actually wrapped in ivalue, eliding that
cuda::synchronize(); // Does this also sync any thread-parallel data transfers?
semaphore.release(); 
out_tensor = out_tensor.to(cpu, dtype, /*non_blocking=*/ false, /*copy=*/ false, memory_format); 
</code></pre>
<p>So multiple threads may be calling <code>in_tensor.to(cuda...)</code> / <code>out_tensor.to(cpu...)</code> in parallel, but only a single thread is calling <code>model.forward()</code>.</p>
<p>I want to ensure that my data transfers CPU ↔ GPU do indeed run in parallel with the <code>model.forward()</code>-call and don’t interfere with the computations on the GPU, but I’m unclear on the precise meaning of <code>non_blocking=false</code> - it blocks the calling thread (great!), but does it <em>also</em> block the cuda stream, thereby potentially interrupting any ongoing <code>model.forward()</code>?</p>
<p>And when I call <code>cuda::synchronize()</code> to ensure <code>model.forward()</code> has actually completed (it returns right away, i.e. non-blocking), does that then wait for any ongoing tensor transfer to complete? That would be very undesirable.</p>

jxtps: <p>I did some performance testing of various permutations, and here are the results:</p>
<pre><code class="lang-auto">to - sem.acq - forward - from - sem.release: 2,474 images processed
to - sem.acq - forward - sem.release - from: 2,491 images processed
sem.acq - to - forward - from - sem.release: 2,484 images processed
to.nonblock - sem.acq - forward - from - sem.release: 2,404 images processed
to.nonblock - forward - from: 2,271 images processed
</code></pre>
<p>(to = CPU → GPU, from = GPU → CPU)</p>
<p>So there doesn’t seem to be any real difference between the first three, and the latter two see a degradation in performance.</p>
<p>That tells me that there’s no successful parallelization between data transfer and kernel computation, and that using <code>non_blocking=true</code> or skipping synchronization leads to some contention that causes delays.</p>
<p><strong>What is the recommended recipe for doing data transfer in parallel with kernel computation?</strong></p>
<p>CC: <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
Attempting to draw dynamo diagrams as a noobee

davidradl: <p>hi ,<br>
I am attempting to draw dynamo diagrams as a noobee, as I understand things easier if I can see a picture.</p>
<p>Here are 2 attempts.</p>
<ol>
<li>
<p>this is some uml around nodes. Is this correct ? I am not sure about the 1 to 1 nature of the nodes. Any suggestions?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/4/f44cbe81c204f62f87b5654e1ced40c6b812a4a2.png" data-download-href="https://discuss.pytorch.org/uploads/default/f44cbe81c204f62f87b5654e1ced40c6b812a4a2" title="nodes.drawio"><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/4/f44cbe81c204f62f87b5654e1ced40c6b812a4a2.png" alt="nodes.drawio" data-base62-sha1="yRaVr7rCX6aEbHsfAeMNPemOdl8" width="539" height="500" data-dominant-color="FBFBFB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">nodes.drawio</span><span class="informations">584×541 6.08 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
</li>
<li>
<p>I am trying to piece together the big animal picture of the compiler, mentioning some key code classes / files.<br>
I would love to get feedback from you all - so I can make this more accurate (I am sure it has errors and big omissions - but it is a start!! ) and put it somewhere useful in the docs / readme.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/0/00d35d8b811ec39c8382c9bd4d75864f76867efc.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/00d35d8b811ec39c8382c9bd4d75864f76867efc" title="dynamo.drawio"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/0/00d35d8b811ec39c8382c9bd4d75864f76867efc_2_690x203.jpeg" alt="dynamo.drawio" data-base62-sha1="7iQASaQXx7Czim0E90ZWFeu3JW" width="690" height="203" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/0/00d35d8b811ec39c8382c9bd4d75864f76867efc_2_690x203.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/0/0/00d35d8b811ec39c8382c9bd4d75864f76867efc_2_1035x304.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/0/0/00d35d8b811ec39c8382c9bd4d75864f76867efc_2_1380x406.jpeg 2x" data-dominant-color="FAF8EF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">dynamo.drawio</span><span class="informations">2361×696 140 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
</li>
</ol>
Visualising Neural Nets, simple implementation

Mah_Neh: <p>I’m trying to implement <a href="https://arxiv.org/pdf/1311.2901.pdf" rel="noopener nofollow ugc">part of the idea in the paper</a></p>
<p>What I find with this approach is that I have to implement each deconvolution manually, and I don’t really see how to automate this more.</p>
<p>When I run the deconvolution, it flows back to the image, which now only looks pixelated.</p>
<p>This is the current network I am using</p>
<pre><code class="lang-auto">import torch
from torch import nn


class Backbone(nn.Module):

    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 16, kernel_size=2, stride=1, padding=2)
        self.batch1 = nn.BatchNorm2d(16)
        self.maxpool1 = nn.MaxPool2d(kernel_size=(3, 3), return_indices=True)

        self.conv2 = nn.Conv2d(16, 32, kernel_size=2, stride=1, padding=2)
        self.batch2 = nn.BatchNorm2d(32)

        self.conv3 = nn.Conv2d(32, 64, kernel_size=2, stride=1, padding=2)
        self.batch3 = nn.BatchNorm2d(64)

        self.conv4 = nn.Conv2d(64, 128, kernel_size=2, stride=1, padding=2)
        self.batch4 = nn.BatchNorm2d(128)
        self.maxpool = nn.MaxPool2d(kernel_size=(3, 3))

        self.conv5 = nn.Conv2d(128, 256, kernel_size=2, stride=1, padding=2)
        self.batch5 = nn.BatchNorm2d(256)
        self.relu = nn.ReLU()

        self.maxupool = nn.MaxUnpool2d(kernel_size=(3, 3))

    def deconv1(
        self,
        image_tensor: torch.Tensor,
    ) -&gt; torch.Tensor:
        activations = {}
        kernels = {}
        indices = {}
        sizes = {}
        name = "maxpool1"
        name1 = "conv1"

        def get_activation_and_indices(name):
            def hook(model, input, output):
                activations[name] = output[0].detach()
                indices[name] = output[1].detach()
                # print(input[0].size())
                sizes[name] = input[0].size()

            return hook

        self.maxpool1.register_forward_hook(get_activation_and_indices(name))
        kernels[name1] = self.conv1.weight.detach()

        self.forward(image_tensor)
        x = self.maxupool(activations[name], indices[name], output_size=sizes[name])
        x = self.relu(x)
        x = nn.functional.conv_transpose2d(x, kernels[name1], stride=1, padding=2)
        x = x.mul(255.0).clamp(0.0, 255.0)
        return x

    def forward(self, x: torch.Tensor) -&gt; torch.Tensor:
        x = self.conv1(x)
        x = self.batch1(x)
        x = self.relu(x)
        x, _ = self.maxpool1(x)
        x = self.conv2(x)
        x = self.batch2(x)
        x = self.relu(x)
        x = self.conv3(x)
        x = self.batch3(x)
        x = self.relu(x)
        x = self.maxpool(x)
        x = self.conv4(x)
        x = self.batch4(x)
        x = self.relu(x)
        x = self.conv5(x)
        x = self.batch5(x)
        x = self.relu(x)
        return x

</code></pre>
<p>Is there any better way to frame this problem of deconvolution so that it becomes more automatic ?</p>
How to use MobileNet_V3 with `depth multiplier`

palguna_gopireddy: <p>The m<a href="https://arxiv.org/abs/1905.02244" rel="noopener nofollow ugc">obileNetV3 paper </a> has <code>depth multiplier</code> parameter, that reduces the number of parameters. How to use this in <code>pytorch</code> implementation of <a href="https://pytorch.org/vision/main/models/generated/torchvision.models.mobilenet_v3_large.html#torchvision.models.mobilenet_v3_large" rel="noopener nofollow ugc">MobileNet_V3_large</a>. I see, no parameters with name <code>depth multiplier</code> or something is mentioned.</p>
<p>Also using <code>different resolutions</code> means just passing the desired resolution image to the model or is there also a parameter need to be mentioned for this?</p>
<p>Just mentioning the table the paper has mentioned with different <code>multipliers</code> and <code>resolutions</code><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/f/1fd8c4e15dbb94204584410f88b7dd8cc58865c9.png" data-download-href="https://discuss.pytorch.org/uploads/default/1fd8c4e15dbb94204584410f88b7dd8cc58865c9" title="Capture"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/f/1fd8c4e15dbb94204584410f88b7dd8cc58865c9_2_439x500.png" alt="Capture" data-base62-sha1="4xJe5cSJPdWPvRzbuwIdClq2MS5" width="439" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/f/1fd8c4e15dbb94204584410f88b7dd8cc58865c9_2_439x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/f/1fd8c4e15dbb94204584410f88b7dd8cc58865c9_2_658x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/f/1fd8c4e15dbb94204584410f88b7dd8cc58865c9.png 2x" data-dominant-color="ECECEC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture</span><span class="informations">693×788 113 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Quantized weights of transformer

Ghadeer_Jaradat: <p>I am trying to print the quantized weights of the BERT transformer,</p>
<pre><code class="lang-auto">&gt; model_name='yoshitomo-matsubara/bert-base-uncased-sst2'
&gt; model = AutoModelForSequenceClassification.from_pretrained(model_name)
&gt; 
&gt; 
&gt; #--Quantize the model to INT8(Fixed point)(pytorch quantization)
&gt; model_dynamic_quantized = torch.quantization.quantize_dynamic(model,
&gt;                                                               qconfig_spec={torch.nn.Linear},
&gt;                                                               dtype=torch.qint8 )
&gt; for v in  model_dynamic_quantized.state_dict().values():
&gt;     print(v.int_repr())
</code></pre>
<p>No matter what I try, keep getting this error</p>
<blockquote>
<p><strong>NotImplementedError</strong>: Could not run ‘aten::int_repr’ with arguments from the ‘CPU’ backend. This could be because the operator doesn’t exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit <a href="https://fburl.com/ptmfixes" class="inline-onebox" rel="noopener nofollow ugc">Internal Login</a> for possible resolutions. ‘aten::int_repr’ is only available for these backends: [QuantizedCPU, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradHIP, AutogradXLA, AutogradMPS, AutogradIPU, AutogradXPU, AutogradHPU, AutogradVE, AutogradLazy, AutogradMeta, AutogradMTIA, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, AutogradNestedTensor, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PythonDispatcher].</p>
</blockquote>
How to speed up training with h5 dataset and shuffle=True?

gt_tugsuu: <p>Hello, I have 200k (7x256x256) images that are loaded into seven h5 file.<br>
When I train with shuffle=True it takes 4 hours / epoch, and without shuffling 30 min / epoch.</p>
<p>The problem is queering from h5 dataset with large index value is slow. And without shuffling the model overfits spectacularly.</p>
<p>Does anyone know how to speed up shuffle=True training with h5 dataset?</p>
Should model.eval() be used when doing inference?

Mah_Neh: <p>I’m loading a model and running inference,</p>
<pre><code class="lang-auto">model = torch.load("model_140.pt")
</code></pre>
<p>Or should it rather be</p>
<pre><code class="lang-auto">
model = torch.load("model_140.pt").to(device)
model.eval()
with torch.no_grad():
  model(image)
</code></pre>
<p>Currently I use <code>eval()</code> because of some dropout and batchnorm layers, and no grad because it was stated elsewhere that this would not create intermediate tensors meant for the backpropagation step.</p>
<p>Is the second approach best or it does not matter when loading a  saved model ?</p>

Ayush_Singhal: <p><code>model.eval()</code> simply tells the model to get into evaluation phase. Means certain layers will now not function the way they does in <code>training</code>. like <code>Layer Normalization/Dropout</code></p>
<p><code>torch.no_grad()</code> simply tells not to take the track of <code>gradients</code> of <code>layers/weights</code></p>
<p>So as of I know, it will be a good option to use both in combination as it will not only be a fair path but <code>torch.no_grad()</code> will also help in reducing <code>CPU/cuda</code> memory usages</p>
Autocast not consistent across different GPUs (A100 and RTX A6000)

Ruhollah_Majdoddin: <p>issue <a href="https://github.com/pytorch/pytorch/issues/108627" rel="noopener nofollow ugc">#108627</a></p>
<h3><a name="describe-the-bug-1" class="anchor" href="#describe-the-bug-1"></a><img src="https://discuss.pytorch.org/images/emoji/apple/bug.png?v=12" title=":bug:" class="emoji" alt=":bug:" loading="lazy" width="20" height="20"> Describe the bug</h3>
<p>I train and inference a classifier using autocast. Result is different accross diffenent GPUs (same .venv, code and data).<br>
The result on A100 is much superior than on RTX A6000.<br>
Not using autocast with <code>ctx = nullcontext()</code> on RTX A6000, gets the similar result to A100 with autocast.<br>
I get no torch warnings on either machine.</p>
<pre><code class="lang-python">ctx = torch.amp.autocast(device_type='cuda', dtype=torch.bfloat16)

#training
with ctx:
    logits, loss = classifier(X, Y) 
#inference
with ctx:
    logits, loss = classifier(X, None)   
</code></pre>
<p>P.S. This has cost me one month of my business time.</p>

ptrblck: <p>Is there a reason you are cross-posting the same issue directly?</p>

Ruhollah_Majdoddin: <p>Just to increase the chances to get a solution. Is it discouraged?</p>

ptrblck: <p>Yes, since multiple users could debug the same issue thus wasting time.</p>

Ruhollah_Majdoddin: <p>OK, now I know, thank you. You can delete the topic please.</p>
Earthquake Forecasting Post-doc in ML at the USGS (remote)

Morgan_Page: <p>The U.S. Geological Survey invites applications to a Mendenhall research post-doctoral fellowship opportunity to apply machine learning techniques to earthquake forecasting. The goal of this opportunity is to develop machine-learning approaches to improve earthquake forecasting capabilities – namely, to better predict the rate, locations, and sizes of future earthquakes.</p>
<p>The confluence of newly available, modern high-resolution seismic catalogs – composed of millions of earthquake locations, times, and sizes – along with recent advances in supervised and unsupervised machine learning, particularly deep learning techniques, provide a unique opportunity to mine seismic catalogs for deeper levels of earthquake predictability than have previously been found. In their proposal, applicants should describe methods and architectures amenable to extracting relevant features from catalogs of earthquake times and locations, and to generating stochastic representations of future activity.</p>
<p>This post-doctoral opportunity is for recent graduates in computer science, geophysics, seismology, statistics, or related fields. The fellowship provides 2 years of salary and benefits, as well as funding to support the proposed research project.</p>
<p>More information on the opportunity (Mendenhall <span class="hashtag-raw">#22-36</span>) is here: <a href="https://tinyurl.com/2h4xtp7c" rel="noopener nofollow ugc">https://tinyurl.com/2h4xtp7c</a></p>
<p>Candidates are encouraged to contact the research advisors to discuss project ideas. Further information on Mendenhall fellowship qualifications is here: <a href="https://on.doi.gov/3edWmm5" rel="noopener nofollow ugc">https://on.doi.gov/3edWmm5</a></p>
<p>Applications are due November 1, 2023. This is a remote job opportunity.</p>
Running functional Conv2D in parallel

Aviv_Shamsian: <p>Hi All,<br>
We try to run multiple functional Conv2D layers in parallel. For example, we have tensor of weights <code>weights = torch.rand(10,12,3,5,5)</code> where each axis represents <code>(number_of_conv2d_layers, out_channels, in_channels, kernel_H, kernel_W)</code>. In addition, we have a input <code>x = torch.rand(20,3,28,28)</code> where each axis represents <code>(batch, channels, H, W)</code>. We want to apply each entry of <code>weights</code> on <code>x</code>. We were able to implement it using for loop in the following manner:</p>
<pre><code class="lang-auto">class ForConv2d(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, x, weights):
        new_x = []
        for w in weights:
            new_x.append(torch.nn.functional.conv2d(input=x, weight=w))
        return torch.stack(new_x)
</code></pre>
<p>We are sure there is a more natural way for this implementation, avoiding the for loop (maybe Conv3D).</p>
<p>Thanks in advance.</p>

ptrblck: <p>You should be able to rewrite your approach into a grouped convolution and use a single call.</p>
Malware dataset malimg

Neurox: <p>Hello everyone,</p>
<p>I’m currently in the process of learning about malware classification using CNNs, and I’ve been making good progress. As we all understand, datasets are a vital component of deep learning. However, I’ve encountered two issues in my journey. I’ve downloaded the “malimg” dataset from Kaggle, but there’s a recurring problem: many individuals upload this dataset, and it gets removed fairly quickly. I’m aware that the original paper associated with this specific dataset is titled “Malware Images: Visualization and Automatic Classification.” Unfortunately, all the links to the “original” dataset are outdated and no longer working. Consequently, I’m facing difficulties in obtaining the original dataset. Does anyone have any suggestions on how to access it or any information regarding why this dataset is no longer accessible to the public?</p>
<p>You might be wondering why I’m interested in the original dataset. Well, I conducted a manual review of the 25 classes within this dataset, and I observed something suspicious about the last class, “Yuner.A.” All the images in this class appear identical, which is quite intriguing. To investigate this further, I developed a small program (which takes some time to run as it checks for identical images). Surprisingly, apart from the file size, all 800 .png files in that class are indeed pixelwise identical.</p>
<p>Does anyone have an explanation for either of these two issues?</p>
torch.onnx.errors.UnsupportedOperatorError: Exporting the operator 'aten::randperm' to ONNX opset version 15 is not supported

zazhil: <p>Hello,</p>
<p>I am trying to use Pytorch FasterRCNN model with ORTModule. However I get the following error when trying to wrap my model:</p>
<h1><a name="torchonnxerrorsunsupportedoperatorerror-exporting-the-operator-atenrandperm-to-onnx-opset-version-15-is-not-supported-1" class="anchor" href="#torchonnxerrorsunsupportedoperatorerror-exporting-the-operator-atenrandperm-to-onnx-opset-version-15-is-not-supported-1"></a>[torch.onnx.errors.UnsupportedOperatorError: Exporting the operator ‘aten::randperm’ to ONNX opset version 15 is not supported]</h1>
<p>I tried adding the function myself with the following code:</p>
<p><span class="mention">@_beartype.beartype</span><br>
def aten_randperm(g, self, *args):<br>
output=g.op(“aten::randperm”, self, *args)<br>
output.setType(self.type())<br>
return output</p>
<p>I registered this operation by using:</p>
<p>register_custom_op_symbolic(‘::randperm’, symbolic_opset14.aten_randperm, 14)</p>
<p>The error dissapeared and then a similar error related to aten::smooth_l1_loss arised. I did similar procedure to add it and moved forward.</p>
<p>Now I am getting the following error:</p>
<p>onnxruntime.training.ortmodule._fallback_exceptions.ORTModuleONNXModelException: There was an error while exporting the PyTorch model to ONNX:<br>
File “/home/op/anaconda3/envs/py310ort/lib/python3.10/site-packages/torch/onnx/symbolic_opset9.py”, line 2244, in _<em>or</em> raise errors.SymbolicValueError( torch.onnx.errors.SymbolicValueError: ONNX export does NOT support exporting bitwise OR for non-boolean input values [Caused by the value ‘4292 defined in (%4292 : Byte(*, strides=[1], requires_grad=0, device=cuda:0) = onnx::ScatterND(%4273, %4277, %4291), scope: onnxruntime.training.ortmodule._io._FlattenedModule::/torchvision.models.detection.faster_rcnn.FasterRCNN::_original_module/torchvision.models.detection.roi_heads.RoIHeads::roi_heads # [/home/op/anaconda3/envs/py310ort/lib/python3.10/site-packages/torchvision/models/detection/_utils.py:65:0]’ (type ‘Tensor’) in the TorchScript graph. The containing node has kind ‘onnx::ScatterND’.] (node defined in <a>/home/op/anaconda3/envs/py310ort/lib/python3.10/site-packages/torchvision/models/detection/_utils.py</a>: <strong>call</strong></p>
<p>Currently I have not being able to work with any of the object detection models (FasterRCNN, Fcos) or Deeplab for Semantic Segmentation inside the ORTModule.</p>
<p>Could you please help me figure out what could possibly be causing this problem? Is it maybe my implementation for the custom operators?</p>
<p>I am currently using Pyhon 3.10, onnx 1.14.1, torch_ort 1.15.0 and Pytorch 2.0.1+cu117</p>
<p>Thanks,</p>
<p>Zazhil-ha</p>
How to get the next element in a nn.ModuleList?

ADONAI_TZEVAOT: <p>how to get the next element in nn.ModuleList in an iteration? Please help</p>
<pre><code class="lang-auto">class A(nn.Module):
    def __init__(self,func):
        super().__init__()
        self.func = nn.ModuleList(func)
      

    def forward(self, x):
        for fun in self.func:
            # get the current and next func 
     return
</code></pre>

ptrblck: <p>You can use <code>zip</code>:</p>
<pre><code class="lang-python">class A(nn.Module):
    def __init__(self):
        super().__init__()
        self.func = nn.ModuleList([
            nn.Linear(1, 1),
            nn.ReLU()
        ])
      

    def forward(self, x):
        for a, b in zip(self.func[:-1], self.func[1:]):
            print(a, b)
    
        return x

model = A()
model(torch.randn(1, 1))
# Linear(in_features=1, out_features=1, bias=True) ReLU()
</code></pre>
How to get the class name as a string?

ADONAI_TZEVAOT: <pre><code class="lang-auto">func1 = [class_A(dim=2, parity=i%2) for i in range(2)]

func2 = [class_B( 2, 2, 1, 1)]

funcs = list(itertools.chain(*zip(func1, func2)))
</code></pre>
<hr>
<pre><code class="lang-auto">class Main_model(nn.Module):
    def __init__(self, funcs):   # funcs is a list of all the classes with the init parameters passed 
        super().__init__()
        self.funcs = nn.ModuleList(funcs)
      

    def forward(self, x):
        .......
        for func in self.funcs:
             ....  #print the name of the func alone as a string

        return
</code></pre>
<p>How to get the names of the classes as a string ?<br>
Please help</p>

ptrblck: <p>The <code>.__class__.__name__</code> attributes should work:</p>
<pre><code class="lang-python">class ClassA(nn.Module):
    def __init__(self):
        super().__init__()


class ClassB(nn.Module):
    def __init__(self):
        super().__init__()
        

class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.a = ClassA()
        self.b = ClassB()
        
model = MyModel()
print(model.__class__.__name__)
# MyModel
print(model.a.__class__.__name__)
# ClassA
print(model.b.__class__.__name__)
# ClassB
</code></pre>
How to store and access multiple values in a single pt file?

ADONAI_TZEVAOT: <p>How do I store 7 tensors and load whichever tensor i need? For example, I should be able to load the 5th tensor if I want to.</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/adonai_tzevaot">@ADONAI_TZEVAOT</a>,</p>
<p>You can use <code>torch.save</code> (docs here: <a href="https://pytorch.org/docs/stable/generated/torch.save.html" class="inline-onebox" rel="noopener nofollow ugc">torch.save — PyTorch 2.0 documentation</a>) and just pass all your tensors within a <code>dict</code> object and serialize the dictionary, for example,</p>
<pre><code class="lang-auto">torch.save({'tensor1':tensor1, 'tensor2':tensor2}, filename)
</code></pre>

ADONAI_TZEVAOT: <p>How can I append values to the file at every iteration ?</p>
Visualise CNNs: how to `unbatchnormalise`

Mah_Neh: <p>I’m in the process of reproducing the seminal paper <a href="https://arxiv.org/pdf/1311.2901.pdf" rel="noopener nofollow ugc">Visualising and Understanding Neural Networks</a> (a bit old, 2013) mostly to visualise my own networks and improve the training.</p>
<p>Now, the paper basically requires this:</p>
<ol>
<li>Have an max unpool layer (which will be approximated in torch.)</li>
<li>then a Relu</li>
<li>Then a convolution transposed which is also available in torch</li>
</ol>
<p>The code currently looks like:</p>
<pre><code class="lang-auto">    def deconv1(self, x: torch.Tensor) -&gt; torch.Tensor:
        x = self.maxupool(x)
        x = self.relu(x)
        x = self.batch1(x) // &lt;-----------probably incorrect
        x = self.uconv1(x)
       // maybe scale it up here ?
        return x
</code></pre>
<p>My only issue is, that there isn’t an obvious layer to undo the batch normalisation.</p>
<p>Do you have any idea if there is any built-in way to perform such operation ?</p>
<p>One way, I am somewhat weary, would be to actually use batch as it is currently done in the code and finally scale up the images manually by <code>*255</code> it, with something like this:<br>
<code>x = x.mul(255.0).clamp(0.0, 255.0)</code></p>
Paper recreation - is it possible to get one dimensional output value from LSTM if hidden_size>1 and no Linear layers are used?

The_Snail: <p>Hi,<br>
I am trying to replicate a paper (<a href="https://ieeexplore.ieee.org/document/7885096" class="inline-onebox" rel="noopener nofollow ugc">Deep Learning for Household Load Forecasting—A Novel Pooling Deep RNN | IEEE Journals &amp; Magazine | IEEE Xplore</a>) and in the paper the authors use LSTM with input of size <code>[batch_size, seq_len, input_size]</code> to get the output of <code>[batch_size, 1]</code> - one value for each input sequence in the batch. I think the standard way to do this is to take the last hidden state and input it to Linear layer to map to this one single value per each batch example, however the authors are not mentioning anything about using a fully connected layer on top of LSTM or how they achieved the output specifically. Is it possible to get this kind of output from LSTM without using fully connected layer on top of it?</p>
<p>The output of LSTM will be of shape <code>[batch_size, seq_len, hidden_size]</code> so you could take the last time step from seq_len, but then you will still have the hidden_size dimension, so it won’t give <code>[batch_size, 1]</code> output. Is there some other way or did the authors just omitted part of the solution?</p>
Torch.onnx.export`s export_params creates separated weight files

111114: <p>Hello<br>
When I tried to export torch model to onnx with export_params=True, parameter files for each module in my model saved separately in folder I assigned(./onnx/). It there any way to pack all weights in asr.onnx file?</p>
<p>Thank you</p>
<pre><code class="lang-auto">torch.onnx.export(
    model, 
    (torch.randn(3, 80, 355), torch.ones(3, 355)), 
    "./onnx/asr.onnx", 
    export_params=True,
    opset_version=14,
    input_names=["input_features", "feature_mask"],
    output_names=["logits"],
    dynamic_axes={
        "input_features" : {0 : "batch_size", 2 : "sequence_length"},
        "feature_mask" : {0 : "batch_size", 1 : "sequence_length"},
        "logits" : {0 : "batch_size", 1 : "half_sequence_length"}
    }
)
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/9/39daa03aba0d2641142e66b22cf84733c877e437.png" data-download-href="https://discuss.pytorch.org/uploads/default/39daa03aba0d2641142e66b22cf84733c877e437" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/9/39daa03aba0d2641142e66b22cf84733c877e437.png" alt="image" data-base62-sha1="8fNCpT1lBx7WZkcdrkwKxVnaOhh" width="162" height="500" data-dominant-color="303030"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">236×726 3.84 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

QJ-Chen: <p>Same problem with torch.<strong>version</strong> ‘2.0.1+cu117’</p>

111114: <p>Hello, When I inputs torch jit traced model to torch.onnx.export, all weights packed in asr.onnx file.</p>
<pre><code class="lang-auto">traced_model = torch.jit.trace(model, [dummy_input_features, dummy_feature_mask])
torch.onnx.export(
    traced_model, 
    (torch.randn(3, 80, 355, dtype=torch.float16, device="cuda:0"), torch.ones(3, 355, dtype=torch.float16, device="cuda:0")), 
    "./onnx/asr.onnx", 
    export_params=True,
    opset_version=14,
    input_names=["input_features", "feature_mask"],
    output_names=["logits"],
    dynamic_axes={
        "input_features" : {0 : "batch_size", 2 : "sequence_length"},
        "feature_mask" : {0 : "batch_size", 1 : "sequence_length"},
        "logits" : {0 : "batch_size", 1 : "half_sequence_length"}
    }
)
</code></pre>

QJ-Chen: <p>Jit traced model not works for me. Putting the model to cuda and changing the dtype to float16 (half) works for me as you also did. Maybe that’s the solution.</p>
CUDNN_STATUS_EXECUTION_FAILED from torch.float16

MartinZhang: <p>Today, I want to use multiple graphics cards(<code>cuda:0, cuda:1</code>) with DP to train the model of torch.float16</p>
<p>But, there is a error about this , as follow:</p>
<pre><code class="lang-auto">
RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED
You can try to repro this exception using the following code snippet. If that doesn't trigger the error, please include your original repro script when reporting this issue.

import torch
torch.backends.cuda.matmul.allow_tf32 = True
torch.backends.cudnn.benchmark = True
torch.backends.cudnn.deterministic = False
torch.backends.cudnn.allow_tf32 = True
data = torch.randn([4, 32, 119, 159], dtype=torch.half, device='cuda', requires_grad=True)
net = torch.nn.Conv2d(32, 64, kernel_size=[5, 5], padding=[0, 0], stride=[2, 2], dilation=[1, 1], groups=1)
net = net.cuda().half()
out = net(data)
out.backward(torch.randn_like(out))
torch.cuda.synchronize()

ConvolutionParams 
    data_type = CUDNN_DATA_HALF
    padding = [0, 0, 0]
    stride = [2, 2, 0]
    dilation = [1, 1, 0]
    groups = 1
    deterministic = false
    allow_tf32 = true
input: TensorDescriptor 000001E834DE5180
    type = CUDNN_DATA_HALF
    nbDims = 4
    dimA = 4, 32, 119, 159, 
    strideA = 605472, 18921, 159, 1, 
output: TensorDescriptor 000001E834DE3AC0
    type = CUDNN_DATA_HALF
    nbDims = 4
    dimA = 4, 64, 58, 78, 
    strideA = 289536, 4524, 78, 1, 
weight: FilterDescriptor 000001E8349A6610
    type = CUDNN_DATA_HALF
    tensor_format = CUDNN_TENSOR_NCHW
    nbDims = 4
    dimA = 64, 32, 5, 5, 
Pointer addresses: 
    input: 0000002363108000
    output: 00000023637A8800
    weight: 0000002305E01600
Additional pointer addresses: 
    grad_output: 00000023637A8800
    grad_input: 0000002363108000
Backward data algorithm: 1

</code></pre>
<p>when I only use (cuda:0) this error does not occur ！！！</p>
<p>Why?</p>
<p>So, how to use multiple graphics cards to train float16 model?</p>
Odd behavior when trying to get activation

Goldname: <p>I created a transformer model with one encoder layer:</p>
<pre><code class="lang-auto">class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.encoder_layer = nn.TransformerEncoderLayer(d_model=2, nhead=1, dim_feedforward=2)
        self.trans = nn.TransformerEncoder(self.encoder_layer, num_layers=1)
    def forward(self, x):
        y = self.trans(x)
        return y

model = MyModel()
model.trans.layers[0].norm1.register_forward_hook(get_activation('x'))

model.eval()
x = torch.tensor([[1., 0]])
print(model(x))
</code></pre>
<p>Getting the activation of the norm succeeds. However, when I follow up with this line:</p>
<pre><code class="lang-auto">model.trans.layers[0].self_attn.register_forward_hook(get_activation('x'))
print(model(x))
</code></pre>
<p>I encounter this error: AttributeError: ‘tuple’ object has no attribute ‘detach’</p>
<p>And then when I followup with this line again:</p>
<pre><code class="lang-auto">model.trans.layers[0].norm1.register_forward_hook(get_activation('x'))
print(model(x))
</code></pre>
<p>I get the same error. So in the first iteration when I get the norm1 activations, all is fine. Then I try to get the multihead attention activation, it fails. And then when I try to go back adn get the norm1 activations, it fails even though it worked at first. Why does the multihead attention and 2nd time I get norm1 result in this error?</p>
<blockquote>
<p>AttributeError: ‘tuple’ object has no attribute ‘detach’</p>
</blockquote>

ptrblck: <p>Once a hook is registered it stays registered for the module and you would need to call <code>handle.remove()</code> to remove it.<br>
I guess you are trying to call into three hooks in your latest run where one of them fails.</p>

Goldname: <p>Oh got it thanks. Do you have any idea why I am seeing this error though? I want to look at the activations of the self attention.</p>

ptrblck: <p>Some modules return <code>tuple</code>s and your hook is most likely calling <code>.detach()</code> on it directly as it expects a tensor. You could add some logic which checks the <code>type</code> of the input and either calls <code>.detach()</code> on a tensor or unpacks the <code>tuple</code> before calling <code>.detach()</code> on all tensors (or some of them; depending what you want).</p>
Android Memory Leak

km04: <p>Hello,</p>
<p>I’ve converted my Android app from using tensorflow to pytorch and recently ran into a bug with memory growth. Below are screenshots from the android memory profiler. As you can see the Native memory grows when calling module.forward:</p>
<p>outputTensor = module.forward(IValue.from(inputTensor)).toTensor();</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/4/d43c6bef20fac0eb40c751423254164da4aa0767.png" data-download-href="https://discuss.pytorch.org/uploads/default/d43c6bef20fac0eb40c751423254164da4aa0767" title="withModuleForward"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/4/d43c6bef20fac0eb40c751423254164da4aa0767_2_690x177.png" alt="withModuleForward" data-base62-sha1="uhwFWhSIyInJfIHlvfPJRY365FB" width="690" height="177" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/4/d43c6bef20fac0eb40c751423254164da4aa0767_2_690x177.png, https://discuss.pytorch.org/uploads/default/optimized/3X/d/4/d43c6bef20fac0eb40c751423254164da4aa0767_2_1035x265.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/d/4/d43c6bef20fac0eb40c751423254164da4aa0767_2_1380x354.png 2x" data-dominant-color="4D5D5B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">withModuleForward</span><span class="informations">2041×526 92.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>As I am a new user I cannot post the second picture but when commenting out this line, memory usage is constant.</p>
<p>Currently I use:<br>
implementation ‘org.pytorch:pytorch_android:1.13.0’<br>
implementation ‘org.pytorch:pytorch_android_torchvision:1.13.0’</p>
<p>I have tried w/ Vulkan backend, without, using the NNAPI and using both the lite and regular libraries. All of them appear to have this same issue.</p>
<p>Just wondering if anyone else has experienced something similar recently.</p>
Efficiently computing per-example gradient

mahdi_hgh: <p>I have a code for efficiently computing per-example gradient.</p>
<pre><code class="lang-auto">  def compute_loss(self, params, buffers, sample, target):
      batch = sample.unsqueeze(0)
      targets = target.unsqueeze(0)
      predictions = functional_call(self.model, (params, buffers), (batch,))
      loss = self.criterion(predictions, targets)
      return loss

  def estimate_grad_incoh(self):
    num_trainset = len(self.train_dataset)
    subsample_size = 2000
    subsample_idx = np.random.choice(a=range(num_trainset), size=subsample_size,
                             replace=False)
    data_subsampled = data_utils.Subset(train_dataset, subsample_idx)
    params = {k: v.detach() for k, v in self.model.named_parameters()}
    buffers = {k: v.detach() for k, v in self.model.named_buffers()}
    with torch.no_grad():
      ft_compute_grad = grad(self.compute_loss)
    ft_compute_sample_grad = vmap(ft_compute_grad, in_dims=(None, None, 0, 0))
    data_loader = torch.utils.data.DataLoader(data_subsampled, batch_size=subsample_size)
    for x, t in data_loader:
      x, t = x.to(device), t.to(device)
      if self.model_type == 'mlp':
          x = x.reshape(-1, self.input_size)
      ft_per_sample_grads = ft_compute_sample_grad(params, buffers, x, t)
</code></pre>
<p>However, when I wrap optimizer and model using privacy engine I got the following error</p>
<p><code>AttributeError: 'Tensor' object has no attribute '_forward_counter'</code></p>
<p>It is interesting that without private engine I do not get any error.</p>
<p>Any idea?</p>
Could gpu memory reserved by torch be set or increasedincreased

serieslover: <p>I have encountered memory problem as below.<br>
<em>torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 1024.00 MiB (GPU 0; 6.00 GiB total capacity; 3.60 GiB already allocated; 69.44 MiB free; 3.66 GiB reserved in total by PyTorch</em></p>
<p>how could i increase the reserved memory. any help is appreciated</p>

ptrblck: <p>You won’t be able to allocate 1GB of memory if ~69MB are only free.<br>
Reduce the batch size or try to lower the memory usage otherwise.</p>
Issues related to PyTorch Installation from Scratch

Khemraj_Shukla: <p>Hi All-<br>
While compiling PyTorch from Source I get the following errors and then installation stops</p>
<p><strong>error: typing-extensions 4.4.0 is installed but typing-extensions&gt;=4.7.1 is required by {‘filelock’}</strong></p>
<p>Could you please help me with this?</p>
<p>Thanks!</p>

ptrblck: <p>Update <code>typing-extension</code> to version <code>&gt;=4.7.1</code> as mentioned in the error message via <code>pip</code> or <code>conda</code> (depending how you’ve installed it before).</p>

Khemraj_Shukla: <p>I did but Issue persists. This is complete error log. Thanks!</p>
<p>" <strong>/global/common/software/e3sm/anaconda_envs/base/envs/e3sm_unified_1.8.1_pm-cpu/lib/python3.10/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.</strong><br>
<strong>warnings.warn(</strong><br>
<strong>/global/common/software/e3sm/anaconda_envs/base/envs/e3sm_unified_1.8.1_pm-cpu/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.</strong><br>
<strong>warnings.warn(</strong><br>
<strong>Checking .pth file support in /pscratch/sd/z/zhan391/DARPA_project/e3sm_model/code/deepONet/pytorch_test/torch/lib/python3.10/site-packages</strong><br>
<strong>/global/common/software/e3sm/anaconda_envs/base/envs/e3sm_unified_1.8.1_pm-cpu/bin/python3 -E -c pass</strong><br>
<strong>TEST FAILED: /pscratch/sd/z/zhan391/DARPA_project/e3sm_model/code/deepONet/pytorch_test/torch/lib/python3.10/site-packages does NOT support .pth files</strong><br>
<strong>bad install directory or PYTHONPATH</strong><br>
<strong>/pscratch/sd/z/zhan391/DARPA_project/e3sm_model/code/deepONet/pytorch_test/torch/lib/python3.10/site-packages/filelock-3.12.3-py3.10.egg</strong><br>
<strong>error: typing-extensions 4.4.0 is installed but typing-extensions&gt;=4.7.1 is required by {‘filelock’}</strong></p>
<p><img src="https://ca.slack-edge.com/T02MTT6T538-U02UBB5SMSB-d29a95355fd4-48" alt="" role="presentation" width="48" height="48"><br>
"</p>

ptrblck: <p>What does <code>pip list | grep typing</code> return?</p>

Khemraj_Shukla: <p>It returns <code>typing_extensions    4.7.1</code></p>

ptrblck: <p>In this case I unfortunately don’t know where PyTorch find this old version from so you might need to check all virtual environments.</p>

Khemraj_Shukla: <p>Thank you. Let me check.</p>
Help with stable diffusion error pls

ihdffuij: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/2/22a3fa330b73d4becf49baba2fd693b4ac4cd530.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/22a3fa330b73d4becf49baba2fd693b4ac4cd530" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22a3fa330b73d4becf49baba2fd693b4ac4cd530_2_690x237.jpeg" alt="image" data-base62-sha1="4WrykUW0EHajGvb4yCdTyBdrZbW" width="690" height="237" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22a3fa330b73d4becf49baba2fd693b4ac4cd530_2_690x237.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22a3fa330b73d4becf49baba2fd693b4ac4cd530_2_1035x355.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22a3fa330b73d4becf49baba2fd693b4ac4cd530_2_1380x474.jpeg 2x" data-dominant-color="262625"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">3629×1250 1.42 MB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Os: windows 10</p>

J_Johnson: <ol>
<li>
<p>What is your GPU?</p>
</li>
<li>
<p>What does nvidia-smi show in cmd?</p>
</li>
<li>
<p>What version of torch is that installing?</p>
</li>
</ol>

ihdffuij: <ol>
<li>Intel(r) hd graphics 520</li>
<li>I can’t find path to my Nvidia it’s neither in program files nor in system32</li>
<li>I wrote “torch.__version__method” in python it wrote “name torch is not defined”</li>
</ol>

ptrblck: <p>Intel’s GPUs are not supported by CUDA and you would need an NVIDIA GPU.</p>

ihdffuij: <p>Okay. Thanks for helping</p>
How to get enclosed mask from points in differentiable way?

MariosOreo: <p>Hello,</p>
<p>I am doing key points regression problem. To be concrete, I selected some points as <strong>gt points</strong> from the boundary of targets with irregular shape. Now, I can get predicted points from a regression network. What I want to do is implementing a function to get the enclosed mask from the predicted points in differentiable way.</p>
<p>Do you have any suggestions? Thanks in advance! <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

duany049: <p>Hello, I also have the entire requirement. Have you found a solution?<br>
Best wishes to you</p>