akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How to calculate projection tensor

username: Airin6096
cooked: <p>Hi, I want to calculate the projection of gradient(conv gradient) on the direction of weight(conv weight),<br>
does pytorch implement it (tensor projection)? or can someone share some information(web, tutorial) about it ?</p>
<p>Thanks</p>

username: tom
cooked: <p>I think some more context and precision would be neat.</p>
<p>These two things that are close to your description and that come to my mind:</p>
<ul>
<li>“conv gradient” could be the gradient of the loss w.r.t. the output of the convolution and the task is to compute the gradient of the loss w.r.t. the weight. So you would take one step in backpropagation / compute the product of the “conv gradient” with the Jacobian. (Not strictly a projection, but somewhat close.)</li>
<li>“conv gradient” could be the gradient of the loss w.r.t. the weight and the task is to project this onto the weight (so essentially you compute the gradient if the weight were restricted to the line through the origin and the current value). This is a projection in the conventional sense. To compute, you would <a href="https://en.wikipedia.org/wiki/Projection_(linear_algebra)#Finding_projection_with_an_inner_product" rel="noopener nofollow ugc">compute <code>(weight * grad).sum() / weight.norm()**2 * weight</code> or so</a>.</li>
</ul>
<p>Best regards</p>
<p>Thomas</p>
Using backward of function for multiple networks

username: Arta_A
cooked: <p>Hi, I have implemented an actor-critic method with linear networks on both actor and critic also I am using a custom function for loss calculation which is loss function used in PPO. The reward function of environment is different from the loss function of networks.<br>
This is the loss calculation:</p>
<pre><code class="lang-auto"> dist = self.actor(states)
critic_value = self.critic(states)
critic_value = torch.squeeze(critic_value)

new_probs = dist.log_prob(actions)
prob_ratio = new_probs.exp() / old_probs.exp()
 weighted_probs = advantage[batch.item()] * prob_ratio
weighted_clipped_probs = torch.clamp(prob_ratio, 1-self.policy_clip,
                                                   1+self.policy_clip)*advantage[batch.item()]
actor_loss = -torch.min(weighted_probs,
                                weighted_clipped_probs).mean()

returns = advantage[batch.item()] + values[batch.item()]
critic_loss = (returns-critic_value)**2
critic_loss = critic_loss.mean()


total_loss = actor_loss + 0.5*critic_loss

</code></pre>
<p>and then model omtimization:</p>
<pre><code class="lang-auto">self.actor.optimizer.zero_grad()
 self.critic.optimizer.zero_grad()
total_loss.backward()
self.actor.optimizer.step()
self.critic.optimizer.step()
</code></pre>
<p>Now my questions:</p>
<ol>
<li>I want to know is it correct to use backward in this way?</li>
<li>Is the backward now returning grad for both actor and critic networks and the both networks are optimizing?</li>
<li>If I add a feature extractor network before inputting the features to network, and input the extracted features to the actor and critic, will this backward works for feature extractor network?</li>
</ol>

username: Arta_A
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Would you mind please helping me?</p>

username: ptrblck
cooked: <ol>
<li>
<p>Yes, calling <code>total_loss.backward()</code> looks correct. I don’t know if the implemented training logic fits your use case, as I’m not familiar with it.</p>
</li>
<li>
<p>Yes, as long as no tensors are (accidentally) detached. To verify it, check the <code>.grad</code> attributes of the parameters of both models before and after the first <code>backward()</code> call. Before the call, they should be set to <code>None</code>, afterwards to a valid gradient tensor.</p>
</li>
<li>
<p>Yes and the same applies as in 2. If you are not detaching the tensors, the feature extractor would also get gradients.</p>
</li>
</ol>
Static quantization inference

username: FengMu1995
cooked: <p>if my model consists of deeper and nested layers, should I  insert quant&amp;dequant into every layer?</p>

username: supriyar
cooked: <p>If all the operators in the model can be quantized, you can insert a <code>quant</code> and <code>dequant</code> at the beginning and end of the model. See <a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">(beta) Static Quantization with Eager Mode in PyTorch — PyTorch Tutorials 1.10.0+cu102 documentation</a> for more details.</p>
<p>If there are only specific layers that can be quantized then you would have to wrap them individually using quant-dequant blocks in eager mode. Alternatively, you could try the FX graph mode quantization flow which should automate the process - <a href="https://pytorch.org/tutorials/prototype/fx_graph_mode_ptq_static.html" class="inline-onebox" rel="noopener nofollow ugc">(prototype) FX Graph Mode Post Training Static Quantization — PyTorch Tutorials 1.10.0+cu102 documentation</a></p>
Importance of NN input features

username: Roy_Brener
cooked: <p>I have a simple NN for binary classification:</p>
<pre><code class="lang-auto">self.layer1  = nn.Linear(44, 128)
self.layer2  = nn.Linear(128, 512)
self.layer3  = nn.Linear(512, 1024)
self.layer4  = nn.Linear(1024, 512)
self.layer5  = nn.Linear(512, 128)
self.layer6  = nn.Linear(128, 40)
self.layer7  = nn.Linear(40, 1)
</code></pre>
<p>which gets 44 input features (some of them zeros) for training and outputs a classification score. It is trained sensibly and results in low loss (&lt;0.1) and good classification power.</p>
<p>I would like to check which of those 44 features are most important <em>i.e.</em> how each one contributed to the minimisation of the loss function during training. I’ve heard of methods to rank input variables on a par with their relative contribution.</p>
<p>Can someone please explain how to implement this in PyTorch? A minimal code snippet which performs some computation based on <code>trained_net.parameters()</code> perhaps?</p>
<p>Many thanks in advance!</p>

username: Roy_Brener
cooked: <p>Just as a comment, I’ve come across this: <a href="https://stats.stackexchange.com/questions/261008/deep-learning-how-do-i-know-which-variables-are-important" class="inline-onebox" rel="noopener nofollow ugc">neural networks - Deep learning : How do I know which variables are important? - Cross Validated</a> and thought something along the lines of:</p>
<pre><code class="lang-auto">import numpy as np
params = []
sum_of_weights={el:0 for el in np.arange(45)}
i = 0
summarised_weight = 0
for param in trained_net.layer1.parameters():
    params.append(param.view(-1))
params = torch.cat(params)
params_list = params.tolist()

sum_of_weights={el:0 for el in np.arange(45)}
i = 0
summarised_weight = 0
for param in params_list:
    sum_of_weights[i%45] += param
    i += 1
</code></pre>
<p>would be a roughly valid implementation. This results in:</p>
<pre><code class="lang-auto">sum_of_weights

{0: 2.1881065031702747,
 1: -0.4291043917473871,
 2: -3.1886899566743523,
 3: -2.0855790052446537,
 4: 0.8537990134755091,
 5: -3.7174374876194634,
 6: -0.44004582360503264,
 7: -4.356606524073868,
 8: -1.028046389692463,
 9: -3.9685422069451306,
 10: -5.608152595435968,
 11: -3.4701311255339533,
 12: -1.8252374697985942,
 13: -0.7488467986695468,
 14: -8.28944860829506,
 15: -3.4464564417139627,
 16: 4.449272771860706,
 17: -4.691695329251161,
 18: -1.7501768926740624,
 19: 0.37166432512458414,
 20: -10.504940060782246,
 21: -0.6779894242063165,
 22: -6.289113475300837,
 23: -1.3348277652694378,
 24: -3.505854993709363,
 25: -0.9910737440804951,
 26: -12.102533261415374,
 27: -1.0398883295420092,
 28: -5.629827822733205,
 29: -6.627877121747588,
 30: -2.635934035963146,
 31: -2.0738988643570337,
 32: -3.8363308065308956,
 33: -2.3784317194658797,
 34: -1.377102676546201,
 35: -3.0660866306789103,
 36: 4.508294189465232,
 37: 2.8147964420095377,
 38: 0.7389443085994571,
 39: 2.061781210359186,
 40: 5.054453145865409,
 41: -5.041792882340815,
 42: -6.194136218487984,
 43: 0.17481421308184508,
 44: -9.710554008881445}
</code></pre>
<p>Any thoughts? Can some sense be made out of this?</p>
How to avoid the object detector to focus on some objects

username: thejoe
cooked: <p>Hello everyone,</p>
<p>I’m working with the Faster RCNN to predict the bounding boxes of some particular objects inside a fixed environment. Basically what I want to predict are out-of-place bolts inside a scene. I already trained my model with synthetic images and validating it on real images and fortunately I get some discreet results. The detector is quite confident on locating the bolts (scores &gt; 0.98) but it gets confused with objects with similar shape. An example could be the rivets.</p>
<p>I created my dataset including only images with objects and their bounding boxes but is there a way to actually “suggest” the model even to <strong>not</strong> consider some objects instead of telling it to locate those bolts?<br>
Thanks in advance</p>

username: ptrblck
cooked: <p>You could add these negative examples to the dataset and don’t provide valid bounding boxes for all negative instances, which would train the model to ignore them.</p>

username: thejoe
cooked: <p>Do you mean by providing empty bounding boxes? Suppose there is an object that I don’t want to be recognized, i can simply annotate the image with</p>
<pre><code class="lang-auto">target = {}
target["boxes"] = torch.zeros((0, 4), dtype=torch.float32)
target["labels"] = torch.zeros((0, 1), dtype=torch.int64)
</code></pre>
<p>… or maybe by considering the whole image as a bounding box</p>

username: ptrblck
cooked: <p>I don’t know if it would be necessary to provide empty boxes or just not annotate the unwanted objects.</p>

username: Pcamellon
cooked: <p>Hi! Thanks for your advice. I think images without annotations would be ignored at all during loading. I’m not labeling the unwanted objects. How to provide images with the unwanted objects only and not give annotations?</p>
Change the input size of timm's ViT model

username: Nimrod_Daniel
cooked: <p>Hi,</p>
<p>I have a trained Vision Image Transformer from time (Google’s ViT paper) and I want to change the input size, and use an input with larger dimensions.<br>
Code: <a href="https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py" rel="noopener nofollow ugc">https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py</a></p>
<p>My model that I want to change its input size:</p>
<blockquote>
<p>model = timm.models.vit_base_patch16_224_in21k(pretrained=True)</p>
</blockquote>
<p>I tried accessing the dictionary with the input size</p>
<blockquote>
<p>timm.models.vision_transformer.default_cfgs[‘vit_base_patch16_224_in21k’][‘input_size’] = (3,400,400)</p>
</blockquote>
<p>But it doesn’t seem to solve it. I still get the following error (Iknow I can use a 384*384 pretrained model, but that’s not the question here):</p>
<blockquote>
<p>f"Input image size ({H}<em>{W}) doesn’t match model ({self.img_size[0]}</em>{self.img_size[1]})."<br>
AssertionError: Input image size (400<em>400) doesn’t match model (224</em>224).</p>
</blockquote>
<p>I think  that I should add a layer before the model to solve that. Any ideas?</p>

username: my3bikaht
cooked: <aside class="quote no-group" data-username="Nimrod_Daniel" data-post="1" data-topic="129104">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nimrod_daniel/48/10282_2.png" class="avatar"> Nimrod_Daniel:</div>
<blockquote>
<p>vit_base_patch16_224_in21k</p>
</blockquote>
</aside>
<p>function</p>
<blockquote>
<p>timm.models.vit_base_patch16_224_in21k(pretrained=True)</p>
</blockquote>
<p>calls for function</p>
<pre><code class="lang-auto">_create_vision_transformer
</code></pre>
<p>which, on it’s turn calls for</p>
<pre><code class="lang-auto">build_model_with_cfg(
</code></pre>
<p>This function creates instance of a class VisionTransformer(nn.Module) (currently line 230) with following (default) parameters:</p>
<blockquote>
<p>img_size=224, patch_size=16, in_chans=3, num_classes=1000, embed_dim=768, depth=12, num_heads=12, mlp_ratio=4., qkv_bias=True, representation_size=None, distilled=False, drop_rate=0., attn_drop_rate=0., drop_path_rate=0., embed_layer=PatchEmbed, norm_layer=None, act_layer=None, weight_init=‘’</p>
</blockquote>
<p>If you don’t need pretrained model, you can create instance of this class with required parameters directly.</p>
What to use to teach the web to html code completion

username: Grzegorz_Banka
cooked: <p>Hi, I’m trying to find a way by which I will be able to teach the network to complete html code with TYPO3 fluid tags.<br>
I used over-trained gpt-neo for this but it works 50/50.<br>
Here code html with fluid tags: <a href="https://github.com/benjaminkott/bootstrap_package/blob/master/Resources/Private/Partials/Page/Navigation/MainNavigation.html" class="inline-onebox" rel="noopener nofollow ugc">bootstrap_package/MainNavigation.html at master · benjaminkott/bootstrap_package · GitHub</a></p>
<p>Did anyone have a problem similar to the relationship?</p>
Poor performance comparing to Matlab in simple classification

username: Kyle_Wang
cooked: <p>I’m experimenting a very simple image classification task: classify chest x-ray to either normal, bacterial infected or virus infected.</p>
<p>Please see my github: <a href="https://github.com/kail85/SimpleTorchClassification" class="inline-onebox" rel="noopener nofollow ugc">GitHub - kail85/SimpleTorchClassification</a></p>
<p>My training script is in Train.ipynb. The data are simple gray scale image of the size 256 x 256, and being spitted to train, val and test sets. In each set, the ratio of the 3 image categories remains the same.</p>
<p>I wrote the script in both torch and Matlab. Using mobilenetv2, Matlab stops at epoch 9 with a final validation loss 0.1714 and accuracy 82.91%.</p>
<p>While for pytorch with the exact same training setting, the validation loss never drops below 1.0 and there’s a severe overfitting.</p>
<p>I’m not sure what happened, is there anything wrong with my validation script? I tried mobilenetv3_small, mobilenetv3_large, and resnet18, they all have the same issue.</p>
<p>Only alexnet can reach a validation loss at about 0.4 at epoch 20.</p>
<p>I’ve been stuck here for a few days. I’m appreciated any suggestion. Thank you.</p>
<p><strong>Update</strong>: I put a break point in the validation function, at the line</p>
<p><code>outputs_val = net(inputs_val)</code></p>
<p>its output is</p>
<pre><code class="lang-auto">tensor([[ 1.1345, -1.1818,  0.0358],
        [ 1.1345, -1.1818,  0.0358],
        [ 1.1345, -1.1818,  0.0358],
        [ 1.1345, -1.1818,  0.0358],
        [ 1.1345, -1.1818,  0.0358],
        [ 1.1345, -1.1818,  0.0358],
        [ 1.1345, -1.1818,  0.0358],...
</code></pre>
<p>Then I tried</p>
<p><code>net(torch.zeros_like(inputs_val))</code></p>
<p>it gives</p>
<pre><code class="lang-auto">tensor([[ 1.1345, -1.1818,  0.0358],
        [ 1.1345, -1.1818,  0.0358],
        [ 1.1345, -1.1818,  0.0358],
        [ 1.1345, -1.1818,  0.0358],
        [ 1.1345, -1.1818,  0.0358],
        [ 1.1345, -1.1818,  0.0358],...
</code></pre>
<p>I’m about to cry…</p>

username: MartinZhang
cooked: <p>I am not sure, how much learning range? <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=10" title=":thinking:" class="emoji" alt=":thinking:"></p>
How to translate Tensor's type in CPP?

username: MartinZhang
cooked: <p>How to change Tensor’s type in CPP?</p>
<p>In Pytorch, we can do that.</p>
<pre><code class="lang-auto">import torch

a = torch.randn(10)  # a.dtype -&gt; torch.float32

b = a.type(torch.int32);  # b.dtype -&gt; torch.int32

</code></pre>
<p>But  how to change the type in CPP?</p>
<pre><code class="lang-auto">torch::Tensor data;  // torch.float32

data.type(torch::kInt32);  // error: no matching function for call to "at::Tensor::type ..."
</code></pre>
<p>please help me, thanks a lot.</p>

username: MartinZhang
cooked: <p>I found it .</p>
<pre><code class="lang-auto">torch::Tensor x = torch::randn({10, 50});
torch::Tensot x_int = x.to(torch::kInt);
</code></pre>
<p>thanks <a class="mention" href="/u/dfalbel">@dfalbel</a></p>
IndexError: Can only index with tensors that are scalars(zero-dim)

username: MartinZhang
cooked: <p><strong>How to use the tensor index in CPP?</strong></p>
<p>In pytorch, we can in this way.</p>
<pre><code class="lang-auto">import torch

data = torch.tensor([1, 2, 3, 4, 5, 6])

index = torch.tensor([0, 2, 4])

result = data[index] 
</code></pre>
<p>We can get result’s value</p>
<pre><code class="lang-auto">&gt;&gt;&gt; tensor([1, 3, 5])
</code></pre>
<p><strong>But how to use in cpp?</strong></p>
<pre><code class="lang-auto">torch::Tensor data;  // just a demo  

torch::Tensor index;  // just a demo

torch::Tensor result = data[index];  // IndexError: Can only index with tensors that are scalars(zero-dim)
</code></pre>
<p>please help me, thanks a lot. <img src="https://discuss.pytorch.org/images/emoji/apple/handshake.png?v=10" title=":handshake:" class="emoji" alt=":handshake:"></p>

username: MartinZhang
cooked: <p><strong>I found a good way</strong></p>
<pre><code class="lang-auto">at::Tensor at::index_select(const at::Tensor &amp;self, at::Dimname dim, const at::Tensor &amp;index)
</code></pre>
<pre><code class="lang-auto">torch::Tensor a = at::index_select(torch::Tensor data, torch::Tensor index)
</code></pre>
<p><strong>But there’s still a problem.</strong></p>
<pre><code class="lang-auto">error : no matching function for call to 'index_select(at::Tensor&amp;, at::Tensor$)'
</code></pre>
<p>Maybe this method takes three arguments， But How to get dim???</p>
<p>Thanks.</p>

username: MartinZhang
cooked: <p>I found it，<code>at::Dimanme dim</code>  is equivalent to <code>int</code></p>
<p>In this way：</p>
<pre><code class="lang-auto">torch::Tensor data;

torch::Tensot index;

torch::result = at::index_select(data, 0, index);
</code></pre>
<p>So funny <img src="https://discuss.pytorch.org/images/emoji/apple/grin.png?v=10" title=":grin:" class="emoji" alt=":grin:"></p>
Jacobian matrix computation

username: Le_ZHENG
cooked: <p>Hi there, I was trying to compute the real Jacobian matrix of the input and output from a network. I figure out two ways to do this, and I think they should be identical. But the results are different and I can’t find why. Below are some example codes to repeat my result.</p>
<pre><code class="lang-auto">import torch
from torch import nn
n = 2
in_dim = n
hidden_dims = [10,20]
rnvpNet = nn.ModuleList([])
for h_dim in hidden_dims:
    rnvpNet.append(nn.Linear(in_dim, h_dim))
    rnvpNet.append(nn.BatchNorm1d(h_dim))
    rnvpNet.append(nn.Tanh())
    in_dim = h_dim

rnvpNet = rnvpNet.append(nn.Linear(h_dim, n))

x = torch.rand(4, n, requires_grad=True)
cur_batch_size = x.shape[0]

# method 1
x1 = x
z1 = x1
for rnvpModule in rnvpNet:
    z1 = rnvpModule(z1)

Jacob1 = torch.eye(n, requires_grad=True).unsqueeze(0).expand(cur_batch_size, n, n)
J1 = Jacob1.clone()
for i in range(n):
    v1 = torch.zeros_like(x)
    v1[:, i] = 1
    J1[:, i, :] = torch.autograd.grad(z1, x1, v1, retain_graph=True)[0]

# method 2
x2 = x.repeat(1, n).view(-1, n)
z2 = x2
for rnvpModule in rnvpNet:
    z2 = rnvpModule(z2)

v2= torch.eye(n).repeat(cur_batch_size, 1)
J2 = torch.autograd.grad(z2, x2, v2, create_graph=True)[0]
J2 = J2.reshape(cur_batch_size, n, n)
</code></pre>
<p>J1 method needs to run a for loop, so I think it is slow if n is large. Then I find another implementation, the J2 method from some online sources. I think J1 should be exactly the same as J2. Can anyone help to find what’s wrong with my code?</p>
How long does it take to train the network in the CIFAR-10 tutorial?

username: benlundell
cooked: <p>Hello- I’m working through the <a href="http://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py" rel="nofollow noopener">CIFAR-10 tutorial</a> and I am wondering how long it usually takes to run the training. In CPU mode, it is consistently taking my machine 1h10m to complete the two epochs in the example. Are other people seeing similar times?</p>

username: smth
cooked: <p>1h10m, oh my that’s a lot. It takes me about a minute.</p>
<p>What’s your OS and version?<br>
How did you install pytorch?<br>
Are you running this in a virtual machine, or on bare metal?</p>
<p>I hope that with this information, I can reproduce it.</p>

username: benlundell
cooked: <p>Thanks for the speedy reply! Here’s my info:</p>
<p>OS- Ubuntu 14.04.1<br>
Python- 3.6.1<br>
Install- Pulled the repo and built it using "python <a href="http://setup.py" rel="nofollow noopener">setup.py</a> install"<br>
VM- No, but I am doing this through Xfce from a Windows box, if that matters.</p>

username: smth
cooked: <p>Your install method is the reason why it’s so slow.<br>
You have two options:</p>
<ul>
<li>Install the binary from <a href="http://pytorch.org">pytorch.org</a> (we provide wheel files and conda binaries)</li>
<li>Install a proper BLAS library on your machine (MKL or OpenBLAS, MKL is available via Conda) and add it to the environment variable <code>CMAKE_PREFIX_PATH</code>. Otherwise pytorch will use super-unoptimized BLAS, and that’s the reason for the slowness. You can read one way of doing this here: <a href="https://github.com/pytorch/pytorch#from-source">https://github.com/pytorch/pytorch#from-source</a>
</li>
</ul>

username: benlundell
cooked: <p>I tried both options, and neither seems to make a difference <img src="//discuss.pytorch.org/images/emoji/apple/frowning.png?v=5" title=":frowning:" class="emoji" alt=":frowning:">  However, I found an issue with torchvision when installing via conda. Where should I report that?</p>

username: smth
cooked: <p>report torchvision issues to <a href="https://github.com/pytorch/vision">https://github.com/pytorch/vision</a></p>

username: benlundell
cooked: <p>Thanks. I’ve opened an issue here: <a href="https://github.com/pytorch/vision/issues/151" rel="nofollow noopener">https://github.com/pytorch/vision/issues/151</a></p>

username: smth
cooked: <p>another reason for your slowdown might be a large number of CPU cores.<br>
Try setting these environment variables:</p>
<pre><code class="lang-auto">export OMP_NUM_THREADS=4
export MKL_NUM_THREADS=4
</code></pre>
<p>See if that fixes it?</p>

username: benlundell
cooked: <p>That did it for me. The training just completed in less then 2 minutes. Thanks for all the help!</p>

username: Brando_Miranda
cooked: <p>why did that fix it? Just curious.</p>

username: RylanSchaeffer
cooked: <p>What effect does setting those two environment variables have on the speed of training?</p>
<pre><code class="lang-auto">export OMP_NUM_THREADS=4
export MKL_NUM_THREADS=4
</code></pre>
Inplace error in PyTorch LSTM

username: term1112
cooked: <p>I am using an LSTM to summarize a trajectory as shown below. The shapes of features and rewards are [50, 1, 2048] and [50, 1, 1] respectively, i.e. the sequence length is 50 and the batch size is 1.</p>
<pre><code class="lang-auto">class RolloutEncoder(nn.Module):
    def __init__(self, config):
        super(RolloutEncoder, self).__init__()
        self._input_size = (
            2048 + 1
        )  # deter_state + imag_reward; fix and use config["deter_dim"] + 1
        self._hidden_size = config["rollout_enc_size"]
        self._lstm = nn.LSTM(self._input_size, self._hidden_size, bias=True)

    def forward(self, traj):
        features = traj["features_pred"]
        rewards = traj["reward_pred"].unsqueeze(1)
        input = torch.cat((features, rewards), dim=2)
        encoding, (h_n, c_n) = self._lstm(input)
        code = h_n.squeeze(0)
        return code
</code></pre>
<p>My training loop goes something like this:</p>
<pre><code class="lang-auto">encoder = RolloutEncoder(config)
for e in range(episodes):
      for step in range(steps):
            print(f"Step {steps})
            # calc traj
            code = encoder(traj)
            # some operations that do not modify code but only concat it with some other tensor
            # calc loss
            opt.zero_grad()
            loss.backward()
            opt.step()
</code></pre>
<p>On running, I get this error:</p>
<pre><code class="lang-auto">Step 0
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
Step 7
Step 8
Step 9
Step 10
Step 11
Step 12
Step 13
Step 14
Traceback (most recent call last):
  File "/path/main.py", line 351, in &lt;module&gt;
    agent_loss.backward()
  File "/home/.conda/envs/abc/lib/python3.9/site-packages/torch/tensor.py", line 245, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/user/.conda/envs/abc/lib/python3.9/site-packages/torch/autograd/__init__.py", line 145, in backward
    Variable._execution_engine.run_backward(
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [16, 2049]] is at version 8; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
</code></pre>
<p>On setting the anomaly_detection to True, it point to this line in the encoder definition:</p>
<pre><code class="lang-auto">encoding, (h_n, c_n) = self._lstm(input)
</code></pre>
<p>I know this is a very common error but I am not using any inplace operation in my code and all the other discussions on the same topic couldn’t solve this. Moreover the error occurs after running some steps successfully which is really weird. On inspecting, I found that the [16, 2049] tensor is one of the weights of the LSTM. I also tried using dummy random tensors in place of <code>features</code> and <code>rewards</code> but the error persists, suggesting that the traj calculation has nothing to do with this error.</p>
<p>Why might this be happening and how can I solve this?</p>
CUDA utilization

username: AamirMKhan
cooked: <p>Hi,</p>
<p>I am trying to run the codes here [tutorials/unetr_btcv_segmentation_3d.ipynb at master · Project-MONAI/tutorials (<a href="http://github.com" rel="noopener nofollow ugc">github.com</a>)]</p>
<p><a href="https://github.com/Project-MONAI/tutorials/blob/master/3d_segmentation/unetr_btcv_segmentation_3d.ipynb" class="inline-onebox" rel="noopener nofollow ugc">tutorials/3d_segmentation/unetr_btcv_segmentation_3d.ipynb at main · Project-MONAI/tutorials · GitHub</a>)</p>
<p>But I encounter the problem of:<br>
RuntimeError: CUDA out of memory. Tried to allocate 286.00 MiB (GPU 0; 4.00 GiB total capacity; 1.39 GiB already allocated; 227.40 MiB free; 1.97 GiB reserved in total by PyTorch) If reserved memory is &gt;&gt; allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF</p>
<p>I am not sure if I am using the GPU optimaly. I have also attached the screenshot of the task maanger. Please let me know if I am doing something wrong?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/1/41c4426d3cd8ff4097a8bd5acbf99f2d96932f7c.png" data-download-href="https://discuss.pytorch.org/uploads/default/41c4426d3cd8ff4097a8bd5acbf99f2d96932f7c" title="Capture1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/41c4426d3cd8ff4097a8bd5acbf99f2d96932f7c_2_690x398.png" alt="Capture1" data-base62-sha1="9nNwdf85oTcG9wc19XESNU45Muw" width="690" height="398" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/41c4426d3cd8ff4097a8bd5acbf99f2d96932f7c_2_690x398.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/41c4426d3cd8ff4097a8bd5acbf99f2d96932f7c_2_1035x597.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/41c4426d3cd8ff4097a8bd5acbf99f2d96932f7c_2_1380x796.png 2x" data-dominant-color="FAFBFC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture1</span><span class="informations">3815×2201 237 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Thanks</p>

username: ptrblck
cooked: <p>The error explains that your GPU is running out of memory, so you would need to reduce the memory usage by e.g. lowering the batch size if possible.</p>

username: AamirMKhan
cooked: <p>Thanks for the reply. I had this query that is there any way of verifying whether I am utilising the GPU capabilities of my system correctly, as I am not able to understand the significance of the shared GPU memory here.</p>

username: ptrblck
cooked: <p>I’m not familiar enough with Windows, so don’t know what each metric shown by the task manager means.<br>
The <code>Cuda</code> window should show the compute utilization, which would show the GPU util. when PyTorch uses the device for computations.<br>
Alternatively, use <code>nvidia-smi</code>, which would show the same.</p>
Object detection algorithms

username: Vijay_Dubey
cooked: <p>Do object detection algorithms like YOLO, SSD etc. require a fixed size input image to be passed to them or can they deal with varying image sizes also</p>

username: Pcamellon
cooked: <p>I think the input image size is due to computational cost. The fixed size would be due to the images sizes the pretrained model was previously trained on.</p>
Implementation of Object Detection Training on Custom Dataset

username: Laya1
cooked: <p>I am training object detectors (Faster RCNN and RetinaNet) for a custom image dataset. I am facing problems with empty/garbage output for the trained detector. I would appreciate any help in resolving these issues.</p>
<ul>
<li>
<strong>Data</strong>: RGB Images of size 3040 x 4048 x 3</li>
<li>
<strong>Task</strong>: Detection of a single type of object in the images</li>
<li>
<strong>Model</strong>:
<ul>
<li>Retinanet: <code>torchvision.models.detection.retinanet_resnet50_fpn(pretrained=False, pretrained_backbone=True, num_classes=2)</code>
</li>
<li>Faster RCNN:</li>
</ul>
</li>
</ul>
<pre><code class="lang-auto">      model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
      num_classes = 2  # 1 class (person) + background
      in_features = model.roi_heads.box_predictor.cls_score.in_features
      model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)
</code></pre>
<ul>
<li>
<strong>Problems</strong>:
<ul>
<li>The Faster RCNN produces bounding boxes that are very small, and always to the top left of the image - they never contain the object.</li>
<li>The RetinaNet produces empty predictions - the “boxes” variable in the output dictionary is a [0x4] tensor</li>
</ul>
</li>
<li>Questions:
<ul>
<li>Why is the Faster RCNN producing garbage boxes even if the loss is decreasing at the time of training?</li>
<li>Why is the RetinaNet model producing empty boxes even before training? What can I do to solve these issues?</li>
<li>How should the model be changed to accommodate the larger size of the images compared to the standard image datasets? Is this needed?</li>
</ul>
</li>
<li>
<strong>Remark</strong>: The RetinaNet model produces empty output even before training.</li>
</ul>
<p>Please let me know if any details of the implementation will help in identifying the problem. Thank you in advance for your help.</p>

username: Pcamellon
cooked: <p>Hi! I’m trying to do the object detection task. Can you share your code or point me to a good tutorial?</p>
Object detection without segmentation

username: FaultyBagnose
cooked: <p>The pytorch tutorial for object detection finetuning includes instance segmentation, which requires segmentation mask annotations. Is there a tutorial for the (presumably simpler) task of object detection without segmentation, where the task is to learn bounding boxes with class labels only?</p>
<p>Thanks in advance!</p>

username: KFrank
cooked: <p>Hi Faulty!</p>
<aside class="quote no-group" data-username="FaultyBagnose" data-post="1" data-topic="107711" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/96bed5/48.png" class="avatar"> FaultyBagnose:</div>
<blockquote>
<p>Is there a tutorial for the (presumably simpler) task of object detection without segmentation, where the task is to learn bounding boxes with class labels only?</p>
</blockquote>
</aside>
<p>I don’t believe that there is a tutorial for detection, per se, but pytorch<br>
does provide two pure detection models in torchvision:</p>
<p><a href="https://pytorch.org/docs/stable/torchvision/models.html#torchvision.models.detection.fasterrcnn_resnet50_fpn" rel="noopener nofollow ugc">Faster R-CNN</a><br>
<a href="https://pytorch.org/docs/stable/torchvision/models.html#torchvision.models.detection.retinanet_resnet50_fpn" rel="noopener nofollow ugc">RetinaNet</a></p>
<p>(I haven’t used either of these.)</p>
<p><a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html" rel="noopener nofollow ugc">Pytorch’s object-detection tutorial</a> illustrates the use of Mask R-CNN,<br>
but does also talk about Faster R-CNN some.  So you ought to be<br>
able to use that tutorial for Faster R-CNN (or RetinaNet), leaving<br>
out the instance-segmentation stuff.</p>
<p>Best.</p>
<p>K. Frank</p>

username: FaultyBagnose
cooked: <p>Thanks,<br>
It does indeed talk about Faster R-CNN, but doesn’t show how to use it. I actually find easier to build a script from scratch based on the doc files than to factor out the segmentation stuff, which is disappointing given the tutorial title.</p>
<p>I think it’s very good for segmentation stuff, and I’m very grateful for the efforts that have been invested in this and other tutorials. Still, a short tutorial on plain object detection would be most welcome.</p>

username: Pcamellon
cooked: <p>Hi! I feel the same way. Did you found some simple tutorial just for get me started?</p>
The computational complexity of some lines

username: DRISS_ELALAOUI
cooked: <p>how can I calculate the complexity O of these lines?<br>
1- a function that returns weights in the form of a tensor PyTorch</p>
<pre><code class="lang-auto">def weights(n, alpha):
    w=torch.tensor([(alpha**i)*(1-alpha) for i in range(n)])
    return w
</code></pre>
<p>2- then I multiply each element of a list by its weight:</p>
<pre><code class="lang-auto">n = 6
alpha = 0.5
List = List * weights(n, alpha) // List is a pytorch tensor which contains integers
</code></pre>
<p>and thank you</p>
Changing the location of pixels in input image and backpropagation algorithm

username: mahsa
cooked: <p>I have two end-to-end networks. The input of the network is an image with size N by N. The output of the first network is an N by N mask that mask[i,j]=k means that pixel k in the input should change location to i-th row and j-th column of the image (so the pixels of the input image will be switched with each other). Then this new image will be input to the second network and at the end, there is a loss function. Is there any way to have this relocation for pixels such that the backpropagation algorithm works well?<br>
Thanks</p>
Error when register_module in Debug mode but not in Release moode

username: kila_suelika
cooked: <p>I’m using Visual Studio 2022 in Windows 10, following code generate a Reading Access error:</p>
<pre><code class="lang-auto">#include &lt;iostream&gt;
#include &lt;torch/torch.h&gt;

class MyLinear : public torch::nn::Module {
public:
	torch::nn::Linear linear;
	MyLinear() :linear(torch::nn::Linear(3, 1)) { register_module("linear", linear); };
};

int main() {
	MyLinear linear;
}
</code></pre>
<p>Screenshot:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/0/10a08f67039485e85ebd11fefedaca0324aa7de5.png" data-download-href="https://discuss.pytorch.org/uploads/default/10a08f67039485e85ebd11fefedaca0324aa7de5" title="torch_error"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/10a08f67039485e85ebd11fefedaca0324aa7de5_2_690x388.png" alt="torch_error" data-base62-sha1="2n5DyQ1MqfmXuwyP0HemZpBQNhP" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/10a08f67039485e85ebd11fefedaca0324aa7de5_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/10a08f67039485e85ebd11fefedaca0324aa7de5_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/10a08f67039485e85ebd11fefedaca0324aa7de5_2_1380x776.png 2x" data-dominant-color="2E2E2E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">torch_error</span><span class="informations">1920×1080 298 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>If I switch to Release mode, then there is no problem. The Debug and Release mode use the same torch library (nightly build).</p>
Applying different data augmentation per image in a mini-batch

username: bellos1203
cooked: <p>As far as I know, the random transformations (e.g. random crop, random resized crop, etc.) from <code>torchvision.transforms</code> module apply the same transformations to all the images of a given batch.<br>
Is there any efficient way to apply different random transformations for each image in a given mini-batch?</p>
<p>Thanks in advance.</p>

username: ptrblck
cooked: <p>I think you would need to apply the random transformations on each sample and could use e.g. <code>transforms.Lambda</code> for it:</p>
<pre><code class="lang-python"># setup
x = torch.zeros(3, 10, 10)
x[:, 3:7, 3:7] = 1.

# same transformation on each sample
transform = transforms.RandomCrop(size=5)
y1 = transform(x)
print(y1)

# apply transformation per sample
y2 = transforms.Lambda(lambda x: torch.stack([transform(x_) for x_ in x]))(x)
print(y2)
</code></pre>

username: bellos1203
cooked: <p>Thanks, your solution works well!</p>
# > TypeError: only integer tensors of a single element can be converted to an index

username: Dealon_let
cooked: <p>I have tested torch==1.7.0, torch==1.9.0, and torch==1.10.0</p>
<p>I have the following error. Can I get away from this?</p>
<blockquote>
<p>torch.tensor([torch.tensor(0.), torch.tensor(1.)]) # works<br>
torch.tensor([torch.tensor(0), torch.tensor(1)], dtype=torch.int32) # works<br>
torch.tensor([torch.tensor(0.), torch.tensor(1.)], dtype=torch.int32) # fails</p>
<h1>&gt; TypeError: only integer tensors of a single element can be converted to an index</h1>
</blockquote>

username: my3bikaht
cooked: <aside class="quote no-group" data-username="Dealon_let" data-post="1" data-topic="139152">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dealon_let/48/44840_2.png" class="avatar"> Dealon_let:</div>
<blockquote>
<p>torch.tensor([torch.tensor(0.), torch.tensor(1.)], dtype=torch.int32)</p>
</blockquote>
</aside>
<p>torch.tensor([torch.tensor(0.), torch.tensor(1.)]).int()</p>
LossFunction, optimizer, and accuracy for a neural network that determines the depth map of the image

username: MACHOCAPTCHA
cooked: <p>Tell me which ones to take LossFunction, optimizer, and accuracy.<br>
I am trying to recreate a UNet network for predicting depth from a picture. And I don’t know which one is better to choose. Tell me or send where it is written in more detail about this please! If I made a mistake with the branch, then I apologize.</p>

username: ZimoNitrome
cooked: <p>U-Net sounds like a good choice. You could use something like L1 or L2 loss with the Adam optimizer. These settings seem to work well in general.</p>
<p>If you want to find studies or comparisons of SOTA models, you can look at:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://paperswithcode.com/task/depth-estimation">
  <header class="source">

      <a href="https://paperswithcode.com/task/depth-estimation" target="_blank" rel="noopener nofollow ugc">paperswithcode.com</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/369;"><img src="https://production-media.paperswithcode.com/tasks/images_c8txmyp.jpg" class="thumbnail" width="690" height="369"></div>

<h3><a href="https://paperswithcode.com/task/depth-estimation" target="_blank" rel="noopener nofollow ugc">Papers with Code - Depth Estimation</a></h3>

  <p>**Depth Estimation** is a crucial step towards inferring scene geometry from 2D images. The goal in monocular Depth Estimation is to predict the depth value of each pixel, given only a single RGB image as input.


Source: [DIODE: A Dense Indoor and...</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
or</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/scott89/awesome-depth">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/scott89/awesome-depth" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/9ab65c3b01e1af20ee3496ca4407ca7e0b21aba11292e447dc4bd105a3381a2d/scott89/awesome-depth" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/scott89/awesome-depth" target="_blank" rel="noopener nofollow ugc">GitHub - scott89/awesome-depth: A curated list of publication for depth...</a></h3>

  <p>A curated list of publication for depth estimation - GitHub - scott89/awesome-depth: A curated list of publication for depth estimation</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


username: MACHOCAPTCHA
cooked: <p>Thank you for your reply! I’m sorry that I was gone for a long time. Was very busy! Now I hope to tackle the topic more tightly!</p>
Quantized Batch norm parameters not saved in state_dict

username: jby
cooked: <p>I found out that the scale and zero point parameters of the BatchNorm2d module after conversion with post-training quantization are not included the model.state_dict(). Therefore, when saving the state dict of the converted model to a file, these parameters are lost. This yields different results when doing inference just after post-training quantization, or whether first saving the state dict of the quantized model to a file and later on loading it again.</p>
<p>This does not happen when the batch norm module is fused with another module (like convolution), because then, because the scale and zero point is saved for the module if it is fused (e.g., with convolution module).</p>
<p>An alternative is to save the state dict of the model before running <code>torch.quantization.convert(model)</code>. However, is there a specific reason why the scale and zero point of the quantized batch norm module is not included in the state dict of the model after conversion?</p>
<p>Reference: <a href="https://pytorch.org/docs/stable/_modules/torch/nn/quantized/modules/batchnorm.html#BatchNorm2d" class="inline-onebox" rel="noopener nofollow ugc">torch.nn.quantized.modules.batchnorm — PyTorch 1.10.0 documentation</a> (here you see that scale and zero point are taken from the <code>activation_post_process.calculate_qparams()</code>, but when loading a new model without doing proper calibration, these values are not set correctly)</p>

username: jerryzh168
cooked: <p>I think this might be a bug, thanks for reporting, opened an issue here: <a href="https://github.com/pytorch/pytorch/issues/69808" class="inline-onebox" rel="noopener nofollow ugc">quantized batchnorm parameters/buffers not saved in state_dict · Issue #69808 · pytorch/pytorch · GitHub</a></p>
NotImplementedError: Could not run 'aten::empty.memory_format' with arguments from the 'QuantizedCPU' backend

username: dalseeroh
cooked: <p>I am trying to quantize a model, but I got an error when I executes:</p>
<pre><code class="lang-auto">model_int = torch.quantization.convert(model_fp_prepared)
print("Conversion completed")
q_output = model_int(img, mask)
</code></pre>
<blockquote>
<p>NotImplementedError: Could not run ‘aten::empty.memory_format’ with arguments from the ‘QuantizedCPU’ backend. This could be because the operator doesn’t exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit <a href="https://fburl.com/ptmfixes" class="inline-onebox" rel="noopener nofollow ugc">Internal Login</a> for possible resolutions. ‘aten::empty.memory_format’ is only available for these backends: [CPU, CUDA, Meta, MkldnnCPU, SparseCPU, SparseCUDA, BackendSelect, Python, Named, Conjugate, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, UNKNOWN_TENSOR_TYPE_ID, AutogradMLC, AutogradHPU, AutogradNestedTensor, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, Tracer, UNKNOWN_TENSOR_TYPE_ID, Autocast, Batched, VmapMode].</p>
</blockquote>
<p>It looks like it happened here:</p>
<pre><code class="lang-auto">output = self.input_conv(input_x * mask)
</code></pre>

username: dalseeroh
cooked: <p>I guess I figured it out. This error seems to happen when I try to multiply those quantized tensors(input_x, mask). The workaround I took is:</p>
<pre><code class="lang-auto"># First, dequantize the quantized tensor
 input_x = self.dequant(input_x)
 mask = self.dequant(mask)
# Do the operation and quantize it back
 masked = input_x * mask
 masked = self.quant(masked)

 input_x = self.quant(input_x)
 mask = self.quant(mask)

 output = self.input_conv(masked)
</code></pre>
<p>Seems like pretty tedious work but it works. However, can I use self.quant() multiple times like that? or Should I use self.quant1(), self.quant2(), self.quant3() separately?</p>

username: jerryzh168
cooked: <p>if you want to quantize multiplication, you’ll need to rewrite <code>*</code> to use functional modules: <a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/quantized/modules/functional_modules.py#L55" class="inline-onebox" rel="noopener nofollow ugc">pytorch/functional_modules.py at master · pytorch/pytorch · GitHub</a>, an example can be found here: <a href="https://github.com/pytorch/pytorch/blob/e61fc1c03b64e61ca4f5bbe278db7ee2cf35e8ff/torch/testing/_internal/common_quantization.py#L1868" class="inline-onebox" rel="noopener nofollow ugc">pytorch/common_quantization.py at e61fc1c03b64e61ca4f5bbe278db7ee2cf35e8ff · pytorch/pytorch · GitHub</a></p>
<p>If you want to dequantize then quantize, it’s better to use different instances of quantize module.</p>
PyTorch distributed data-parallel (multi GPU, multi-node)

username: MayfieldMob
cooked: <p>I have access to 18 nodes each with different numbers of GPUs all with at least 1 to my understanding you have to declare all the nodes to have the same number of GPUs.</p>
<p>first of all, am I correct?</p>
<p>and second, If I am is there any way around this?</p>
Hardware suggestion

username: AamirMKhan
cooked: <p>Hi,</p>
<p>I am trying to develop a deep learning based model for automatic segmentation of medical data. I would be using the UNETR model for this purpose. I wanted to get some suggestions on the hardware that I can use to develop this model. Currently my laptop has:</p>
<ol>
<li>OS: Windows 10</li>
<li>RAM: 32GB</li>
<li>SSD: 1 TB</li>
<li>GPU1: NVIDIA QUADRO T2000 with Max Q design with dedicated GPU memory of 4GB, and shared<br>
GPU memory of 16 GB.</li>
<li>GPU2: INTEL(R) UHD Graphics with shared GPU memory of 16GB.</li>
</ol>
<p>The training dataset would consist of around 300-500 CTs. I needed some suggestions on what should I go with: an external GPU or a cloud based service. And what external GPUs would be a good option.</p>
<p>Thanks.</p>