akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How normalization aids CNN?

username: aleemsidra
cooked: <p>I am new to CNN and PyTorch. I am working on the classification of diabetic ulcers. I have achieved some results using ResNet. I want to do regularization- one of the techniques I am using is augmentation. How normalization helps CNN? The batch below is without normalization and is visually more perceivable to the human eye.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/0/30403247d1fe04ab6544e779ec84ecde5d27b6e3.png" data-download-href="https://discuss.pytorch.org/uploads/default/30403247d1fe04ab6544e779ec84ecde5d27b6e3" title="not_normalized"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/0/30403247d1fe04ab6544e779ec84ecde5d27b6e3_2_690x183.png" alt="not_normalized" data-base62-sha1="6SQsrruGJXdFvDc9pvbXKbSzV8D" width="690" height="183" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/0/30403247d1fe04ab6544e779ec84ecde5d27b6e3_2_690x183.png, https://discuss.pytorch.org/uploads/default/original/3X/3/0/30403247d1fe04ab6544e779ec84ecde5d27b6e3.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/0/30403247d1fe04ab6544e779ec84ecde5d27b6e3.png 2x" data-dominant-color="8E6B66"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">not_normalized</span><span class="informations">840×223 214 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Below is the normalized one, which is not well perceivable visually. Then how is it helping CNN?<br>
<a href="/uploads/short-url/w71ByFCARuRcJsqbA0K4td05NzI.png">normalized|690x195</a></p>
<p>How do we choose the values for normalize?</p>
<p><code>transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)</code></p>

username: Rituraj_19_Dutta_100
cooked: <p>First of all Regularization and Normalization are two different terms. Regularization is used to prevent overfitting by either knocking out few parameters to zero(L1) or bringing some parameters very close to being zero(L2). Normalization is used to bring the data to desired centered mean( Gaussian Distribution). Normalization basically bounds the data to be in a specific distribution rather than having outliers and other stuff which in turns helps your model to learn the parameters efficiently.</p>

username: aleemsidra
cooked: <p>Actually, my question is that how the normalized version is better for CNN as compared non-normalized version? As the visually non-normalized version is more perceivable.</p>

username: Rituraj_19_Dutta_100
cooked: <p>Normalization actually makes the data have a mean of 0 and a Standard deviation of 1. Most of the times normalization aids in speeding up the training process by making the model converge faster. You can even train in your network without normalization but normalization is highly recommended.</p>
Cross-Compiling PyTorch for Android : undefined symbol: torch::cuda::is_available()

username: madbats
cooked: <p>I’m trying to cross-compile using AndroidNDK22 on Ubuntu 20.04.<br>
I start by compiling 2 cpp files that include &lt;torch/torch.h&gt; using <code>clang++ -isystem /opt/libtorch_android/headers/ -isystem /opt/libtorch_android/headers/torch/csrc/api/include/</code> (with other operators obviously), this goes well. However when I try to link the rest of my program into a .so and add <code>-L/opt/libtorch_android/jni/x86_64 -ltorch -ltorch_cpu -lc10</code>, I get these errors:</p>
<pre><code class="lang-auto">ld: error: undefined symbol: torch::cuda::is_available()
ld: error: undefined symbol: torch::nn::Module::named_buffers(bool) const
ld: error: undefined symbol: torch::nn::Module::named_parameters(bool) const
ld: error: undefined symbol: torch::nn::Module::eval()
ld: error: undefined symbol: torch::nn::EmbeddingImpl::forward(at::Tensor const&amp;)
ld: error: undefined symbol: torch::nn::detail::RNNImplBase&lt;torch::nn::LSTMImpl&gt;::flatten_parameters()
ld: error: undefined symbol: torch::nn::LSTMImpl::forward(at::Tensor const&amp;, c10::optional&lt;std::__ndk1::tuple&lt;at::Tensor, at::Tensor&gt; &gt;)
ld: error: undefined symbol: typeinfo for torch::nn::Module
[And many more]
</code></pre>
<p>The compiler I’m using is <code>${ANDROID_NDK}/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ -target x86_64-none-linux-android${ANDROID_API} --sysroot=${ANDROID_NDK}/toolchains/llvm/prebuilt/linux-x86_64/sysroot </code><br>
The pytorch libraries I’m using come from the /script/build_pytorch_android.sh<br>
Ultimately I run : <code>${ANDROID_NDK}/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ -o *MYOUTPUT.so* [MYCOMPILEDOBJECTS.o] s -Wl,-soname,$(@F) -shared ${STL_LIB} -flto -no-canonical-prefixes -target x86_64-none-linux-android${ANDROID_API} -Wl,--build-id=sha1 -Wl,--no-rosegment -nostdlib++ -Wl,--no-undefined -Wl,--fatal-warnings -lgcc -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libgcc_real.a -latomic -Wl,--exclude-libs,libatomic.a -ldl -lc -lm -mfma -mavx2 -arch x86_64 -v -DANDROID_STL=c++_shared -isystem /opt/libtorch_android/headers -isystem /opt/libtorch_android/headers/torch/csrc/api/include -ltorch -ltorch_cpu -lc10  -Wl,-rpath,/opt/libtorch_android/jni/x86_64 -L/opt/libtorch_android/jni/x86_64 -L/lib64 </code></p>
<p>Does anyone have any ideas as to how to solve these errors ?</p>

username: madbats
cooked: <p>After looking up the symbol’s in my libtorch_cpu.so file it would seem that <code>torch::cuda::is_available()</code> and the other symbols are never exported when <code>build_pytorch_android.sh</code> is run.<br>
Furthermore: <code>${PyTorch_Source}/torch/csrc/api/src/cuda.cpp</code> which declares this <code>is_available()</code> function is never compiled.</p>
Different loss and performance when training an autoencoder on horizontally flipped data

username: Atlas00
cooked: <p>Hello, I’m currently training an autoencoder on a dataset consisting of images of a circular object. This object should possess mirror symmetries and rotational symmetries. In order to debug my issues with augmentations, I tried training on data that had been horizontally flipped. Despite the object being a circular object, the loss takes a distinctly different trajectory during training (this is repeatable) and results in different performance. Any ideas what might be causing this or how I might go about diagnosing this? I feel understanding this is a prerequisite to using augmentations. Thank you!</p>

username: InnovArul
cooked: <p>Would it be possible to show some example images/a script to look at and reproduce?</p>
Torch to onnx conversion

username: adityat
cooked: <p>while using torch.export.onnx(…) [<a href="https://pytorch.org/docs/stable/onnx.html" class="inline-onebox" rel="noopener nofollow ugc">torch.onnx — PyTorch 1.9.0 documentation</a>]<br>
is there a way to get all the tensors along with the jit trace? this is primarily for layerwise validation for the model conversion …</p>
System of linear equations, find x,y

username: Samuel_Bachorik
cooked: <p>Hi my question is pure mathematical problem, this is my code that finds x and y of System of linear equations. In this case, for this easy example equation:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/f/d/fd87332c2cac97979fa0cd363bde0200855db522.png" alt="image" data-base62-sha1="AaOsNnlGT2eZCGBrICig1HgJCtc" width="205" height="88"></p>
<p>My question is- How can I transform equations like these to from Ax - b = 0 and then find x,y with Pytorch</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/b/3bdc1ea3d9e43bfc3ba161de6444655d9a29c26b.png" data-download-href="https://discuss.pytorch.org/uploads/default/3bdc1ea3d9e43bfc3ba161de6444655d9a29c26b" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/b/3bdc1ea3d9e43bfc3ba161de6444655d9a29c26b_2_690x250.png" alt="image" data-base62-sha1="8xxM398jNyWCjnGGLbzLoEP2KNR" width="690" height="250" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/b/3bdc1ea3d9e43bfc3ba161de6444655d9a29c26b_2_690x250.png, https://discuss.pytorch.org/uploads/default/original/3X/3/b/3bdc1ea3d9e43bfc3ba161de6444655d9a29c26b.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/b/3bdc1ea3d9e43bfc3ba161de6444655d9a29c26b.png 2x" data-dominant-color="FAFAFA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">867×315 15.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Equations like these to from Ax - b = 0</p>
<pre><code class="lang-auto">x = torch.nn.Parameter (torch.zeros (2))
print(x.shape)
print(x)

def model(x):
    k = x[0] + x[1] -5
    z = x[0] - x[1] -1

    p =  torch.tensor([k, z])
    return k,z

optimizer = torch.optim.Adam([x], lr=0.1)

for i in range(500):
    target = torch.nn.Parameter (torch.zeros (2))

    k,z = model(x)
    loss = (0 - k)**2 + (0 - z)**2
    print(loss,loss)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()


print(x[0],x[1], "X Y ")



</code></pre>

username: JuanFMontesinos
cooked: <p>I think if you understand the meaning of Ax=b it’s straightforward<br>
Basically you have to figure out A and b<br>
for a)<br>
A=(1,1)<br>
(1,-1)<br>
and b = (5,1)<br>
in the example<br>
b) 2x+2=x-y → x+y=-2<br>
then<br>
A = (1,1)<br>
(3,2)<br>
b=(-2,0)</p>
<p>Obtaining A and b allow you to rewrite the problem in a matricial way reusing the code.</p>

username: Samuel_Bachorik
cooked: <p>Thank you I understand now</p>
How to load the fc weights into its corresponding convolution layer weights?

username: fanq15
cooked: <p>I know how to convert the fc layer to convolution layer.<br>
But its weights are initialized randomly.<br>
I want to load the pretrained fc weights to the convolution layer transformed from this fc layer.<br>
How should I do?</p>

username: aitutakiv
cooked: <p>For the FC weights, you can use <code>view</code>. For example, let <code>old_fc6_weight</code> be the fc6 weight loaded from the <code>state_dict</code> of the pre-trained model:<br>
<code>new_fc6_param.data = old_fc6_weight.view(new_fc6_param.size())</code></p>

username: fanq15
cooked: <p>Thank you for your kind answer.<br>
It works!<br>
For convenient reference for others, I post my concrete code here.</p>
<pre><code class="lang-auto">self.fc6_to_conv = nn.Conv2d(512, 4096, kernel_size = 7, stride = 1, padding = 0)
self.fc6_to_conv.weight.data = vgg16.fc6.weight.data.view(self.fc6_to_conv.weight.size())
</code></pre>

username: fanq15
cooked: <p>However, it seems difficult to train.<br>
An I find that it’s not necessary to train with converted conv layer.<br>
I just need to convert fc layer to conv layer when testing. Am I right?</p>

username: aitutakiv
cooked: <p>If your output is spatial (as in an FCN), then the fully connected layers must be made fully convolutional in order to preserve spatial dimensions (during training and testing).</p>

username: sandipan1
cooked: <p>Does loading the pretrained fc weights to weights to conv layers work?  Seems unintuitive</p>
Binary classification- target size and input size

username: cekik
cooked: <p>I followed the same code, provided in <a href="https://towardsdatascience.com/pytorch-tabular-binary-classification-a0368da5bb89" class="inline-onebox" rel="noopener nofollow ugc">PyTorch [Tabular] — Binary Classification | by Akshaj Verma | Towards Data Science</a> for my dataset.</p>
<p>here is the model<br>
class binaryClassification(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(binaryClassification, self).<strong>init</strong>()<br>
# Number of input features is 25<br>
self.layer_1 = nn.Linear(25, 64)<br>
self.layer_2 = nn.Linear(64, 64)<br>
self.layer_out = nn.Linear(64, 1)</p>
<pre><code>    self.relu = nn.ReLU()
    self.dropout = nn.Dropout(p=0.1)
    self.batchnorm1 = nn.BatchNorm1d(64)
    self.batchnorm2 = nn.BatchNorm1d(64)
    
def forward(self, inputs):
    x = self.relu(self.layer_1(inputs))
    x = self.batchnorm1(x)
    x = self.relu(self.layer_2(x))
    x = self.batchnorm2(x)
    x = self.dropout(x)
    x = self.layer_out(x)
    
    return x
</code></pre>
<p>model.train()<br>
for e in range(1, EPOCHS+1):<br>
epoch_loss = 0<br>
epoch_acc = 0<br>
for X_batch, y_batch in train_loader:<br>
X_batch, y_batch = X_batch.to(device), y_batch.to(device)<br>
optimizer.zero_grad()</p>
<pre><code>    y_pred = model(X_batch)
    
    loss = criterion(y_pred, y_batch.unsqueeze(1))
    acc = binary_acc(y_pred, y_batch.unsqueeze(1))
    
    loss.backward()
    optimizer.step()
    
    epoch_loss += loss.item()
    epoch_acc += acc.item()
    

print(f'Epoch {e+0:03}: | Loss: {epoch_loss/len(train_loader):.5f} | Acc: {epoch_acc/len(train_loader):.3f}')
</code></pre>
<p>at  the end it gives following error:</p>
<p>Target size [64,1,1] should be the same with inpur size  [64,1]</p>
<p>what is the issue?</p>
Why transforms.ToTensor() behaves as a True when used in conditional statements?

username: Balaji_AI
cooked: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/e/ce95e013a6a3aec28416b81c5578b97477f63c09.png" alt="Screenshot 2021-07-27 at 09-30-09 Docking ISS - Jupyter Notebook" data-base62-sha1="ttxsQsQkVtkYjcWFC8wSywfQ38Z" width="604" height="69"></p>

username: ptrblck
cooked: <p>I don’t think this behavior is specific to <code>transforms.ToTensor()</code>, but it seems that each valid object would return <code>True</code> in this condition:</p>
<pre><code class="lang-python">if nn.Linear(1, 1):
    print('True')

if logging.Logger(name='test'):
    print('True')
</code></pre>

username: Balaji_AI
cooked: <p>Oh. Thank you so much.<br>
Can I know why?</p>

username: ptrblck
cooked: <p>I think it’s used to check, if an object is initialized, e.g. if it’s optional such as a transformation.<br>
You would often see Python code as:</p>
<pre><code class="lang-python">def __init__(self, transform=None): # initialize transform as None
    self.transform = transform

def __getitem__(self, x):
    if self.transform:
        x = self.transform(x)
</code></pre>
<p>and could use it with any other Python object as well.</p>

username: Balaji_AI
cooked: <p>Thank you so much <a class="mention" href="/u/ptrblck">@ptrblck</a>.</p>
How to implement a structure like GRU?

username: 335621
cooked: <p>Hi~<br>
I have a model A which is pretrained. This model A will take x_{t-1} and p_{t} to predict x_{t}. Since it is actually a PyTorch neural network it is differentiable.<br>
What I want to do is I want this model to move along P ={p_i} (a sequence predicted from last layer) and predict X = {x_i}.<br>
However, to predict x_{t} this model needs to take the output of the last frame x_{t-1}. Thus, I implemented this structure using a for loop which makes the process quite slow…</p>
<p>Is there a way to accelerate the process?</p>
Dynamic linear layer doesn't change output when weights shape is changed

username: JosueCom
cooked: <p>If I implement a dynamic layer as present below, it’s output shape doesn’t get affected. See code below for greater details.</p>
<pre><code class="lang-python">&gt;&gt;&gt; import torch as th
&gt;&gt;&gt; import torch.nn as nn
&gt;&gt;&gt; import torch.nn.functional as F
&gt;&gt;&gt; class DyLinear(nn.Module):
...     def __init__(self, in_fea, out_fea, device = 'cuda'):
...         super(DyLinear, self).__init__()
...         self.device = device
...         self.in_fea = in_fea
...         self.out_fea = out_fea
...         self.weight = nn.Parameter(th.randn(out_fea, in_fea)).to(self.device)
...     def forward(self, x):
...         x = F.linear(x, self.weight)
...         return x
...     def update_size(self, size): #slow; doesn't matter much since perform infrequenly
...         if size &lt;= self.out_fea:
...             raise ValueError(f"New size ({size}) needs to be bigger than current output size ({self.out_fea})")
...         else:
...             with th.no_grad():
...                 self.out_fea = size
...                 self.weight = nn.Parameter(th.cat((self.weight, th.randn(size - self.out_fea, self.in_fea).to(self.device)), 0)).to(self.device)
... 
&gt;&gt;&gt; a = th.rand(5)        
&gt;&gt;&gt; model = DyLinear(5, 5)
&gt;&gt;&gt; model(a)
tensor([ 8.9683e-42,  0.0000e+00, -3.2745e+27,  8.4919e-43,  0.0000e+00], grad_fn=&lt;SqueezeBackward3&gt;)
&gt;&gt;&gt; model.update_size(6) 
&gt;&gt;&gt; model(a)
tensor([2.1514e-34, 2.6429e-38, 1.4125e+14, 4.2039e-45, 1.4466e-36], grad_fn=&lt;SqueezeBackward3&gt;)
</code></pre>

username: ptrblck
cooked: <p>In your code you are setting <code>self.out_fea</code> to <code>size</code> before the actual creation of the new <code>nn.Parameter</code> so that <code>torch.randn</code> will yield an empty tensor (<code>size - self.out_fea</code> will be <code>0</code>).<br>
Move the update of <code>self.out_fea</code> after the new parameter assignment and it should work.</p>

username: JosueCom
cooked: <p>Ohhhh Thank you so much!!!</p>
Error when creating MaxPool

username: bucky
cooked: <p>assertion when trying to create a maxpool node, I have created a mock backend and tried this in the <code>transformPostLowering()</code> function</p>
<pre><code class="lang-auto">mnist: /home/bucky/Desktop/compiler/glow/lib/Graph/NodeValue.cpp:10: glow::NodeValue::NodeValue(qsnn::Node *): Assertion `(!N || (N-&gt;getNumResults() == 1)) &amp;&amp; "Constructing a value for a multi-res node"' failed.
Aborted (core dumped)

</code></pre>
<p>How can I over come this?</p>

username: jfix
cooked: <p>MaxPool has two outputs, <code>Result</code> and <code>Argmax</code>. For multi-output nodes you need to select a specific output when using it, for example <code>maxPoolNode-&gt;getResult()</code> or <code>maxPoolNode-&gt;getArgmax()</code>, instead of just using <code>maxPoolNode</code> as an input to another node.</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/glow/blob/d5b32ee8a6f29166219963623648f9b15f1f94b8/tools/ClassGen/NodeGen.cpp#L161-L162">
  <header class="source">

      <a href="https://github.com/pytorch/glow/blob/d5b32ee8a6f29166219963623648f9b15f1f94b8/tools/ClassGen/NodeGen.cpp#L161-L162" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/glow/blob/d5b32ee8a6f29166219963623648f9b15f1f94b8/tools/ClassGen/NodeGen.cpp#L161-L162" target="_blank" rel="noopener nofollow ugc">pytorch/glow/blob/d5b32ee8a6f29166219963623648f9b15f1f94b8/tools/ClassGen/NodeGen.cpp#L161-L162</a></h4>



  <pre class="onebox">    <code class="lang-cpp">
      <ol class="start lines" start="161" style="counter-reset: li-counter 160 ;">
          <li>.addResultFromCtorArg("Result")</li>
          <li>.addResultFromCtorArg("Argmax")</li>
      </ol>
    </code>
  </pre>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
How to build the network code definition from model.eval() output?

username: vickyx398
cooked: <p>Hi there,</p>
<p>I am new to pytorch and trying to understand.<br>
I have a pretrained model from a paper. The code in the paper is immense and hard to understand, how the network was actually defined.<br>
Still, I managed to load the pretrained model and evaluate it, so I got the following structure:</p>
<pre><code class="lang-auto">basic_conv1d(
  (0): Sequential(
    (0): Sequential(
      (0): Conv1d(12, 128, kernel_size=(8,), stride=(1,), padding=(3,), bias=False)
      (1): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU(inplace=True)
    )
  )
  (1): Sequential(
    (0): Sequential(
      (0): Conv1d(128, 256, kernel_size=(5,), stride=(1,), padding=(2,), bias=False)
      (1): BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU(inplace=True)
    )
  )
  (2): Sequential(
    (0): Sequential(
      (0): Conv1d(256, 128, kernel_size=(3,), stride=(1,), padding=(1,), bias=False)
      (1): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU(inplace=True)
    )
  )
  (3): Sequential(
    (0): AdaptiveConcatPool1d(
      (ap): AdaptiveAvgPool1d(output_size=1)
      (mp): AdaptiveMaxPool1d(output_size=1)
    )
    (1): Flatten()
    (2): BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=256, out_features=128, bias=True)
    (5): ReLU(inplace=True)
    (6): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (7): Dropout(p=0.5, inplace=False)
    (8): Linear(in_features=128, out_features=5, bias=True)
  )
)
</code></pre>
<p>I would love to define the network code myself, so I don’t have to work with the<br>
supplied and very incomprehensible code.</p>
<p>Is there some kind of parser, that takes such a network evaluation and builds the code definition?<br>
Or could anybody help me with this?</p>
<p>Thank you very much!!</p>

username: ptrblck
cooked: <p>I don’t think there is a parser, as the <code>forward</code> method is undefined.<br>
I.e. while the submodules are created, it’s unclear how they are used (e.g. in a sequential manner or are different paths used inside the <code>forward</code>).<br>
Assuming these layers are used in a sequential manner, you could recreate the modules by wrapping them in <code>nn.Sequential</code> containers, e.g.</p>
<pre><code class="lang-python">self.layer1 = nn.Sequential(
    nn.Conv1d(12, 128, 8, 1, 3, bias=False),
    nn.BatchNorm1d(128),
    nn.ReLU(inplace=True)
</code></pre>
<p>One unknown would be the <code>AdaptiveConcatPool1d</code>, which doesn’t seem to be a core PyTorch layer, but seems to be defined in FastAI.</p>
Dataloader with single image

username: aqib
cooked: <p>Hi all,</p>
<p>I am new to Pytorch and not sure if it is possible to create a DataLoader that can read and repeat the same image over and over. I have an example implementation in Tensorflow:</p>
<p>N = 1000<br>
img = tf.keras.preprocessing.image.load_img(path)<br>
tds = tf.data.Dataset.from_tensor_slices(img).repeat(N)</p>
<p>Thanks in advance.</p>

username: InnovArul
cooked: <p>You could try something as below:</p>
<pre><code class="lang-auto">import torch, torch.utils.data as data
x = torch.randn(3,25,25)
N = 10
ds = data.TensorDataset(x.unsqueeze(0).repeat(N,1,1,1))
for item in ds:
    img = item[0]
    print(img.shape)
</code></pre>

username: arya47
cooked: <p>You can also create a Dataset class which returns the same image, len of 1000 for 1000 instances and returns the same image in getitem</p>
<pre><code class="lang-auto">from torch.utils.data import Dataset
class data(Dataset):
    def __init__(self, image_path, transform=None):
        self.image_path = image_path
        self.transform = transform

    def __len__(self):
        return 1000

    def __getitem__(self, x):
        # open image here as PIL / numpy
        image = #load image here
        label = # if label
        if self.transform is not None:
            image = self.transform(image)

        return image, label
</code></pre>
<p>Then pass this dataset to the Dataloader</p>
Best way to understand last_epoch

username: deJQK
cooked: <p>There seems to be a transition for the convention of defining last_epoch in lr scheduler, starting from the version of 1.1.0. To my understanding, last_epoch should mean the epoch that has already been done (i.e. training finished), and for example, when I load a pretrained model to resume training, where it has trained for 100 epochs (from 0 to 99), the last epoch should be 99, and the new start one should be 100; and for a new model not trained before, since we need to start from epoch 0, last epoch should be -1 as -1 is the one already trained. This is the convention before (and including) version 1.0.1. See <a href="https://pytorch.org/docs/1.0.1/_modules/torch/optim/lr_scheduler.html" rel="noopener nofollow ugc">lr_scheduler_1.0.1</a>. However, since 1.1.0, during initialization, the step function is called after setting the last_epoch, which makes last_epoch to 0 instead of -1, see <a href="https://pytorch.org/docs/1.1.0/_modules/torch/optim/lr_scheduler.html" rel="noopener nofollow ugc">lr_scheduler_1.1.0</a>. I wonder the reason for this change and if there is any advantage of the modification compared to previous convention. Thanks!</p>
Convolution with empty values in tensor

username: mlizhardy
cooked: <p>I have a tensor which has empty values.  I want to apply a 2d blurring convolution kernel just to the non-empty values. For simplicity let’s assume kernel weights are all equal.</p>
<p>Maybe something like dynamic kernel weights, so kernel weights = 1/n, where n is the number of valid pixels in the region.</p>
<p>Is there currently pytorch functionality which supports something like this?</p>

username: JuanFMontesinos
cooked: <p>I think you can just apply a convolution using a mask. In the end a convolution is product + sum between the elements  the kernel and the tensor. If you just multiply your tensor by a binary mask (ones for the non-empty values and zeros for empty values) the result should be that.</p>
<p>Besides you have torch.fold and unfold which you can use to emulate an sliding kernel implementing ur own operation.<br>
<a href="https://pytorch.org/docs/stable/generated/torch.nn.Fold.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.Fold.html</a></p>
<p>But basically masking either the input or the output of the convolution should work.</p>
<p>Ooor maybe you can use sparse tensors (Not really aware how deep they have been implemented as i don’t use them).</p>
Need to resize my tensor on the current graph(backward)

username: tianle-BigRice
cooked: <p>The problem is that I need to resize my tensor on the current graph as shown below:</p>
<pre><code class="lang-auto">img = skimage.transform.resize(img, (C, H * scale, W * scale), mode='reflect',anti_aliasing=False)
</code></pre>
<p>How can I resize my tensor without detached the current graph. Or resize to the current graph after adjusting my tensor while detached the current graph.</p>

username: JuanFMontesinos
cooked: <p>You can use pytorch resize which should match that one.<br>
<a href="https://pytorch.org/docs/stable/generated/torch.nn.functional.interpolate.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.functional.interpolate.html</a></p>
<p>As a  layer:<br>
<a href="https://pytorch.org/docs/stable/generated/torch.nn.Upsample.html?highlight=resize" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.Upsample.html?highlight=resize</a></p>
Difference between loss.item() and loss.detach().cpu().numpy()

username: bibekx
cooked: <p>Can anyone please help me with the difference between:</p>
<p>loss.item() and loss.detach().cpu().numpy()</p>
<p>Which one should I use in the training loop ?<br>
Thanks</p>

username: JuanFMontesinos
cooked: <p>Well, one returns a python float and the other one a numpy array.<br>
It really depends on what do you want to use them for.</p>

username: bibekx
cooked: <p>Thanks for the reply.</p>
<p>If I have a batch size of 16, how will a python float (I am assuming that you meant a single number) help?</p>
<p>I want to collect loss over all the batches in each epoch and average them to get/ calculate “batch loss per epoch” after the end of epoch.</p>
<p>Would like to draw your attention, <a class="mention" href="/u/ptrblck">@ptrblck</a></p>

username: amirhf
cooked: <p>loss.item() will return a float that is the average of loss for all samples in your batch in a single iteration. If you want to average loss across all iterations or epochs, you have to implement a class that averages all your losses in iterations or epochs: something like this: <a href="https://github.com/pytorch/examples/blob/2639cf050493df9d3cbf065d45e6025733add0f4/imagenet/main.py#L268" class="inline-onebox" rel="noopener nofollow ugc">examples/main.py at 2639cf050493df9d3cbf065d45e6025733add0f4 · pytorch/examples · GitHub</a></p>

username: bibekx
cooked: <p>Thanks,<br>
I thought that a good measure to know that a model is actually learning something could be “average batch loss per epoch”</p>
<p>which could be calculated by collecting all the loss.item() in the epoch and then taking an average.</p>
<p>Also, seems like loss.detach().cpu().numpy() should not be used while training.</p>
How to get the module names of nn.Sequential

username: M_Tu
cooked: <p>For example,</p>
<pre><code class="lang-auto">model = nn.Sequential()
model.add_module('conv0', conv0)
model.add_module('norm0', norm0)
</code></pre>
<p>Is there a way to get the names of these added modules? Thanks</p>

username: ptrblck
cooked: <p>You could print all names and sub-modules using:</p>
<pre><code class="lang-python">for name, module in model.named_modules():
    print(name)
</code></pre>
<p>If you want to directly access these modules, you can just use:</p>
<pre><code class="lang-python">print(model.conv0)
</code></pre>

username: Brando_Miranda
cooked: <p>note that <code>model.children()</code> is not recursive as outlined by fmass in <a href="https://discuss.pytorch.org/t/module-children-vs-module-modules/4551/11" class="inline-onebox">Module.children() vs Module.modules()</a></p>

username: Sudhir_Deshmukh
cooked: <p>Hi,</p>
<p>i am getting following error:</p>
<p>TypeError: named_modules() missing 1 required positional argument: ‘self’</p>

username: ptrblck
cooked: <p>Did you override this method in your <code>nn.Model</code>? If not, could you post an executable code snippet to reproduce this issue, please?</p>
BatchNorm2d layer in DCGAN

username: Rituraj_19_Dutta_100
cooked: <p>I’ve implemented a Discriminator which uses Batch Normalization layers. But unfortunately, the discriminator loss is stuck and remains constant throughout but if I remove the batch normalization layer the discriminator performs well. Any suggestions as to what changes are necessary to include Batch Normalization layers in DCGAN.</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

class Flatten(torch.nn.Module):

    def forward(self, x):
        return x.view(x.size()[0], -1)

class Discriminator(nn.Module):

    def __init__(self):
        super(Discriminator, self).__init__()
       
        self.layer = nn.Sequential(

          ## Block 1
  
          nn.Conv2d(in_channels=5,out_channels=32,kernel_size=(3,3),padding='same'),
          #nn.BatchNorm2d(32,0.8),
          nn.ReLU(inplace=False),

          nn.Conv2d(in_channels=32,out_channels=32,kernel_size=(3,3),padding='same'),
          #nn.BatchNorm2d(32,0.8),
          nn.ReLU(inplace=False),

          nn.MaxPool2d(kernel_size=(2,2)),
          
          ## Block 2

          nn.Conv2d(in_channels=32,out_channels=64,kernel_size=(3,3),padding='same'),
          #nn.BatchNorm2d(64,0.8),
          nn.ReLU(inplace=False),

          nn.Conv2d(in_channels=64,out_channels=64,kernel_size=(3,3),padding='same'),
          #nn.BatchNorm2d(64,0.8),
          nn.ReLU(inplace=False),

          nn.MaxPool2d(kernel_size=(2,2)),
          
          ## Block 3

          nn.Conv2d(in_channels=64,out_channels=64,kernel_size=(3,3),padding='same'),
          #nn.BatchNorm2d(64,0.8),
          nn.ReLU(inplace=False),

          nn.Conv2d(in_channels=64,out_channels=64,kernel_size=(3,3),padding='same'),
          #nn.BatchNorm2d(64,0.8),
          nn.ReLU(inplace=False),

          nn.MaxPool2d(kernel_size=(2,2)),  
      


        )
        self.flatten = Flatten()
        self.final = nn.Sequential(
            nn.Linear(50176,100),
            nn.Linear(100,1),

            nn.Sigmoid()  
        )

    def forward(self, input):

      '''Forward pass; map samples to confidence they are real [0, 1]'''
      x = self.layer(input)
      x = self.flatten(x)
      x = self.final(x)


      return x
</code></pre>
Training with multiple neural nets

username: marcus_pazini
cooked: <p>I’m fairly new to pytorch and currently i would like to train a model wich consists in the following:<br>
I have one already trained model (net1) wich receives as an input the output of a untrained model (net2), and generates on output. (net2 → net1 → output)<br>
Then my question the is, is it possible compute the gradient through net1 and net2 but only run an optimizer.step() and update the weights of net2?</p>

username: ptrblck
cooked: <p>Yes, that’s possible and you could directly pass the output of <code>net2</code> to <code>net1</code>, compute the loss, and call <code>loss.backward()</code> to compute the gradients.<br>
If you don’t want to train <code>net1</code>, you could freeze all parameters of it via:</p>
<pre><code class="lang-python">for param in net1.parameters():
    param.requires_grad = False
</code></pre>
<p>To update the parameters of <code>net2</code>, pass its parameters to the optimizer via:</p>
<pre><code class="lang-python">optimizer = torch.optim.SGD(net2.parameters(), lr=1e-3)
</code></pre>

username: marcus_pazini
cooked: <p>Thank you for the quick answer, I  have modified my code so it follows the instructions you suggested, is it correct?</p>
<pre><code class="lang-auto">for param in net1.parametsrs():
     param.requires_grad = False

optimizer = torch.optim.SGD(net2.parameters(), lr=1e-3)

for i in torch.randperm(len(train_inputs)):
     x = train_inputs[i]
     y_hat = net2(net1(x))
     loss = criterion(y[i], y_hat)
     optimizer.zero_grad()
     loss.backward()
     optimizer.step()
</code></pre>

username: ptrblck
cooked: <p>The code looks generally alright, but you’ve switched the model order.<br>
In your initial post you’ve claimed <code>net2 -&gt; net1 -&gt; output</code>, while you are now using <code>net1 -&gt; net2 -&gt; output</code>.<br>
Also, I don’t know which criterion you are using, but often the model output is the first argument and the target the second, so you might want to switch this as well.</p>

username: marcus_pazini
cooked: <p>Yes, I switched the order the second time, sorry for my mistake! <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=9" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"><br>
My criterion is the MSELoss wich I use to compute the loss with the chained model output.<br>
It seems to be working allwright so far!</p>
<pre><code class="lang-auto">criterion = nn.MSELoss()
</code></pre>
Reusing optimizer after model deletion

username: omarfoq
cooked: <p>Hello,</p>
<p>Suppose I have some <code>model</code> (<code>nn.Module</code>) and an associated <code>optimizer</code> to <code>model</code>, e.g., <code>model = optim.SGD([param for param in model.parameters()], lr=LR)</code> and an <code>lr_scheduler=optim.lr_scheduler.MultiStepLR(optimizer, milestones=MILESTONES, gamma=GAMMA)</code></p>
<p>Suppose I want to free memory for some time, to do some other tasks (e.g., train some other models), then reload the same <code>model</code> to the memory while keeping track of  <code>optimizer</code> and <code>lr_scheduler</code> state.</p>
<p>My questions are</p>
<ul>
<li>what is the best way to delete <code>model</code> is it <code>del model</code> or should I do do for all <code>param in model.parameters()</code>?</li>
<li>if I delete the model, the values of its parameters are still held by the optimizer, so I need to delete them also from the optimizer, in that case I will need to delete those also to free memory, what is the correct way of doing this, is it to delete all the parameters  for <code>param_group in optimizer.param_groups</code>?</li>
<li>How to relaod the recreate the model when needed, should I recreate a new <code>optimizer</code> and a new <code>lr_scheduler</code> or can I just add param_groups to to the <code>optimizer</code>?</li>
</ul>
How to drop layers from a custom PyTorch model

username: sidu28
cooked: <p>I have a custom model that I’m using, from this <a href="https://github.com/cattaneod/CMRNet" rel="noopener nofollow ugc">git repo</a></p>
<p>Here is the basic architecture (there are some additional conv layers):</p>
<pre><code class="lang-auto">  (lconv6aa): Sequential(
    (0): Conv2d(128, 196, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
    (1): LeakyReLU(negative_slope=0.1)
  )
  (lconv6a): Sequential(
    (0): Conv2d(196, 196, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): LeakyReLU(negative_slope=0.1)
  )
  (lconv6b): Sequential(
    (0): Conv2d(196, 196, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): LeakyReLU(negative_slope=0.1)
  )
  (corr): Correlation()
  (leakyRELU): LeakyReLU(negative_slope=0.1)
  (conv6_0): Sequential(
    (0): Conv2d(81, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): LeakyReLU(negative_slope=0.1)
  )
  (conv6_1): Sequential(
    (0): Conv2d(209, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): LeakyReLU(negative_slope=0.1)
  )
  (conv6_2): Sequential(
    (0): Conv2d(337, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): LeakyReLU(negative_slope=0.1)
  )
  (conv6_3): Sequential(
    (0): Conv2d(433, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): LeakyReLU(negative_slope=0.1)
  )
  (conv6_4): Sequential(
    (0): Conv2d(497, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): LeakyReLU(negative_slope=0.1)
  )
  (fc1): Linear(in_features=529, out_features=512, bias=True)
  (fc1_trasl): Linear(in_features=512, out_features=256, bias=True)
  (fc1_rot): Linear(in_features=512, out_features=256, bias=True)
  (fc2_trasl): Linear(in_features=256, out_features=3, bias=True)
  (fc2_rot): Linear(in_features=256, out_features=4, bias=True)
  (dropout): Dropout(p=0.0)
</code></pre>
<p>The model is running with pre-trained weights, and I want to replace these last fc layers with my own.  Is there some way I can drop those layers, and replace it with my own custom ones?  I’m a beginner in pytorch and wasn’t able to figure this out based on the posts that I found here.  Any help would be appreciated</p>

username: ptrblck
cooked: <p>Yes, you can directly assign new modules to the model via e.g.:</p>
<pre><code class="lang-python">model.fc1 = nn.Linear(...)
</code></pre>
<p>to replace the linear layer assigned to <code>fc1</code>.</p>

username: sidu28
cooked: <p>Thank you!  A quick followup - on PyTorch versions before 1.1.0, is there an alternative for torch.nn.Identity.  I want to replace some of these fc layers with dummies if possible.  Thank you! I am constrained to using an older version of torch (1.0.2)</p>

username: ptrblck
cooked: <p>You could write a custom layer, which would also just forward the input as:</p>
<pre><code class="lang-python">class Identity(nn.Module):
    def __init__(self):
        super().__init__()
        
    def forward(self, x):
        return x
</code></pre>
Unet segmentation - Maximum amount of classes

username: Caipi
cooked: <p>Hello there,<br>
do you guys know some good paper or a source researching how many classes should be used as max in Unet segmentation tasks? Like is there a limit when in the last 1x1 conv. layer of you you push the channels up to 1000 classes?</p>
<p>Or is there maybe some advice to use layers in between, to not directly increase the channels this much?</p>
<p>As a reminder, heres Unet:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/a/4/a49f0d8bf998728fa1fe5b98d9c93e3a9e2d323e.png" alt="Download" data-base62-sha1="nuj4CdaqJ4FZBgna9A4qs6PnmtE" width="275" height="183"></p>
Torch::linalg::linalg_norm() usage

username: Hiperdyne19012
cooked: <p>Using torch::linalg::linalg_norm() like torch::linalg::linalg_norm(tensor, 2, 2) gives this error<br>
<code>error: no matching function for call to ‘linalg_norm(at::Tensor&amp;, int, int)’</code></p>
<p>I want to do this</p>
<blockquote>
<p>np.linalg.norm(diff, ord=2, axis=2)</p>
</blockquote>
<p>in Libtorch.</p>

username: ptrblck
cooked: <p><a href="https://github.com/pytorch/pytorch/blob/548765d9a5b377928afd0512b8f3b083ae6a3fc9/aten/src/ATen/native/LinearAlgebra.cpp#L2245-L2251"><code>torch::linalg_norm</code></a> is defined as:</p>
<pre><code class="lang-auto">Tensor linalg_norm(const Tensor&amp; self, const optional&lt;Scalar&gt;&amp; opt_ord, optional&lt;IntArrayRef&gt; opt_dim, bool keepdim, optional&lt;ScalarType&gt; opt_dtype)
</code></pre>
<p>so you might need to change the input arguments.</p>

username: Hiperdyne19012
cooked: <p>I also did <code>torch::linalg::linalg_norm(tensor, 2, {2}, false);</code> I did not work.</p>
<p>Then tried</p>
<pre><code class="lang-auto">      int64_t arr[1] = {2};
      c10::IntArrayRef dim_arr = c10::IntArrayRef(arr, 1);
      result = torch::linalg::linalg_norm(
          tensor, 2, dim_arr, false, torch::dtype(torch::kFloat));
</code></pre>
<p>Still same error.</p>

username: Lezcano
cooked: <p>These functions are defined <a href="https://github.com/pytorch/pytorch/blob/548765d9a5b377928afd0512b8f3b083ae6a3fc9/torch/csrc/api/include/torch/linalg.h" rel="noopener nofollow ugc">here</a>.</p>
<p>You might want to use in this case <code>linalg::vector_norm</code>. It has signature:</p>
<pre><code class="lang-auto">Tensor vector_norm(const Tensor&amp; self, Scalar ord, optional&lt;IntArrayRef&gt; opt_dim, bool keepdim, optional&lt;ScalarType&gt; opt_dtype)
</code></pre>
<p>Resulting in</p>
<pre><code class="lang-auto">result = torch::linalg::vector_norm(tensor, 2, 2, false, c10::nullopt);
</code></pre>
How does index_put_/copy work under the hood?

username: F10w10
cooked: <p>I am following the <code>index_put_</code> operation through the code, as it would be applied in this case:</p>
<pre><code class="lang-auto">using namespace torch::indexing;
auto tOne = torch::ones(new long[] {10, 100, 100, 100});
auto tZero = torch::zeros(new long[] {10, 100, 100, 100});
tZero.index_put_({ "...", Slice(0, 50), Slice(0, 50), Slice(0, 50) },  tOne .index({ "...", 50, 50, 50 }));
</code></pre>
<p>I find it implemented <a href="https://github.com/pytorch/pytorch/blob/42d6543c7b1aacdf1ce7b26d531b93f4ca62ae71/aten/src/ATen/TensorIndexing.cpp#L75" rel="noopener nofollow ugc">in aten/src/ATen/TensorIndexing.cpp</a>:</p>
<pre><code class="lang-auto">Tensor &amp; Tensor::index_put_(ArrayRef&lt;at::indexing::TensorIndex&gt; indices, Tensor const &amp; rhs) {
  TORCH_CHECK(indices.size() &gt; 0, "Passing an empty index list to Tensor::index_put_() is not valid syntax");
  OptionalDeviceGuard device_guard(device_of(*this));
  at::indexing::set_item(*this, indices, rhs);
  return *this;
}
</code></pre>
<p>Which uses <code>set_item</code>, defined <a href="https://github.com/pytorch/pytorch/blob/42d6543c7b1aacdf1ce7b26d531b93f4ca62ae71/aten/src/ATen/TensorIndexing.h#L544" rel="noopener nofollow ugc">in aten/src/ATen/TensorIndexing.h</a></p>
<p>If I see it correctly, it applies the different strides, i.e. creating new views of the existing tensor by fidling with stride etc, but not actually fiddling with the actual storage underneath:</p>
<pre><code class="lang-auto">Tensor sliced = impl::applySlicing(self, indices, tensorIndices, disable_slice_optimization, self_device, self_sizes);
</code></pre>
<p>And then a <code>copy_to</code> is <a href="https://github.com/pytorch/pytorch/blob/42d6543c7b1aacdf1ce7b26d531b93f4ca62ae71/aten/src/ATen/TensorIndexing.h#L580" rel="noopener nofollow ugc">performed</a>:</p>
<pre><code class="lang-auto">  if (tensorIndices.empty()) {
    copy_to(sliced, value);
    return;
  }
</code></pre>
<p>I tried to follow that but could not follow allong. I am wondering how it eventually works that we have a view of a tensor, i.e. it is not continuous in memory and we then perform a copy operation. Does it boil down to element wise copy or are there further optimizations involved? Where is it implemented?</p>
RAM exhausted while training on GPU

username: Fanny
cooked: <p>Hi everyone,<br>
I’m trying to train an instance segmentation model on GPU based on this pytorch tutorial <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html" rel="noopener nofollow ugc">https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html</a>.</p>
<p>However, after a few epochs (3 with a batch_size of 1, 6 with a batch_size of 2), all of the available RAM is used and the training is stopped.</p>
<p>How can that be fixed ?</p>
<p>I have a GPU with 16Go memory, my PC has 32Go RAM and I try to train the model with ~5000 images.</p>
<p>It seems that the rise in RAM usage is happening during losses.backward() in the train_one_epoch function (available <a href="https://github.com/pytorch/vision/blob/master/references/detection/engine.py" rel="noopener nofollow ugc">here</a>).</p>
<p>Thanks.</p>

username: JuanFMontesinos
cooked: <p>Can you try to remove<br>
<code>metric_logger.update(loss=losses_reduced, **loss_dict_reduced)</code><br>
It seems a tensor which carries the whole graph hist.<br>
If you want to log the loss  log this one <code>loss_value = losses_reduced.item()</code> which is a python float.</p>

username: Fanny
cooked: <p>Thanks for the answer, I tried but it didn’t change anything.</p>

username: JuanFMontesinos
cooked: <p>Then it’s difficult to know.<br>
I would recommend you to remove everything but the basics and from there keep adding stuff until you find it</p>

username: Fanny
cooked: <p>Thanks.<br>
Just like I said, when I comment out the line <code>losses.backward()</code>, the RAM usage is constant over time.</p>
Question Answering - how to get Context in real world applications?

username: newbieee
cooked: <p>Hello Guys,</p>
<p>how do Voice Assistant Devices answer Questions? If we Take a look at SQuaD and other QA Datasets we always get a Question and the Context to answer Questions. But how would this work in the real world?<br>
Would i just parse the Question and use the Wiki Article as a Context or are QA Systems not Suitable for this kinda task?</p>
<p>Ty in advcanded!</p>
jit::script::Module::farward usage stopped compiling when upgraded to libtorch 1.9.0

username: Silvina
cooked: <p>Hi,<br>
I have code that used to compile and run before I upgraded to libtorch 1.9.0.<br>
Now I get an error that leads to:<br>
C10_DEPRECATED_MESSAGE(<br>
“IValues based on std::unordered_map&lt;K, V&gt; are slow and deprecated. Please use c10::Dict&lt;K, V&gt; instead.”)<br>
/// \endcond<br>
IValue(std::unordered_map&lt;Key, Value&gt; v);</p>
<p>The code:<br>
//In h file<br>
torch::jit::script::Module m_module;<br>
//In cpp file<br>
torch::Tensor tensor = torch::unsqueeze(inputPatch, 0);<br>
std::vector<a>torch::jit::IValue</a> inputs;<br>
inputs.push_back(tensor);<br>
torch::Tensor out = m_module.forward(inputs).toTensor();</p>
<p>What needs to change in my code to make it compatible with libtorch 1.9.0 ?</p>
How to implement Squared-Loss Penalty so that it runs fast and for different parameters ranges in PyTorch?

username: TheUnnamed22
cooked: <p>How to implement the Squared-Loss Penalty so that is runs fast? According to this thesis?</p>
<p><a href="https://wiredspace.wits.ac.za/bitstream/handle/10539/29568/cbester_msc_dissertation.pdf?sequence=1&amp;isAllowed=y" rel="noopener nofollow ugc">Multipass Deep Q-Networks (wits.ac.za)</a></p>