akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Cricket need help to understand

username: Raidon5555
cooked: <p>Hello can anyone help me? I am starting PyTorch and i dont know to explain this code:</p>
<p>the batch-size is 32</p>
<p>self.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1)<br>
self.relu1 = nn.ReLU()<br>
self.conv2 = nn.Conv2d(64, 64, kernel_size=3, stride=1)</p>
<p>How many neurons has the input layer, hidden layer and the output layer? And what is the size of the hidden layer?</p>
<p>Everyone answer is welcome</p>

username: Raidon5555
cooked: <p>Can anyone help me pls?</p>

username: Sanjan_Das
cooked: <p>Perhaps you may get a much better answer (and understanding) by looking at some videos on CNN basics (one here: <a href="https://www.youtube.com/watch?v=m8pOnJxOcqY" class="inline-onebox" rel="noopener nofollow ugc">Convolution Ne</a>).</p>
<p>The model you’ve mentioned here seems incomplete for any deep learning task. Note that your input and output layer is largely defined by what you want to achieve. It may be more efficient if you have a go at tutorials and beginner resources on CNNs available online first! (there are plenty out there)</p>
I am getting this error : mat1 and mat2 shapes cannot be multiplied (20x10 and 20x512)

username: sam101
cooked: <p>latent_size = 20</p>
<p>encoder = ConditionalNormal(MLP(3000, latent_size,<br>
hidden_units=[512,256],<br>
activation=‘relu’,<br>
in_lambda=lambda x:  x.view(x.shape[0], 3000).float() - 1))<br>
decoder = ConditionalNormal(MLP(latent_size, 3000,<br>
hidden_units=[512,256],<br>
activation=‘relu’,<br>
out_lambda=lambda x:  x.view(x.shape[0], 3000).float() - 1))</p>
<p>model = Flow(base_dist=StandardNormal((latent_size,)),<br>
transforms=[<br>
VAE(encoder=encoder, decoder=decoder)<br>
]).to(device)</p>

username: albanD
cooked: <p>Hi,</p>
<p>Your code is pretty hard to read, you can use triple backticks ``` to nicely format it (like you would do on github).</p>
<p>And the error happens because the second dimension of the first matrix must match the first dimension of the second, but here you have 10 and 20. I guess you want to transpose the first matrix?</p>

username: sam101
cooked: <p>Hi Alban, sorry for this. Here is the updated code.</p>
<pre><code class="lang-auto">encoder = StandardNormal(MLP(3000, latent_size,
                                hidden_units=[512,256],
                                activation='relu',
                                x =  x.view(x.shape[0], 3000).float() - 1))
decoder = StandardNormal(MLP(latent_size, 3000,
                                   hidden_units=[512,256],
                                   activation='relu',
                                   x =  x.view(x.shape[0], 3000).float() - 1))

model = Flow(base_dist=StandardNormal((latent_size,)),
             transforms=[
                VAE(encoder=encoder, decoder=decoder)
             ]).to(device)
</code></pre>
<p>No I have input matrix to the encoder with 3000 dimensions and I want to get back the 3000 dimensions from the decoder.</p>
AttributeError: 'tuple' object has no attribute 'cpu'

username: Najeh_Nafti
cooked: <p>Help please, i’m trying to generate the file.npz but i got this error</p>
<p>File “calculate_inception_moments.py”, line 146, in run<br>
labels += [np.asarray(y.cpu())]<br>
AttributeError: ‘tuple’ object has no attribute ‘cpu’<br>
0% 0/1 [00:15&lt;?, ?it/s]</p>
<pre><code class="lang-auto">  with torch.no_grad():
        pool_val, logits_val = net(x)
        pool += [np.asarray(pool_val.cpu())]
        logits += [np.asarray(F.softmax(logits_val, 1).cpu())]
        labels += [np.asarray(y.cpu())]

</code></pre>

username: cpeters
cooked: <p>What is Y? It’s quite possible that it doesn’t need to be moved onto the CPU in the first place.</p>

username: Najeh_Nafti
cooked: <p>yes thank you i got it</p>

username: Najeh_Nafti
cooked: <p>I got this error now:</p>
<p>File “/content/drive/.shortcut-targets-by-id/1EDEEgDBcJ3ekzn4l1EeLMvH0obpYsF1y/unetgan/utils.py”, line 1159, in to<br>
dist = Distribution(self)<br>
TypeError: expected CPU (got CUDA)</p>

username: cpeters
cooked: <p>Whatever object you’re operating on there was initialised on the GPU (hence CUDA) and needs to be moved to the CPU.</p>
How to calculate the weights for the CrossEntropy loss function?

username: jS5t3r
cooked: <p>Instead of the cifar100. I am using just 4 classes (hair color) of the CelebAHQ dataset. The data is unbalanced and I need to change the loss function by adding weights. I am using an existing framework:</p>
<p>(Source: <a href="https://github.com/weiaicunzai/pytorch-cifar100/blob/2149cb57f517c6e5fa7262f958652227225d125b/train.py#L149" class="inline-onebox" rel="noopener nofollow ugc">pytorch-cifar100/train.py at 2149cb57f517c6e5fa7262f958652227225d125b · weiaicunzai/pytorch-cifar100 · GitHub</a>)</p>
<p>Replacing in <code>train.py</code> the line:</p>
<p><code>loss_function = nn.CrossEntropyLoss()</code></p>
<p>with</p>
<pre><code class="lang-auto">w = np.array([1/black/all_samples, 1/blond/all_samples, 1/brown/all_samples, 1/gray/all_samples])
cw = torch.tensor([w[0], w[1], w[2], w[3]], dtype=torch.float32).cuda() # class weights for 0, 1, 3, 4
loss_function = nn.CrossEntropyLoss(weight=cw)
</code></pre>
<p>The output is not that good… How can I improve the weights? Should I use the keywords <code>reduce</code>? (<code>        # loss_function = nn.CrossEntropyLoss(ignore_index=255, weight=cw, reduction='none'</code>)</p>

username: RaLo4
cooked: <p>I am not sure what kind of answer you are looking for. Could you specify a bit more, what you want to do.</p>
<p>To address the error message you are getting:<br>
The error tells you, that you have 100 classes (CIFAR100) but only give weights for 4 classes and that you need to give weights for all 100 classes if you want to use the <code>weight</code> parameter of CELoss.</p>

username: jS5t3r
cooked: <p>Yes, I did not adapt the number of classes.</p>

username: RaLo4
cooked: <p>I saw you completely changed your question by editing it.<br>
I almost missed that, it might have been better to post your new question below.</p>
<aside class="quote no-group" data-username="jS5t3r" data-post="1" data-topic="117224">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/js5t3r/48/40545_2.png" class="avatar"> jS5t3r:</div>
<blockquote>
<p>The output is not that good… How can I improve the weights? Should I use the keywords <code>reduce</code>?</p>
</blockquote>
</aside>
<p>As it is now you are not doing anything inherently wrong. As long as the all_samples value is correct.<br>
Take a look at <a href="https://discuss.pytorch.org/t/weight-in-cross-entropy-loss/78256/2">this</a>.</p>
<p>What <code>reduction='none'</code> changes compared to <code>reduction='mean'</code> (which is default), is that instead of getting the weighted mean of the output you will now get the output without any reduction applied, meaning you will get an output of the same size as the target size.<br>
But keep in mind that, if you want to use the <code>weight</code> parameter of CELoss and set <code>reduction</code> to <code>none</code>, you would have to manually normalize your loss. See <a href="https://discuss.pytorch.org/t/passing-the-weights-to-crossentropyloss-correctly/14731/10">here</a>.</p>
How to deal with the labels having drastically different values?

username: Capo_Mestre
cooked: <p>Hello,</p>
<p>I am dealing with a regression problem where inputs are images 98x98, and outputs are vectors of 16 elements.</p>
<p>Some examples of the outputs are:</p>
<p>[12023, 0.1, 2.0, 11982, 0.8, 1.2, 0.3, 0.9, 1.9, 1.1, 0.4, 0.5, 1.0, 0.9, 0.9, 1.7]<br>
[11975, 0.6, 2.1, 11145, 0.4, 1.1, 0.9, 0.2, 1.3, 1.6, 0.1, 0.4, 1.5, 0.4, 0.8, 1.0]<br>
etc.</p>
<p>As you can see the first and the fourth elements are several orders of magnitudes larger than the rest of the vectors.</p>
<p>The question is if this going to affect the learning process negatively and if labels need to be preprocessed somehow (e.g. normalized, or something else)?</p>

username: omarfoq
cooked: <p>Hi,</p>
<p>This will affect your training if you simply use MSE, because the model will neglect all dimensions but the first and the fourth. The easy solution is to normalize your outputs across dimension.</p>

username: Capo_Mestre
cooked: <p>Thanks! I will try that!</p>

username: Capo_Mestre
cooked: <p>Hi <a class="mention" href="/u/omarfoq">@omarfoq</a> ,<br>
What would be better:</p>
<ul>
<li>to normalize the labels only across those dimensions that have huge values, or</li>
<li>to normalize labels across all dimensions (separately), so that the labels across dimensions have mean 0 and standard deviation 1?</li>
</ul>

username: omarfoq
cooked: <p>Hello,</p>
<p>It’s better to standarize all dimensions, because:</p>
<ul>
<li>You may have some dimension with a scale that is smaller then all the rest, and then it will be neglected.</li>
<li>I believe that standardizing helps during optimization</li>
</ul>
Simple installation

username: ss_ts
cooked: <p>Hi,</p>
<p>I am installing pytorch locally on MacOs El Capitan 10.11.6 in a conda environment.  In attempts at both pip3 install torch torchvision and the<br>
conda install pytorch torchvision -c pytorch, I haven’t been able to import torch (“module not found”).  Would anybody know what I am doing wrong and what is the best way to fix it?  Thanks.</p>

username: albanD
cooked: <p>Hi,</p>
<p>You want to make sure that the “pip3” you’re using is matching the “python” that you are using. In particular, make sure that they are both for python3.<br>
To make sure you use the right pip to match the command line <code>python3</code>, you can do <code>python3 -m pip the_rest_of_your_pip_command</code>.</p>

username: ss_ts
cooked: <aside class="quote no-group" data-username="ss_ts" data-post="1" data-topic="117249">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ss_ts/48/30231_2.png" class="avatar"> ss_ts:</div>
<blockquote>
<p>pip3 install torch torchvision</p>
</blockquote>
</aside>
<p>Thank you so much.  That indeed was the issue.</p>

username: ss_ts
cooked: <p>Thank you so much.  That indeed was the issue.</p>
Is it possible for register_full_backward_hook() to be use for the gradient of an intermediate layer?

username: AlphaBetaGamma96
cooked: <p>Hi All,</p>
<p>I was wondering if it were at all possible to use <code>register_full_backward_hook</code> for the gradient of an intermediate layer? What I mean by this is, let’s say we have code which goes like…</p>
<pre><code class="lang-auto">X = torch.randn(B, N)  #get some inputs
Y = model(X) #calculate my output, Y (a R^N -&gt; R^1 function)
loss = calc_loss(target, Y) #calculate my loss
loss.backward() #calc. gradients of loss
</code></pre>
<p>Now if I were to use <code>register_full_backward_hook</code> that places a backward hook on the layers of my <code>model</code>. Let’s say I have a feed-forward model which composes purely of <code>torch.nn.Linear</code> modules. The hooks record the gradient of loss w.r.t to the output of the linear layers, so d(loss)/ds (where s is the output of a given linear layer within the network).</p>
<p>I was wondering if it’s at all possible to get <code>register_full_backward_hook</code> to record the gradient of the model output (rather than the loss) with respect to the output of the linear layers, so dY/ds ? Is this at all possible with backward hooks or is it purely restrict to the gradient from the loss?</p>
<p>Thank you!</p>
Theoretical Question: How is FLOPS counted in a simple matrix-vector multiplication?

username: zfzhang
cooked: <p>Hi All,<br>
We’ve seen many research papers comparing <strong>FLOPS</strong> of different models. I just want to understand clearly how the hand calculation is done by convention.<br>
For example, I’m multiplying a <strong>MxN</strong> matrix with a <strong>Nx1</strong> vector.<br>
Each matrix row takes dot product with the vector to produce a single scalar value in final output, which costs N multiplications and N-1 additions.<br>
Repeating for all M matrix rows, we have MxN multiplications and Mx(N-1) additions.<br>
In this example, do we count the total <strong>FLOPS</strong> as Mx(2N-1) (additions &amp; multiplications), or MxN (multiply-add operations) ?</p>
<p>Thank you very much !</p>
C++ manual element-wise slicing

username: Omroth
cooked: <p>Hey guys, just wanted to check that the following kind of assignment will retain the gradients/autograd graph properly:</p>
<pre><code>torch::Tensor t = torch::zeros({4,1}, torch::dtype(torch::kFloat32).requires_grad(false));

t.index({0,0}) = ot.index({start_index});
t.index({1,0}) = ot.index({start_index+1});
t.index({2,0}) = ot.index({start_index+2});
t.index({3,0}) = 0;
</code></pre>
<p>Thanks!</p>
Is there any way that I can "bypass" the device-side assert triggered error?

username: seankala
cooked: <p>Hi. This might sound like a bit of a strange question, but I’m wondering if there’s any way that I can simply skip over samples that cause the device-side assert triggered error.</p>
<p>There are a few samples within my dataset that cause that error, and I’m wondering if I have to find the error and debug it or reformulate the data; or if I can simply do something like a try-except-continue block within my training loop in order to bypass it.</p>
<p>I’ve tried doing precisely that and my program still crashes with the error. Is it even possible to bypass it?</p>

username: ptrblck
cooked: <p>You shouldn’t disable device assert errors, as they are pointing towards critical errors in your code.<br>
E.g. you might run into an illegal memory access (the PyTorch device assert statements should trigger the assertion before hitting the memory violation), which might corrupt your data in all imaginable ways.<br>
E.g. in case you are indeed finding a way to disable all asserts, your code might use an illegal indexing value and the (now faulty) indexing kernel might write (any) value to this invalid memory location.<br>
If your data (e.g. a forward activation) would be stored there, it might be overwritten, which might even result in random NaN values during your training.<br>
In short: don’t do it and fix your code <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>
Per-sample weighting leads to no training progress (even when all weights are one)

username: Tobue
cooked: <p>I am working on a model which classifies images between two classes, 0 and 1, whereas there are example of varying “strength” of 1. Some of them are so weak that it would be almost okay for my model to classify them as 0.</p>
<p>Now I have tried the following:<br>
I am using a CrossEntropyLoss with <code>reduction = 'none'</code>.<br>
I split up my classes into 5 classes, whereas 1 to 4 are all mapped to class “1” with a weight corresponding to their evidence strength.<br>
Then I am weighting the examples by multiplying with a weight-vector of length BatchSize.</p>
<p>I implement this as follows:</p>
<blockquote>
<p>for inputs,classes in dataloaders[phase]:</p>
<blockquote>
<p>inputs = inputs.to(device).float()<br>
prefactor_dict = {0:1, 1:1,  2:0.1,  3:0.05, 4:0.01}<br>
prefactors = np.array([prefactor_dict[cls] for cls in classes.numpy()])<br>
prefactors = torch.from_numpy(prefactors)<br>
prefactors = prefactors.to(device)<br>
labels = torch.from_numpy(np.array([0 if cls == 0 else 1 for cls in classes]))<br>
labels = labels.to(device)<br>
optimizer.zero_grad()<br>
with torch.set_grad_enabled(phase == ‘train’):</p>
<blockquote>
<p>outputs = model(inputs[:,None,…])<br>
_,preds = torch.max(outputs,1)<br>
loss = criterion(outputs, labels)*prefactors<br>
loss = loss.mean()<br>
if phase == ‘train’:</p>
<blockquote>
<p>loss.backward()<br>
optimizer.step()</p>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<p>If I don’t use the prefactors and don’t set reduction to ‘none’, my model manages to overfit the data in a few epochs. However even if I set the prefactors all to 1, my model doesn’t learn anything. It simply gets stuck at exactly 50% accuracy.</p>
Synchronize() is called, but I have never called that function

username: 111480
cooked: <p>When I created a .densenet201pt from Torchvison using Torchscript and ran it on c++, I profiled it with nvprof and found that cudaStreamSyncronize() and cudaDeviceSyncronize() were called.<br>
But I didn’t explicitly call those functions.<br>
Where are these functions called?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/f/6f30cbe7ddceb2537006b4fecb93a20262bcbeab.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/6f30cbe7ddceb2537006b4fecb93a20262bcbeab" title="nvprof"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/f/6f30cbe7ddceb2537006b4fecb93a20262bcbeab_2_690x381.jpeg" alt="nvprof" data-base62-sha1="fRDxNX9puoHPaCZqN7LvZBsczvt" width="690" height="381" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/f/6f30cbe7ddceb2537006b4fecb93a20262bcbeab_2_690x381.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/6/f/6f30cbe7ddceb2537006b4fecb93a20262bcbeab.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/f/6f30cbe7ddceb2537006b4fecb93a20262bcbeab.jpeg 2x" data-dominant-color="44434D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">nvprof</span><span class="informations">828×458 114 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>These methods might be called internally, e.g. if you transfer device tensors to the CPU, as the GPU would need to be synchronized before the values can be read. <a href="https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html#avoid-unnecessary-cpu-gpu-synchronization">This section</a> of the performance guide explains it in a bit more detail.</p>
GRU cannot update hidden_state in-place

username: iMusic
cooked: <p>I got a problem when using pytorch to implement a GRU network:</p>
<p>My codes are as below:</p>
<pre><code class="lang-auto">import torch

class GRU_model(torch.nn.Module):
    def __init__(self, device):
        super(GRU_model, self).__init__()
        self.h = torch.randn((1,1,5), device=device, dtype=torch.float)

        self.GRU_1 = torch.nn.GRU(input_size=5, hidden_size=5)

    def forward(self, a):
        output, self.h = self.GRU_1(a, self.h)
        return output

if __name__ == '__main__':
    learn_rate=1e-4
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    model = GRU_model(device).to(device=device)
    optimizer = torch.optim.Adam(model.parameters(), lr=learn_rate)

    for i in range(10):
        a = torch.randn((1, 1, 5), device=device, dtype=torch.float)
        output = model(a)
        loss = (a - output).mean()

        optimizer.zero_grad()
        loss.backward(retain_graph=True)
        optimizer.step()
</code></pre>
<p>and I got an error like this:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "C:/Users/Administrator_/Desktop/Graduation_Project/MIDI_Music_style_transfer/GRU_toy_in-place_hidden_states_change/main.py", line 40, in &lt;module&gt;
    loss.backward(retain_graph=True)
  File "C:\Users\Administrator_\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\tensor.py", line 245, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "C:\Users\Administrator_\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\autograd\__init__.py", line 147, in backward
    allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [15, 5]] is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
</code></pre>
<p>I just want to update the hidden_state in the GRU after an epoch but it just doesn’t work!</p>
<p>I’d be really appreciated if you could me help out!</p>
Unable to download libtorch(cuda 10.2, 1.8.1)

username: DongkyuLee
cooked: <p>Hi guys, I got some problem.</p>
<p>I tried to download libtorch(c++, with cuda10.2, 1.8.1 stable version), but the browser shows access denied, do you guys have 1.8.1 libtorch(with cuda10.2)?<br>
(the other versions can be installed, but this version couldn’t be)</p>
<p>And If I wanna download previous c++ torch(a.k.a. libtorch), how can I install it? I entered the previous version page, but I couldn’t find previous c++ version.</p>
<p>Thanks for reading</p>
How should I detach a channels_last format tensor?

username: seungjun
cooked: <p>Hi,</p>
<p>I am using channels_last format when using amp for training.<br>
From pytorch 1.8, when I detach a tensor that is in channels_last format and feed it to another model, pytorch breaks with error.</p>
<p>Here’s an example snippet.</p>
<pre><code class="lang-python">import torch
import torch.nn as nn
import torch.cuda.amp as amp

device = torch.device('cuda')
dtype = torch.float32
memory_format = torch.channels_last
# memory_format = torch.contiguous_format

model1 = nn.Conv2d(3,3,1,1).to(device=device, dtype=dtype, non_blocking=True, memory_format=memory_format)
model2 = nn.Conv2d(3,3,1,1).to(device=device, dtype=dtype, non_blocking=True, memory_format=memory_format)

input = torch.randn(1,3,4,4).to(device, dtype=dtype, memory_format=memory_format)

with amp.autocast():
    out1 = model1(input)
    out2 = model2(out1.detach())
</code></pre>
<p>Here’s the error message:</p>
<pre><code class="lang-bash">RuntimeError: set_sizes_and_strides is not allowed on a Tensor created from .data or .detach().
If your intent is to change the metadata of a Tensor (such as sizes / strides / storage / storage_offset)
without autograd tracking the change, remove the .data / .detach() call and wrap the change in a `with torch.no_grad():` block.
For example, change:
    x.data.set_(y)
to:
    with torch.no_grad():
        x.set_(y)
</code></pre>
<p>If I change the memory format to <code>torch.contiguous_format</code>, it works fine.<br>
Is detaching a channels_last tensor forbidden?<br>
As there is no resizing/reshaping operations in the second model, the error message seems to be irrelevant.</p>
<p>A related issue is here: <a href="https://github.com/pytorch/pytorch/issues/55301" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/issues/55301</a></p>
<p>PyTorch version: 1.8.1</p>
Errors when building static linked LibTorch

username: jingzhao.ou
cooked: <p>I tried to compile a static linked LibTorch on Ubuntu 18.04.</p>
<pre><code class="lang-auto">git clone -b v1.8.1 --recurse-submodule https://github.com/pytorch/pytorch.git &amp;&amp; \
mkdir build_libtorch_static &amp;&amp; cd build_libtorch_static &amp;&amp; \
cmake -DBUILD_SHARED_LIBS:BOOL=OFF -DCMAKE_BUILD_TYPE:STRING=Release -DUSE_CUDA:BOOL=OFF -DCMAKE_INSTALL_PREFIX:PATH=~/local/libtorch ../pytorch -DCMAKE_CXX_FLAGS:STRING=-fPIC &amp;&amp; \
cmake --build . --target install --parallel
</code></pre>
<p>I ran into the following errors:</p>
<pre><code class="lang-auto">[100%] Linking CXX shared module python/caffe2_pybind11_state.cpython-39-x86_64-linux-gnu.so
[100%] Built target torch_shm_manager
[100%] Built target caffe2_pybind11_state
[100%] Linking CXX shared library ../../lib/libtorch_python.so
/usr/bin/ld: ../../lib/libtensorpipe_uv.a(process.c.o): relocation R_X86_64_PC32 against symbol `environ@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
caffe2/torch/CMakeFiles/torch_python.dir/build.make:1721: recipe for target 'lib/libtorch_python.so' failed
</code></pre>
<p>I tried PyTorch 1.8.1 and 1.8.0. No luck with both versions. The compilation works just fine if I set <code>-DBUILD_SHARED_LIBS:BOOL=ON</code>.</p>
<p>Any help would be highly appreciated.</p>
<p>Best regards,<br>
Jingzhao</p>

username: jingzhao.ou
cooked: <p>I figured out the solution myself. I set <code>-DBUILD_PYTHON:BOOL=OFF</code> and that allows me to build the static libtorch library successfully. This is a bit tricky IMO.</p>
Loading all MRI slices of a patient to train on model

username: rtan
cooked: <p>Hello!</p>
<p>I am quite new to using PyTorch, and I’m currently researching ways to create a custom dataset for my issue. I currently have around 287 patients, and I was able to preprocess the DICOM files to create images and mask files for all of them.</p>
<p>The problem I’m currently facing is as follows: I would like to train my UNET model on all the slices for each patient that’s in the train group. I currently a root directory called <strong>data</strong>, which consists of two folders, called <strong>images</strong> and <strong>masks</strong> respectively. Both folders contain 287 patient folders and within each patient folder, includes the same <em>n</em> amount of MRI slices. The task I’m currently trying to work on is semantic segmentation using UNET, and from what I can see - <code>ImageFolder</code> only works for image classification.</p>
<p>What would be the best way to create a custom dataset for this, and how would I load the data to my UNET model? Please let me know! Thank you in advance!</p>

username: ptrblck
cooked: <p>Your approach of using a custom <code>Dataset</code> sounds correct.<br>
To implement it, you could have a look at <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html">this tutorial</a> first and adapt it so that it fits your use case.<br>
In particular, you would store the image/patient input images and the corresponding mask paths in the <code>__init__</code> method and would then lazily load each pair in the <code>__getitem__</code>.<br>
The critical step would be to make sure the paths for the DICOM images and masks are sorted correctly, so that indexing these <code>lists</code> of paths yields the desired corresponding pairs and doesn’t mix patients etc.<br>
Also, you could take a look at e.g. <a href="https://monai.io/">MONAI</a>, which might already provide some <code>Dataset</code> implementations for similar use cases.</p>
How to import caffe2 weights in pytorch?

username: tejal567
cooked: <p>I have init_net.pb and predict_net.pb files of a caffe2 model. I have created the same network in pytorch. Now, I want to copy all the weights from caffe2 to pytorch model. Can someone please guide me how to do that?<br>
These are the layers with trainable weights in caffe2:<br>
conv<br>
spatial_bn<br>
conv_transpose</p>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a> please help</p>
Efficient way to turn a linear layer vector into an arrangement of a symmetric matrix?

username: ADONAI_TZEVAOT
cooked: <p>I have a vector A = [ a1,a2,a3,a4,…,an, b1,b2,…,bn-1,c1,c2,…,cn-2,…,p1]. I want to create a symmetric matrix from this such as a Matrix M such that:</p>
<pre><code class="lang-auto">M = 
[a1  b1  c1  d1  ...................  p1
 b1  a2  b2  c2  ...................  o2
 c1  b2  a3  b3  ...................  n3
 .......................................
 ....................................... 
p1  ................................. an]
</code></pre>
Torch.nansum yields NaN gradients unexpectedly

username: seongmin
cooked: <p>Hi,</p>
<p>Let L be the result of a loss function before reduction, possibly with NaN values.<br>
I expected <code>torch.nansum(L)</code> to accumulate non-NaN values from L so that <code>.backward()</code> would result in valid gradient.</p>
<p>However, my expectation does not match the result.</p>
<p>Here is a snippet to reproduce the problem.</p>
<pre><code class="lang-python">def test():
    import torch

    torch.manual_seed(3)

    # net, opt
    net = torch.nn.Linear(3,2)
    opt = torch.optim.SGD(net.parameters(), lr=0.1)

    # data (x), target (y) with NaN
    x = torch.randn(2,3)
    y = torch.randn(2,2)
    y = torch.where(y &lt; 0, torch.tensor(float('nan')), y) # NaN at (1,0)

    # calculate loss
    out = net(x)
    loss = torch.nn.MSELoss(reduction='none')(out, y)
    
    # accumulate losses except NaN
    loss = loss.nansum()

    # backward, train
    opt.zero_grad()
    loss.backward()
    opt.step()

    # see if any parameter became NaN
    assert not torch.any(net.weight.isnan())

    print('Test passed')
</code></pre>
<p>As <code>loss</code> contains three valid numbers and one NaN,<br>
I expected <code>loss.nansum()</code> to replace single NaN with 0 so that the following <code>loss.backward()</code> would calculates gradients without NaN.</p>
<p>However, all the values in <code>net.weight.grad</code> becomes NaN.</p>
<p>I wonder:<br>
(1) if this is a bug, and<br>
(2) what would be a correct / better way to calculate the loss in my problem.</p>
<p>Edit: added import statement to be stand-alone example.<br>
I tested under Python 3.7.10, PyTorch 1.8.0.</p>

username: albanD
cooked: <p>Hi,</p>
<p>You can use <code>tensor.register_hook(your_fn)</code> to be able to print the gradient for each given Tensor.<br>
In particular, here, I think that the loss before the nansum won’t have any nan as you expect.<br>
The problem is that the backward of the mse will create nan values because some of the y has nan. You will need to filter these out before computing the MSE to avoid nan in the gradients flowing back.</p>

username: seongmin
cooked: <p>Hi Alban,</p>
<p>Thank you so much for your help.<br>
My understanding now is that because MSELoss will use its activation for back propagation, once NaN appeared then it will be accounted for regardless of whatever operation I use afterwards.</p>
<p>I used the following code instead to resolve the issue.</p>
<pre><code class="lang-python">    # loss = torch.nn.MSELoss(reduction='none')(out, y)
    nan = torch.isnan(y)
    y = torch.where(nan, torch.tensor(0.0), y)
    out = torch.where(nan, torch.tensor(0.0), out)
    loss = (out - y) ** 2
</code></pre>
<p>Thanks,<br>
Seongmin</p>
What is the best data storage option for PyTorch?

username: ccook
cooked: <p>I’ve been using PyTorch to estimate discrete choice models with large choice sets. The data are “sessions” where each session consists of ~10k rows corresponding to the choice set.</p>
<p>Currently, I’m storing the data in an h5 indexed by a unique session key, then reading in sessions in batches with a DataLoader. This works, but is very slow (the IO time is ~10x the computation time). On disk, the h5 file is about 500gb.</p>
<p>Is there a better option? I’ve seen WebDataset, which I’m planning to try next barring any better suggestions!</p>

username: ccook
cooked: <p>Some quick benchmarks on a large-but-not-all sample (~83 million rows), reading in subsets of data using <code>query</code> for H5 and <code>filters</code> kwarg for parquet.</p>
<p>H5 with session id as index<br>
– On disk: 5.05gb<br>
– Read in time, 1 session: 0.018 seconds<br>
– Read in time, 5000 sessions: 26.9 seconds</p>
<p>Parquet (pyarrow engine) with session id partitions:<br>
– On disk: 490mb<br>
– Read in time, pandas, 1 session: 2.6 seconds<br>
– Read in time, pandas, 5000 sessions: 7.6 seconds<br>
– Read in time, dask, 1 session: 2.2 seconds<br>
– Read in time, dask, 5000 sessions: 47.1 seconds</p>
<p>Somewhat surprised that dask was far slower than Pandas for reading in larger numbers of sessions. Also surprised at the disk space differences. For now, I think parquet + pandas will be sufficient for me – but would love to hear if anyone has other options that are worth a shot!</p>
<p>Edit:<br>
Ok just for fun I also tried loading it to BigQuery with session id partitions (for other reasons, having it in BQ would be useful so was curious how much of a hit it’d be to compute time)<br>
– On disk: 5.5gb<br>
– Read in time, 1 session: 2.5 seconds<br>
– Read in time, 5000 sessions: 26.4 seconds</p>
torch.nn.Module.children for duplicate children

username: pierresegonne
cooked: <p>Hello there,</p>
<p>I have a quick question regarding the use of the methods <code>children</code>, <code>named_children</code>, <code>modules</code> or <code>named_modules</code> from <code>torch.nn.Module</code>.</p>
<p>In the doc, a note says that “Duplicate modules are returned only once”. Is there a way to obtain the children <strong>with</strong> duplicates?</p>
<p>For example:</p>
<p>Let’s imagine that I have the following module:</p>
<pre><code class="lang-auto">Sequential(
  (0): Sequential(
    (0): Linear(in_features=2, out_features=256, bias=True)
    (1): BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): LeakyReLU(negative_slope=0.01)
    (3): Linear(in_features=256, out_features=512, bias=True)
    (4): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (5): LeakyReLU(negative_slope=0.01)
    (6): Linear(in_features=512, out_features=784, bias=True)
  )
  (1): Sigmoid()
)
</code></pre>
<p>I’d like to iterate over (the difference being <code>5 LeakyReLU(negative_slope=0.01)</code> that does not appear)</p>
<pre><code class="lang-auto">0 Sequential(
  (0): Linear(in_features=2, out_features=256, bias=True)
  (1): BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): LeakyReLU(negative_slope=0.01)
  (3): Linear(in_features=256, out_features=512, bias=True)
  (4): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): LeakyReLU(negative_slope=0.01)
  (6): Linear(in_features=512, out_features=784, bias=True)
)
0 Linear(in_features=2, out_features=256, bias=True)
1 BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
2 LeakyReLU(negative_slope=0.01)
3 Linear(in_features=256, out_features=512, bias=True)
4 BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
5 LeakyReLU(negative_slope=0.01)
6 Linear(in_features=512, out_features=784, bias=True)
1 Sigmoid()
</code></pre>
<p>Instead of the no-duplicate version</p>
<pre><code class="lang-auto">0 Sequential(
  (0): Linear(in_features=2, out_features=256, bias=True)
  (1): BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): LeakyReLU(negative_slope=0.01)
  (3): Linear(in_features=256, out_features=512, bias=True)
  (4): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): LeakyReLU(negative_slope=0.01)
  (6): Linear(in_features=512, out_features=784, bias=True)
)
0 Linear(in_features=2, out_features=256, bias=True)
1 BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
2 LeakyReLU(negative_slope=0.01)
3 Linear(in_features=256, out_features=512, bias=True)
4 BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
6 Linear(in_features=512, out_features=784, bias=True)
1 Sigmoid()
</code></pre>
<p>Thanks!</p>

username: pierresegonne
cooked: <p>Answer:<br>
It’s because I was instantiating the <code>LeakyReLU</code> only once!</p>
How to convert the X-axis of the graphs displayed on tensorboard to float

username: pavs
cooked: <p>Hi,<br>
I have created simple training accuracy/loss graphs and validation accuracy/loss graphs.<br>
The x-axis is displayed in exponential format.<br>
I would like to know how to change it to float format<br>
Image of graph<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/4/b/4bdf2ce6efcf8cc3ad8d575084f848bdacfc3df1.png" alt="image" data-base62-sha1="aPbXBvuw8wMLEHj8RxQLUgmf0mR" width="602" height="184"></p>
<p>Code to write the graphs on tensorboard:</p>
<pre><code class="lang-auto">board_writer.add_scalar(f'Validation/Loss/Average', val_loss_in_this_epoch/no_of_batches_in_this_epoch, epoch)
board_writer.add_scalar(f'Validation/Accuracy/Average', val_correct_in_this_epoch/(no_of_batches_in_this_epoch*batch_size), epoch)
board_writer.add_scalar(f'Validation/TimeTakenInMinutes', (time()-start_time)/60, epoch)
board_writer.flush()
</code></pre>
Jacobian-matrix product or matrix-Jacobian product

username: pomonam
cooked: <p>Is there any method to support Jacobian-matrix products? For example, let’s say I have a scalar loss and parameters of dimension p. My “v” vector in JVP is a matrix of dimension “batch_size x p”. I want to compute the JVP for each batch so that the output is “batch_size x 1”. I was wondering If there any ways to do this without using a loop.</p>

username: soulitzer
cooked: <p>Not yet, but its in the works. Check out the issue tracking forward-mode AD here <a href="https://github.com/pytorch/pytorch/issues/10223" class="inline-onebox" rel="noopener nofollow ugc">[feature request] Forward-mode automatic differentiation · Issue #10223 · pytorch/pytorch · GitHub</a>.</p>
Custom layer with Numpy computation

username: otterspace
cooked: <p>Hi PyTorch community,</p>
<p>I am designing a network with a custom layer, in which I am using a third-party framework that requires numpy arrays. Since my layer was not behaving as expected, I tried to find the problem by drastically simplifying it.</p>
<p>At this point I am simply echoing the input to my layer back into the network, all parameters’ <code>requires_grad</code> are set to <code>False</code>, all custom gradients return <code>None</code>. So I would assume that my custom layer should have zero impact on the training, comparing to performance of the network without this layer at all. All I do is to convert a tensor to numpy and back and yet there is a palpable negative impact on training convergence.</p>
<p>A toy model of what I am attempting looks something like this, following the <a href="https://pytorch.org/tutorials/advanced/numpy_extensions_tutorial.html" rel="noopener nofollow ugc">official tutorial</a></p>
<pre><code class="lang-auto">def forward(self, x):
        x = torch.tanh(self.fc_1(x))
        ...

        # Disassemble the batch into data points
        x_out = []
        for _, x_in in enumerate(x):
            # --- This should be the output from the custom layer ---
            x_numpy = x_in.detach().numpy()
            # -------------------------------------------------------
            x_out.append(x_in.new(x_numpy).reshape(1, -1))
        # Assemble them back into the batch
        x = torch.cat(x_out, dim=0)
        
        ...
        return torch.tanh(self.fc_n(x))
</code></pre>
<p>Trying the same weird construction without convertion to numpy has zero influence on training. Is there a way to convert <code>numpy.array</code> to <code>torch.tensor</code> in the middle of you network without training penalty?</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="otterspace" data-post="1" data-topic="116972">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/o/b77776/48.png" class="avatar"> otterspace:</div>
<blockquote>
<p>At this point I am simply echoing the input to my layer back into the network, all parameters’ <code>requires_grad</code> are set to <code>False</code>, all custom gradients return <code>None</code>.</p>
</blockquote>
</aside>
<p>If you are returning <code>None</code> gradients, you would detach the computation graph at this point and all preceding layers wouldn’t get any gradients, which could explain the issues you are seeing.<br>
In case you want to write a “no-op” numpy layer, you could pass on the gradients to the preceding layer.</p>
Resizing the ground truth of 3D CT scan

username: moreshud
cooked: <p>How can I resize a 3D image tensor of size <code>143 x 512 x 512</code> to <code>143 x 256 x 256</code>?</p>

username: Sanjan_Das
cooked: <p>Most image transformations can be done using PyTorch transforms. For your particular question, you can can use <code>torchvision.transforms.Resize</code>.</p>
<pre><code class="lang-auto">import torch
from torchvision import transforms

resize = transforms.Resize((256, 256)) # the output shape you want

# an example 3D tensor
t = torch.rand(143, 512, 512)

t_resized = resize(t) # you should get its shape as (143, 256, 256), preserving the channel dimension
</code></pre>

username: moreshud
cooked: <p>That’s exactly what I did.</p>
<pre><code class="lang-auto">gt_transform = transforms.Compose([
            transforms.ToPILImage(),
            transforms.Resize((256, 256), interpolation=0),
            transforms.ToTensor()
            ])

label = torch.from_numpy(label.npy) # [143, 512, 512]
label_transform = gt_transform(label)
</code></pre>
<p>But I got the error</p>
<pre><code class="lang-auto">ValueError                                Traceback (most recent call last)
&lt;ipython-input-31-da837537eac8&gt; in &lt;module&gt;()
----&gt; 1 label = gt_transform(label)

2 frames
/usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py in to_pil_image(pic, mode)
    229         # check number of channels
    230         if pic.shape[-3] &gt; 4:
--&gt; 231             raise ValueError('pic should not have &gt; 4 channels. Got {} channels.'.format(pic.shape[-3]))
    232 
    233     elif isinstance(pic, np.ndarray):

ValueError: pic should not have &gt; 4 channels. Got 143 channels.
</code></pre>

username: Sanjan_Das
cooked: <p>You’re getting the error because you’re using the transform <code>ToPILImage</code> on a tensor that just cannot form a PIL Image. PIL images have a maximum channel dimension of 4 that they can work with, whereas you have 143 channels.</p>
<p>You can see how your error arises if you check out the source code for <code>ToPILImage</code> here:<br>
<a href="https://pytorch.org/vision/stable/_modules/torchvision/transforms/functional.html#to_pil_image" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/vision/stable/_modules/torchvision/transforms/functional.html#to_pil_image</a></p>

username: moreshud
cooked: <p>Thanks, I will check as suggested.</p>

username: Kapil_Rana
cooked: <p>My question is why you are using ToPILImage transformation?<br>
can you try this:</p>
<p>gt_transform = transforms.Compose([<br>
transforms.Resize((256, 256), interpolation=0),<br>
transforms.ToTensor()<br>
])</p>
<p>label = torch.from_numpy(label.npy) # [143, 512, 512]<br>
label_transform = gt_transform(label)</p>

username: moreshud
cooked: <p>Got this error</p>
<pre><code class="lang-auto">TypeError: pic should be PIL Image or ndarray. Got &lt;class 'torch.Tensor'&gt;
</code></pre>

username: moreshud
cooked: <p>I remove the <code>ToTensor()</code> and it works. Thanks to all</p>
Create Kotlin library for both Android App and Desktop App

username: manatttta
cooked: <p>TLDR: I guess my question can be summarised to: Is there a way to create a pure Kotlin library with Torch as a dependency, that can be imported both from an Android App and a Desktop Kotlin/Java App?</p>
<p>I’ve created a Kotlin library for segmentation (let’s call it <code>MySeg</code>). It should use PyTorch as a dependency.<br>
This <code>MySeg</code>should run on both an Android device, on on a pure Kotlin/Java Desktop application for test purposes.<br>
I’ve added the PyTorch dependency but the compiler can find references to PyTorch.</p>
<p>This is what my <code>build.gradle</code> dependencies look like:</p>
<pre><code>dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
    implementation 'org.pytorch:pytorch_android:1.4.0'
    implementation 'org.pytorch:pytorch_android_torchvision:1.4.0'
    implementation project(path: ':opencv', configuration: 'default')
}
</code></pre>
<p>Is there any way I can create this <code>MySeg</code> library, with Torch as a dependency, so that <code>MySeg</code> can be imported both by an Android App and a pure Kotlin Desktop app?</p>
SentencePiece Model: Passing Flag into Function

username: J_Johnson
cooked: <p>The code is:</p>
<pre><code class="lang-auto">generate_sp_model('example.csv', vocab_size=20000, model_prefix='example', model_type='char')
</code></pre>
<p>But I get the following because my lines are 7000 characters long:</p>
<pre><code class="lang-auto">trainer_interface.cc(287) LOG(WARNING) Found too long line (7034 &gt; 4192).
trainer_interface.cc(289) LOG(WARNING) Too long lines are skipped in the training.
trainer_interface.cc(290) LOG(WARNING) The maximum length can be changed with --max_sentence_length=&lt;size&gt; flag.
trainer_interface.cc(315) LOG(INFO) Loaded all 0 sentences
trainer_interface.cc(321) LOG(INFO) Skipped 12048 too long sentences.
</code></pre>
<p>I see that there may be some way to run this from command prompt(see the third WARNING above). But is there a way to pass the <code>max_sentence_length</code> flag into the above code to make this work as a python function?</p>
<p>Thanks!</p>
Concatenate the input vector of the NN a constant vector

username: Javier_Martin
cooked: <p>Hi.</p>
<p>I am trying to invert a physical model in which constant data is used as a vector. I would like to input both the model output and this constant vector data to the NN.<br>
How can I do this?</p>
<p>Thanks and regards …</p>

username: Dwight_Foster
cooked: <p>Have you tried just using torch.cat like this:</p>
<pre><code class="lang-auto">input = torch.cat([out, input])
</code></pre>

username: Javier_Martin
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/9/490aa9d3c27abed1ca7fbb4dfc94075503599f7f.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/490aa9d3c27abed1ca7fbb4dfc94075503599f7f" title="Picture1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/9/490aa9d3c27abed1ca7fbb4dfc94075503599f7f_2_690x166.jpeg" alt="Picture1" data-base62-sha1="aq9HuJOVaUoaevqC5NXTjaIRM6P" width="690" height="166" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/9/490aa9d3c27abed1ca7fbb4dfc94075503599f7f_2_690x166.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/4/9/490aa9d3c27abed1ca7fbb4dfc94075503599f7f_2_1035x249.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/9/490aa9d3c27abed1ca7fbb4dfc94075503599f7f_2_1380x332.jpeg 2x" data-dominant-color="DFEAF3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Picture1</span><span class="informations">1719×414 29.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I don’t have much experience in pytorch or neural networks, but from what I understand, if I concatenate the constant vector, I have to do it to each of the inputs of the training set, so I multiply by 4 the memory, and since I am using a very large set of training, I think that is not an option.<br>
I have drawn a diagram of the inputs and outputs of the model, I use it to generate the training set, as you can see the model uses only the three vectors of size 80.<br>
I wonder if there is any way to introduce this information in the NN to invert the model and without quadrupling the information of the training set.</p>
Log_prob calculation process

username: Yan_Wen
cooked: <p>Make the output of a MLP network as a logits, and then calculate Categorical, then calculate the log_prob, what is the process of calculation and what values are involved in the calculation of the log_prob function?</p>

username: H-Huang
cooked: <p>This question is not related to the distributed package so please change the category.</p>
<p>But I think this is the logic for log_prob that you are looking for: <a href="https://github.com/pytorch/pytorch/blob/master/torch/distributions/categorical.py#L115-L121" class="inline-onebox" rel="noopener nofollow ugc">pytorch/categorical.py at master · pytorch/pytorch · GitHub</a></p>
Need help in fine-tuning EditSQL - an NLP2SQL architecture

username: plustimon
cooked: <p>Hello,<br>
I am trying to apply transfer learning with the help of this <a href="https://github.com/ryanzhumich/editsql" rel="noopener nofollow ugc">GitHub Repo</a>. I am new to PyTorch. Any help with how can we approach implementing transfer learning with the same will be appreciated.</p>
<p>Thank You <img src="https://discuss.pytorch.org/images/emoji/apple/grinning.png?v=9" title=":grinning:" class="emoji" alt=":grinning:"></p>