akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How to implement imitation learning with Pytorch?

xsquared: <p>Hello,</p>
<p>Are there any books or tutorials that show how to implement imitation learning networks in Pytorch?</p>
<p>I have googled this question and searched on Amazon for imitation learning books but I cannot find anything.</p>
<p>I am looking for code so I can learn how to implement imitation algorithms myself.</p>
<p>Any help will be appreciate it. Thanks!</p>

Dwight_Foster: <p>Are you looking for something like <a href="https://medium.com/@kengz/fast-implementation-of-self-imitation-learning-ffcd0b2b6c6b" rel="noopener nofollow ugc">this</a>?</p>

xsquared: <p>yeah, something like that but I am looking for an imitation learning implementation. that is self imitation learning implementation.</p>

yipliu: <p>Hi, do you find some useful tutorials?</p>
Is there a standard way of accessing the loss from an optimizer class?

p-enel: <p>I am implementing improved Rprop with weight backtracking and I would like to follow the standard way of PyTorch to do so. I see that a closure function can be used inside the <code>step</code> function of an <code>optimizer</code> class to get access to the loss. However, I have no need to calculate the gradients again, I just want to have access to the loss value. Is there anyway to do that other than passing the loss as an argument in the <code>step</code> function?</p>

zetyquickly: <p>Hey, <a class="mention" href="/u/p-enel">@p-enel</a></p>
<p>As long I can see from the source code <code>torch.optim.Optimizer</code> has nothing to do with loss it’s only updates the <code>params</code> based on gradients. So, when the user calls <code>loss.backward()</code> in a</p>
<pre><code class="lang-auto">for input, target in dataset:
    optimizer.zero_grad()
    output = model(input)
    loss = loss_fn(output, target)
    loss.backward()
    optimizer.step()
</code></pre>
<p>parameters that are tracked by optimizer are already have everything for a <code>step()</code>.</p>
<p>Regarding the closure. <a href="https://pytorch.org/docs/stable/_modules/torch/optim/lbfgs.html#LBFGS" rel="noopener nofollow ugc">LBFGS</a> calculates loss multiple times in its work but the <a href="https://pytorch.org/docs/stable/_modules/torch/optim/sgd.html#SGD" rel="noopener nofollow ugc">SGD</a> doesn’t eve expect the loss to come.</p>
<p>In short I’d say if you need to work with an exact loss values you should pass it as a closure like it’s done in LBFGS, and please refer to its source code</p>
Libtorch Polymorphism

jackshi: <p>Libtorch uses CRTP to make cloneable modules. For example, <code>torch::nn::Softmax</code> is a <code>ModuleHolder&lt;SoftmaxImpl&gt;</code> and <code>SoftmaxImpl</code> is a <code>torch::nn::Cloneable&lt;SoftmaxImpl&gt;</code>.</p>
<p>Say I want a <code>std::vector&lt;Module&gt;</code> or a function that returns a activation function that’s also a <code>Module</code>. I have to cast various modules such as <code>torch::nn::Softmax</code> to <code>torch::nn:Module</code>. This means I loose <code>Cloneable</code> which I’m not sure is a problem. If I return a reference by simply dereferencing <code>ModuleHolder</code>, since it’s a shared pointer, I’m not sure if the underlying module is guaranteed to survive.</p>
<p>Any advice on how to get around <code>Cloneable</code> and <code>ModuleHolder</code> to work with generic <code>torch::nn::Module</code> (so I can have a function that returns it or a <code>std::vector</code> of it) is much appreciated!</p>

jackshi: <p>Solved: use AnyModule.</p>
RNN many-to-many classification with cross entropy loss

Johannes_L: <p>Hello everyone,<br>
I have a short question regarding RNN and CrossEntropyLoss:<br>
I want to classify every time step of a sequence. For this I want to use a many-to-many classification with RNN. So I forward my data (batch x seq_len x classes) through my RNN and take every output. My target is already in the form of (batch x seq_len) with the class index as entry.<br>
Now I use the CrossEntropyLoss to train my net and that’s the point I’m not sure about my solution.<br>
The CrossEntropyLoss wants the output as (NxC) and the target as (N). So what I do is reshaping the output to (batch<em>seq_len x C) and the target to (batch</em>seq_len).<br>
My minimal working example looks like this:</p>
<pre><code class="lang-auto">import torch

# 3 batches, sequence length 10, 5 features to classify into 3 classes
data = torch.rand((3, 10, 5))  # (batch x seq_len x features)
target = torch.randint(0, 3, (3, 10))  # (batch x seq_len), class index for each timestep

model = torch.nn.RNN(input_size=5, hidden_size=3, batch_first=True)
output, _ = model.forward(data)  # (batch x seq_len x number_of_classes)

# reshape output and target for cross entropy loss
output = output.reshape(output.size(0)*output.size(1), -1)  # (batch * seq_len x classes)
target = target.reshape(-1)  # (batch * seq_len), class index

criterion = torch.nn.CrossEntropyLoss()
loss = criterion(output, target)
</code></pre>
<p>Is this correct? Is the reshape() doing what I want to do?</p>
<p>Thanks for your help!</p>

ptrblck: <aside class="quote no-group" data-username="Johannes_L" data-post="1" data-topic="106197">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/johannes_l/48/30296_2.png" class="avatar"> Johannes_L:</div>
<blockquote>
<p>The CrossEntropyLoss wants the output as (NxC) and the target as (N).</p>
</blockquote>
</aside>
<p>This would be the expected shapes for the “standard” multi-class classification use case, but as described in <a href="https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html?highlight=crossentropyloss#torch.nn.CrossEntropyLoss">the docs</a> the expected tensor can have additional dimensions (<code>d1, d2, ... , dk</code>).<br>
For your use case you could thus use the model output as <code>[batch_size, nb_classes, seq_len]</code> and the target as <code>[batch_size, seq_len]</code>.</p>

Chin_GM: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>,</p>
<p>Good day.<br>
May I know is it possible for sequence length for both data and target are different?<br>
For example, seq for data is 60 and seq for target is 5.</p>
<p>Thanks</p>

ptrblck: <p>It could be possible to use them, but it depends on the actual use case.<br>
I.e. in case the model outputs a tensor with a different sequence length than the target tensor, how would the loss be calculated? If your criterion handles inputs of different lengths, then it would work.</p>
Where to look in codebase for methods that change values of tensor?

nickleus27: <p>Hi everyone this is my first post.  I would like to contribute to an issue on github’s pytorch codebase.  I am having a hard time finding where in the codebase functions are that change values inside of a tensor.  For example if you have:</p>
<blockquote>
<blockquote>
<blockquote>
<p>x = torch.tensor([[1,2],[3,4]])<br>
and then you<br>
x[0,1] = 1<br>
What is happening under the hood or what functions are being called to allow index [0,1] = 1<br>
I want to do some error checking in this scenario to throw an error if trying to write to a read only memory if tensor was created from a numpy array with mmap =‘r’ .  I have an idea of how i want to error check, but I am having a hard time finding where to add my code.</p>
</blockquote>
</blockquote>
</blockquote>
<p>I hope posting this in the c++ category was okay, because I am not sure where the correct category would be.<br>
Cheers</p>

nickleus27: <p>Ok i found x[0,1] =1 in the codebase.<br>
The function being called is THPVariable_setitem in torch/csrc/autograd/python_variable_indexing.cpp<br>
Posting this hear in case this could help anyone else in the future.<br>
Now I have an idea of where to look for other functions that are setting values in memory.</p>

tom: <p>That is the Python level, but the right level to solve this at is likely in ATen / the PyTorch dispatcher.<br>
The functions setting values will be the ones ending in <code>_</code> or <code>_out</code>. You can also look at native_functions.yaml and the alias informaiton there.</p>
<p>Best regards</p>
<p>Thomas</p>

nickleus27: <p>That is really helpful, Thanks Thomas.</p>

tom: <p>BTW, personally, I ccould imagine that an inplace hook that can throw errors might be neat, but I have no idea if that would cost too much overhead to be considered. This way one could either modify updates on the fly (e.g. for computed parameters) or prohibit them by raising an exception.</p>

nickleus27: <p>Thanks for the suggestion <a class="mention" href="/u/tom">@tom</a>.  What were you thinking in regards to hooks?  Maybe like a signal handler that catches a sig error from the os?  I am a second year computer science student, so I am still learning, and not totally experienced with hooks.  A couple other students and I are working on this as a team.  Also, any suggestions how we could determine if the Tensor is in read only memory?  If you are dealing with a PyObject you can cast to PyArrayObject and check with PyArray_IsWriteable, but I don’t think that will work  with Tensor object.</p>
Torch.nn loss functions

Ghasem_Abdi: <p>Hi,<br>
my inputs into the loss functions are shaped as NCHW (8, 2, 256, 256) for y_pred and as NHW (8, 256, 256) for y_true. Here C mean number of classes. During the training phase, nn.CrossEntropyLoss() works well but other loss functions like L1Loss receives errors due to that number of classes. Please advise!</p>

tom: <p>Some loss functions (CrossEntropyLoss, NLLLoss) take class “indices” as targets.</p>
<p>Others, KLDivLoss, L1Loss, … take probability distributions as the targets, so you’d need to translate your target to “one-hot” encoding. In the olden days, we used scatter for this, but now there is a utility function: <a href="https://pytorch.org/docs/stable/generated/torch.nn.functional.one_hot.html" class="inline-onebox" rel="noopener nofollow ugc">torch.nn.functional.one_hot — PyTorch 1.9.1 documentation</a></p>
<p>Best regards</p>
<p>Thomas</p>

Ghasem_Abdi: <p>Thanks Thomas for you reply.<br>
I cannot solve it. Suppose we have torch.randn(1, 2, 256, 256).to(‘cpu’) that includes 2 channels with probability. First off, how we convert it to (1, 256, 256) and where we have to apply it in the network? I mean in the loss function or in the learner?</p>

tom: <p>You want the y_true to have shape (1, 2, 256, 256) if you want to measure the L1 loss between measures. The one_hot way is to set (_, CLASS, _, <em>) to one and (</em>, 1- CLASS, <em>,</em>) to 0.</p>
Simple calculations using multiple gpu

aclifton314: <p>I am trying to use <code>pytorch</code> to perform simple calculations across multiple gpu. I am not wanting to train a machine learning model. I’ve posted this in the distributed forum <a href="https://discuss.pytorch.org/t/multi-gpu-calculations/130787/14">here</a>, but I haven’t gotten a response back about a particular question. Here is the code I have thus far:</p>
<pre><code class="lang-python">import torch
import torch.multiprocessing as mp
import torch.distributed as dist
import torch.nn.functional as F
import pandas as pd


def calc_cos_sims(rank, world_size):
    dist.init_process_group('gloo', rank=rank, world_size=world_size)
    cuda_device = torch.device('cuda:'+str(rank))
    data_path = './embed_pairs_df_million_part_' + str(rank) + '.pkl'
    tmp_df = pd.read_pickle(data_path)

    embeds_a_list = [embed_a for embed_a in tmp_df['embeds_a']]
    embeds_b_list = [embed_b for embed_b in tmp_df['embeds_b']]

    embeds_a_tensor = torch.tensor(embeds_a_list, device=cuda_device)
    embeds_b_tensor = torch.tensor(embeds_b_list, device=cuda_device)
    cosine_tensor = F.cosine_similarity(embeds_a_tensor, embeds_b_tensor)
    


def main():
    world_size = 4 #since I have 4 GPUs on a single machine
    mp.spawn(calc_cos_sims,
             args=(world_size,),
             nprocs=world_size,
             join=True)

if __name__ == 'main':
    main()
</code></pre>
<p>Basically, the code calculates the cosine similarity between two different embeddings. I have 4 GPU available to me and I have split my data into 4 slices to run on a given GPU.</p>
<p>It was recommended to use the <a href="https://pytorch.org/docs/stable/distributed.html#synchronous-and-asynchronous-collective-operations" rel="noopener nofollow ugc">pytorch collective api</a> to aggregate the results. I read through it, but I’m not entirely sure how to implement it. How would that be done in this case or is there a better way to do all of this? I’d like to be able to save off the aggregated results to a file or have available for use at a further point in my program.</p>
<p>I welcome any feedback about potential improvements. Thank you in advance!</p>

aclifton314: <p>any thoughts on this?</p>