akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Topk based on weight matrix destroys gradient

NicoAdamo: <p>I have a matrix of features <code>feat = [n_shot, n_way, n_dim]</code> and a weight matrix <code>weight = [n_shot, n_way]</code>. I  want to take the top-k features from the first dimension, according to the weight matrix. Since top-k doesn’t support this natively, I’ve implemented it in the following way:</p>
<pre><code class="lang-auto">topk, ind = torch.topk(sim, k, dim=0)
feat_topk = Variable(torch.zeros(k, n_way, n_dim))
for way in range(n_way):
   for shot in range(k):
     feat_topk[shot][way] = feat[ind[shot][way]][way]
</code></pre>
<p>This works, but obviously the in-place assignment means that feat_topk has no gradient information, which I would definitely like to have as feat_topk is then used to directly compute the loss. I know torch.topk is backprop-able, but I’m not sure how to make a version of this that is. I would appreciate any help on this, thank you!</p>
Correct way to compare one model with different architectural implementation

ajoshi944: <p>I have to compare two different implementations of a network architecture with corresponding parameters. I know the number of parameters of the models is one way to make sure the implementations are similar. But, is there any other/ correct way to do this?<br>
Thank you.</p>

ptrblck: <p>You could additionally try to print the graphs of both models (assuming you want to compare the forward/backward pass) and check if the computation graphs are also equal.</p>