akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Operation on diagonals of matrix batch

calincru: <p>I have two tensors with shapes <code>[n, d, d]</code> and <code>[n, 1]</code>, respectively, and I would like to add the latter to the diagonals of the matrices in the former.  What’s the most straightforward way of doing it?  It shouldn’t be in-place.</p>
<p>LE: I’m curious if there’s a better way than stacking <code>torch.eye</code>s.</p>

tom: <p>I think inplace is the best way, but I’ll throw in a <code>.clone()</code>, so you get to keep the input:</p>
<pre><code class="lang-python">a = torch.randn(5,4,4, requires_grad=True)
b = torch.randn(5,1, requires_grad=True) 
c = a.clone()
c.diagonal(dim1=-2, dim2=-1)[:]  +=  b

# backward works as expected:
c.sum().backward()
print(a.grad, b.grad) # ones_like(a) and full_like(b, 4)
</code></pre>
<p>Best regards</p>
<p>Thomas</p>

calincru: <p>Thanks, <a class="mention" href="/u/tom">@tom</a>, looks good.  BTW, the reason I wanted it to not be inplace was because I need it to be differentiable.  Does backward work even if it’s inplace?</p>

tom: <p>The rule of thumb is that inplace works unless it does not. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>So the two things that usually break are</p>
<ul>
<li>you move a leaf tensor into the graph (if you remove the cloning in above example - and cloning helps),</li>
<li>when <code>a</code> isn’t a leaf and whatever computed <code>a</code> wants to have <code>a</code> to compute the backward (cloning helps here, too).</li>
</ul>
<p>So the conventional wisdom is to not use inplace ops, but looking deeper, it can usually be made to work. I always joke to write a non-deep-learning PyTorch book with <a class="mention" href="/u/ptrblck">@ptrblck</a> where we would have a section on inplace ops.</p>
<p>Best regard</p>
<p>Thomas</p>
I am looking for an example

L0RE: <p>I looking for an example</p>
<p>Datum,Visitors, Holliday,weekend<br>
1.7.2019 ,  100,  0, 0<br>
2.7.2019 ,  110,  0, 0<br>
3.7.2019 ,  180,  1, 0<br>
3.7.2019 ,  110,  0, 0<br>
4.7.2019 ,  120,  0, 0<br>
5.7.2019 ,  130,  0, 0<br>
6.7.2019 ,  200,  0, 1<br>
7.7.2019 ,  180,  0, 1<br>
8.7.2019 ,  150,  0, 0</p>
<p>TO Predict 9.7.2019</p>
<p>Most examples i found are with random or with Pictures.<br>
so i Searching an Practical Example , so Visitors of an Hompage,Shop, Or Birds when using (Rain,Sun) instead weekends</p>
<p>I Have Problems with Dateobject i Tried using Unix TimeStamp,…</p>
<p>Mayby someone has an simple example that is just with some random Numbers, without beeing to complicated</p>
<p>Hope someone could help</p>

ajhanwar: <p>Just to clarify, are you looking for an example in which you predict a date given features? Or are you trying to use a date feature in prediction?</p>

L0RE: <p>I want predict visitors, when giving date+ holiday+weekend</p>

ajhanwar: <p>Perhaps you could try encoding the date values into some ordinal values (like 1.7.2019 as 0, 2.7.2019 as 1, etc.) to preserve some sort of order within the dates or one-hot-encoding if you’d prefer to leave the dates as categorical values.</p>
How to make DataParallel balance over token size rather than batch size?

luinx: <p>If I remember correctly, the default setting of DataParallel tries to split the minibatch equally into smaller patches, but in NLP model, the minibatches are sorted by length in decreasing order, that makes the first GPU under heavier load. Ideally if we can explicitly controls the splitted batch size so that the total number of tokens are roughly balanced. Is there anyway to do this efficiently?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/d/d27985cbb1cbde9319bc554a3a4b68bcccd1d1ab.png" data-download-href="https://discuss.pytorch.org/uploads/default/d27985cbb1cbde9319bc554a3a4b68bcccd1d1ab" title="30%20AM"><img src="https://discuss.pytorch.org/uploads/default/original/2X/d/d27985cbb1cbde9319bc554a3a4b68bcccd1d1ab.png" alt="30%20AM" data-base62-sha1="u1WD38vE7axSkQGvY6zvV20np23" width="690" height="185" data-dominant-color="001403"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">30%20AM</span><span class="informations">870×234 14.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Prerna_Dhareshwar: <p>Hey,</p>
<p>If you are using <code>Dataset</code> and <code>DataLoader</code>, if you set the argument <code>shuffle = True</code> in DataLoader(), it will automatically shuffle the examples within a batch.</p>