akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Trying to understand the use of Depthwise separable convolutions in GANs

username: pasindu
cooked: <p>I’m currently trying to improve the performance of a CycleGAN model which has a couple of downsampling layers and upsampling layers combined with 6 ResNet blocks for the bottleneck.</p>
<p>I used the following implementation for it,</p>
<pre><code class="lang-auto">class DSConv1D(nn.Module):
def __init__(self, in, out, kernel, stride, padding, bias=False):
    super(DSConv1D, self).__init__()
    self.d = nn.Conv1d(in, in, kernel_size=kernel, stride=stride, padding=padding, groups=in, bias=bias)
    self.p = nn.Conv1d(in, out, kernel_size=1, stride=stride, bias=bias)

def forward(self, x):
    out = self.d(x)
    out = self.p(out)
    return out
</code></pre>
<p>Currently, I’ve swapped every possible conv1d and conv2d layer with DSC layers and it was able to reduce parameters from 22059265 to 7906710 and I gained a boost of performance.</p>
<p>So my question is, is it a good practice to replace all conv layers with DSCs and reduce this much of parameters, or is it better to only change the bottleneck layers with this? will it hurt the accuracy because of the parameter reduction?</p>
UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead. warnings.warn(warning.format(ret))

username: Mona_Jalal
cooked: <p>I am getting this warning without any line number so I am unsure which line of code it relates to. Could you please shed some light on this?</p>
<p><code>UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.</code></p>
<p>I would like to know where I am expected to used the following?<br>
<code>reduction='none'</code></p>
<p>Full line of warning:</p>
<pre><code class="lang-auto">/home/jalal/research/venv/dpcc/lib/python3.8/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
</code></pre>
<p>The dpcc virt env python is:</p>
<pre><code class="lang-auto">$ python
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.10.1+cu113'
</code></pre>
<p>and</p>
<pre><code class="lang-auto">$ lsb_release -a
LSB Version:	core-11.1.0ubuntu2-noarch:security-11.1.0ubuntu2-noarch
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.3 LTS
Release:	20.04
Codename:	focal
</code></pre>
<p>and</p>
<pre><code class="lang-auto">$ uname -a
Linux echo 5.4.0-91-generic #102-Ubuntu SMP Fri Nov 5 16:31:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
</code></pre>

username: ptrblck
cooked: <p>This warning is raised if you use the deprecated <code>reduce</code> or <code>size_average</code> argument in a criterion.<br>
Based on your last posts (e.g. <a href="https://discuss.pytorch.org/t/predicted-labels-stuck-at-1-for-test-set-where-class-0-is-20-of-data/142751/6">this one</a>) you are using:<br>
``python</p>
<pre><code class="lang-auto">self.criterion = nn.BCEWithLogitsLoss(reduce=False)
</code></pre>

username: Mona_Jalal
cooked: <p>Gotcha, thank you.<br>
Changed<br>
<code>self.criterion = nn.BCEWithLogitsLoss(reduce=False) # weighted loss</code><br>
to<br>
<code>self.criterion = nn.BCEWithLogitsLoss(reduction='none') # weighted loss</code><br>
based on your suggestion.</p>
What does "NOTE: Redirects are currently not supported in Windows or MacOs." mean in PyTorch?

username: Brando_Miranda
cooked: <p>I was running a distributed training and got:</p>
<pre><code class="lang-auto">❯ python -m torch.distributed.run --nproc_per_node=1 ~/ultimate-utils/tutorials_for_myself/my_l2l/dist_maml_l2l_from_seba.py


NOTE: Redirects are currently not supported in Windows or MacOs.

</code></pre>
<p>code: <a href="https://github.com/learnables/learn2learn/blob/master/examples/vision/distributed_maml.py" rel="noopener nofollow ugc">https://github.com/learnables/learn2learn/blob/master/examples/vision/distributed_maml.py</a></p>
<p>what does this mean?</p>

username: ptrblck
cooked: <p>I guess you are using <code>torch.distributed.elastic</code> with the <code>redirect</code> argument as seen <a href="https://pytorch.org/docs/stable/elastic/agent.html#torch.distributed.elastic.agent.server.WorkerSpec">here</a>, which isn’t supported on the mentioned platforms.</p>
Layer weight shape (10500, 768) not compatible with provided weight shape (9, 768)

username: Sammo1
cooked: <p>I got<br>
<code>ValueError: Layer weight shape (10500, 768) not compatible with provided weight shape (9, 768)</code></p>
<pre><code class="lang-auto"> embedding_matrix = torch.stack(embeddings)
 np_arr = embedding_matrix.cpu().detach().numpy()   
</code></pre>
<p>I got from <code>embedding_matrix</code> for each sentence the shape like  <code>torch.Size([6, 768])</code> then I tried to convert it into an <code>array</code> to be passed to the model but got the above problem.<br>
I need to sum all values I got from <code>embedding_matrix</code> then convert it into an array to become in this shape <code>(10500, 768)</code></p>

username: anantguptadbl
cooked: <p><a class="mention" href="/u/sammo1">@Sammo1</a> This is not how you should use embedding. You will have to revisit your architecture</p>

username: Sammo1
cooked: <p>excuse me i didn’t get what do you mean ?</p>

username: ptrblck
cooked: <p>Are you trying to pass a numpy array to a PyTorch model? This won’t work, as tensors are expected.<br>
Could you share more information about your use case and how to reproduce the issue?</p>

username: Sammo1
cooked: <p>i need to pass it to keras model but the calculations used pytoch for extract embedding so i need to pass the result as array . the problem is to concatenate every shape i got from embedding_matrix as every sentence store in embedding_matrix alone i need to sum every shape i got like ([9+8+13+10],768)</p>
LSTM Time Series Regression - How does Pytorch train on multiple batches?

username: AFordon
cooked: <p>Hello,</p>
<p>I’m following along with the Pytorch Time Series Regression (TSR) example and this article:</p>
<ul>
<li><a href="https://github.com/pytorch/examples/tree/master/time_sequence_prediction" rel="noopener nofollow ugc">Pytorch TSR Example</a></li>
<li><a href="https://towardsdatascience.com/pytorch-lstms-for-time-series-data-cd16190929d7" rel="noopener nofollow ugc">Toward Data Science TSR Example</a></li>
</ul>
<p>I would like more insight into how Pytorch trains on multiple sequences. It seems that the batches are trained in parallel, so how does loss and backpropagation get calculated? Will the individual batches receive different models (weights/bias)?</p>