akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
CrossEntroy backward not correct

username: andeyeluguo
cooked: <p>we can get the grad of output</p>
<pre><code class="lang-auto">&gt;&gt;&gt; loss = nn.CrossEntropyLoss()
&gt;&gt;&gt; input = torch.randn(3, 5, requires_grad=True)
&gt;&gt;&gt; target = torch.empty(3, dtype=torch.long).random_(5)
&gt;&gt;&gt; output = loss(input, target)
&gt;&gt;&gt; output.backward()
&gt;&gt;&gt; input.grad
</code></pre>
<p>the input.grad value is deemed equal to the softmax(input)'s value, and the other is equal to 1 - softmax(input),<br>
but the result above is not.</p>
<p>ps: why the web has  no email notification</p>

username: InnovArul
cooked: <aside class="quote no-group" data-username="andeyeluguo" data-post="1" data-topic="150168">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/andeyeluguo/48/47823_2.png" class="avatar"> andeyeluguo:</div>
<blockquote>
<p>the other is equal to 1 - softmax(input),</p>
</blockquote>
</aside>
<p>Can you point out the incorrectness by writing the output?</p>
<p>In my observation, the gradient seems correct.<br>
For the logits <code>z</code>, the target class <code>i</code> and loss L,</p>
<pre><code class="lang-auto">dL / dz_i = softmax(z_i) - 1
dL / dz_j = softmax(z_j)
</code></pre>
Is there a way to dynamically use image size and batch size when training?

username: ysleer
cooked: <p>Hello</p>
<p>I am going to use multiprocessing to do my training.</p>
<p>When learning, I want to train using various image sizes and batch sizes.</p>
<p>For example, when the image size is 512x512, the batch size is 4, when the image size is 256x256, the batch size is 8, and when the image size is 128x128, the batch size is 16.</p>
<p>I want to be able to train the image size dynamically in 512x512, 256x256, 128x128, etc. during the learning process.</p>
<p>For example, you could create a dataloader for each image size and put it in a list, and use multiprocessing.</p>
<p>This is just my idea and I don’t know what to do specifically.</p>
<p>Please let me know if there is a way to dynamically use image size and batch size during training.</p>
<p>Please help, thank you.</p>

username: Dennis_Sakva
cooked: <p>Good question. I would like to know that too. The only thing that I was able to come up with is creating batches 16x1x512x512 and then cropping them to the desired size. But it’s terribly inefficient.</p>

username: c_cj
cooked: <p>You can set the batchsize with maxsize (e.g. 16) in Dataloader. Then during the training, if you achieve 16x3x512x512, you can split them into 4 groups as (4x3x512x512)x4. And optimize them four times.<br>
If the input size is 16x3x256x256, you can split them into 2 groups.</p>
Demand forecasting with the Temporal Fusion Transformer

username: Aditya_Srivastava
cooked: <p>Hello everyone,</p>
<p>I want to use TFT model for my use case. I am able to train the model using the tutorial provided in the below link<br>
<a href="https://pytorch-forecasting.readthedocs.io/en/stable/tutorials/stallion.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch-forecasting.readthedocs.io/en/stable/tutorials/stallion.html</a></p>
<p>However, I am facing issues with the shape of the output that the TFT Model is producing.</p>
<p>I am using a tabular data where there are 6 columns. I am using these 6 columns to predict the values of a single column among these 6. I am trying to test the trained TFT model on 5000 samples but when I am checking the shape of ouput it is only 2851x1.</p>
<p>Also I am not sure whether this model is built to predict multiple targets i.e. values of multiple columns. Because the same output shape issue is coming in this case as well.</p>
<p>Is there any blog or help where I can understand the solution to this issue?</p>
<p>Please help me it is little urgent.</p>
Yolov5 convert to TorchScript

username: liyuyuan6969
cooked: <h1><a name="describe-1" class="anchor" href="#describe-1"></a>Describe</h1>
<p>hi,guys,i want to convert yolov5 pytorch model to TorchScript,then i find source code have <code>if else</code> in forward function,so i decide to use torch.jit.script rather than torch.jit.trace,bacause i know it is not support control statement for example <code>if else</code>,and i change many details to fit torchScript,but i failed,so i google how to convert yolov5 to torchScript and find it has a export.py,which can convert to torchScript or onnx,i find it uses torch.jit.trace,why?torchScript support if else?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/5/35663eb3bc8ab3dc321a4185440b325a7d3d5086.png" data-download-href="https://discuss.pytorch.org/uploads/default/35663eb3bc8ab3dc321a4185440b325a7d3d5086" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/5/35663eb3bc8ab3dc321a4185440b325a7d3d5086.png" alt="image" data-base62-sha1="7ComBQVlImZEn8e9yTG4vQDXQZo" width="690" height="405" data-dominant-color="CFCFAC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1289×758 44.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="liyuyuan6969" data-post="1" data-topic="150180">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/liyuyuan6969/48/48999_2.png" class="avatar"> liyuyuan6969:</div>
<blockquote>
<p>i find it uses torch.jit.trace,why?torchScript support if else?</p>
</blockquote>
</aside>
<p>No, as you’ve already said, tracing a model will not capture data-dependent control flow, conditions etc.<br>
I don’t know how the <code>export.py</code> method is defined, but I would guess that <code>model.eval()</code> might be a requirement and the traced model will then use the <code>not self.training</code> code path only.</p>

username: liyuyuan6969
cooked: <p>yes,you are right,self.training is false,you means if i define a global variable to control <code>if else</code> conditions will be success,right?</p>
Precision of half tensor

username: Anikily
cooked: <p>I encounter a problem when using the torch.amp, and i found that the half type tensor leads to the different results.</p>
<pre><code class="lang-auto">a = torch.tensor([2111., 2010., 1970., 1781., 1925., 1993., 2122., 2078., 2094., 2093., 1887., 2103., 1807., 2101., 1766., 1688., 1564., 2119., 1830., 1230.])
b = torch.tensor([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
a@b -&gt; tensor(2111.)
a.half()@b.half() -&gt; tensor(2112., dtype=torch.float16)
</code></pre>
<p>How to avoid this problem?<br>
I will appreciate your help!</p>

username: ptrblck
cooked: <p>You won’t be able to avoid the rounding in <code>float16</code> as the integer rounding starts at <code>2049</code> and your result is larger. If you need more precision using these values, disable <code>autocast</code> for this region of your code.</p>
Converting DataLoader and transform code to Android for SuperSlowMo

username: filloax
cooked: <p>I’m currently making a (private) fork of <a href="https://github.com/avinashpaliwal/Super-SloMo" rel="noopener nofollow ugc">this implementation</a> of NVIDIA’s SuperSlowMo to adapt it to Android, for a university project. In short, it increases the frames in a video. In it’s main video_to_slomo.py you can see that, after converting the input video to png frames, it transforms and loads them with this block of code:</p>
<pre><code class="lang-python">    mean = [0.429, 0.431, 0.397]
    std  = [1, 1, 1]
    normalize = transforms.Normalize(mean=mean, std=std)
    negmean = [x * -1 for x in mean]
    revNormalize = transforms.Normalize(mean=negmean, std=std)

    if (device == "cpu"):
        transform = transforms.Compose([transforms.ToTensor()])
        TP = transforms.Compose([transforms.ToPILImage()])

    else:
        transform = transforms.Compose([transforms.ToTensor(), normalize])
        TP = transforms.Compose([revNormalize, transforms.ToPILImage()])

    # Load data

    videoFrames = dataloader.Video(root=extractionPath, transform=transform)
    videoFramesloader = torch.utils.data.DataLoader(videoFrames, batch_size=args.batch_size, shuffle=False)
</code></pre>
<p>Of course, the Pytorch Android API is lightweight, and it doesn’t look like it has either something analogous to the dataloader or the transform modules. I’m a bit lost in how I could adapt that to Java for making the app, though.</p>

username: Linbin
cooked: <p>PyTorch Android API is just a JNI wrapper on top of PyTorch C++ API. You may need to write a JNI wrapper to call the underlying C++ API (if exists) of that library, or just do it in Android native (C++) library.</p>
[IOS metal] Could not run 'metal_prepack::linear_run' with arguments from the 'CPU' backend

username: kunle12
cooked: <p>Hi, I’m very new for using PyTorch on iOS so my question may sound silly. I am keep having this <strong>Could not run ‘metal_prepack::linear_run’ with arguments from the ‘CPU’ backend</strong> error. While the error message is very clear, I have already converted my input tensor with <strong>at::Tensor(input).metal()</strong> before pass it into the model forward method. I am not sure at what step I did something wrong. The optimised torchscript graph (show below) seems to be taking my input directly (which is correct). so I am lost.</p>
<pre><code class="lang-auto">%prepack_folding_forward._jit_pass_packed_weight_0 : __torch__.torch.classes.metal.LinearOpContext = prim::GetAttr[name="prepack_folding_forward._jit_pass_packed_weight_0"](%self.1)
  %124 : Tensor = metal_prepack::linear_run(%encoder_outputs.1, %prepack_folding_forward._jit_pass_packed_weight_0)
</code></pre>
<p>Furthermore, the error message from the PyTorch library on gives<br>
<strong>’metal_prepack::linear_run’ is only available for these backends: [Named, VE, QuantizedCUDA, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID].</strong></p>
<p>This doesn’t really help the debugging process <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=12" title=":frowning:" class="emoji" alt=":frowning:" loading="lazy" width="20" height="20"></p>
<p>Anyway, I hope someone who can help me to diagnose my problem.</p>
<p>Thanks!</p>
<p>Xun.</p>

username: Linbin
cooked: <aside class="quote no-group" data-username="kunle12" data-post="1" data-topic="148783">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kunle12/48/48500_2.png" class="avatar"> kunle12:</div>
<blockquote>
<p>metal_prepack::linear_run</p>
</blockquote>
</aside>
<p>Xun, did you follow this tutorial (<a href="https://pytorch.org/tutorials/prototype/ios_gpu_workflow.html#metal-compatible-model" class="inline-onebox" rel="noopener nofollow ugc">(Prototype) Use iOS GPU in PyTorch — PyTorch Tutorials 2.1.1+cu121 documentation</a>) to build pytorch with metal support (or download the nightly from cocoapods) and optimize the model using metal backend? The error message suggests you are trying to run this model on CPU.</p>
Some questions about feeding 2 inputs to any MLP model & train type

username: tnguyen87
cooked: <p>Greetings,</p>
<p>I am constructing a simple MLP model and trying to use 2 different inputs (A and B) to predict y. Before training a model, I need to feed inputs to the model correctly. I define a first layer to be nn.Linear(2,next_dim). I would like to ask whether I need to flatten out all inputs by concatenate(A,B) before feeding to a model? I believe that I have to but not really confident about this! Can anyone confirm this for me by chance, please?</p>
<p>Secondly, if I want to train 2 different models (more like multi model), can I train each model with different type of sending data to a model differently? what I meant is that I use a dataloader to train model A and use a normal batch (dataloader does not involved) to train model B</p>
<p>Thank you in advances,<br>
Tommy</p>

username: InnovArul
cooked: <aside class="quote no-group" data-username="tnguyen87" data-post="1" data-topic="150149">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/a698b9/48.png" class="avatar"> tnguyen87:</div>
<blockquote>
<p>nn.Linear(2,next_dim)</p>
</blockquote>
</aside>
<p>What is 2 here?<br>
Does it mean A, B are of 1-D each?</p>
<aside class="quote no-group" data-username="tnguyen87" data-post="1" data-topic="150149">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/a698b9/48.png" class="avatar"> tnguyen87:</div>
<blockquote>
<p>whether I need to flatten out all inputs by concatenate(A,B)</p>
</blockquote>
</aside>
<p>It would help if you could spell out the dimensions of A &amp; B.</p>
<aside class="quote no-group" data-username="tnguyen87" data-post="1" data-topic="150149">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/a698b9/48.png" class="avatar"> tnguyen87:</div>
<blockquote>
<p>can I train each model with different type of sending data to a model differently?</p>
</blockquote>
</aside>
<p>Yes, you can. There is no restriction.</p>
How to get manhattan distance matrix between two set of vectors?

username: Audrey
cooked: <p>Hi, I want to know if there is a packed function in PyTorch to calculate the Manhattan distance between vectors. For your information, the Manhattan distance between vector <strong>a</strong> and vector <strong>b</strong> is calculated as:</p>
<p>distance = sum(abs(<strong>a</strong>-<strong>b</strong>))</p>
<p>Now I have a large set of vectors <strong>A</strong> in the shape of (5000, 100), and a large set <strong>B</strong> in the shape of (150000, 100), I want to get the distance matrix that is in the shape of (5000, 150000). Each element [i][j] in the distance matrix is the manhattan distance between <strong>A</strong>[i] and <strong>B</strong>[j].</p>
<p>I’ve tried a naive method as below, but it’s unbearably slow.</p>
<pre><code class="lang-auto">for i in range(...):
    for j in range(...):
        matrix[i][j] = sum(abs(A[i]-B[j]))
</code></pre>
<p>I want to know if there is a much quicker way to calculate this, such as an optimized packed function? Thank you!</p>

username: Audrey
cooked: <p>Hi, I think I figured out the solution. I can use <code>torch.cdist(x1 , x2, p=2.0)</code> function which computes batched the p-norm distance between <strong>x1</strong> and <strong>x2</strong>. What I need to do is just turn into <strong>x1</strong> and <strong>x2</strong> batched before calling this function.</p>
<pre><code class="lang-auto">x1.unsqueeze_(0)
x2.unsqueeze_(0)
output = torch.cdist(x1, x2, p=1)
</code></pre>
<p>Here 1-norm distance is Manhattan distance.</p>

username: mb1996
cooked: <p>Hi @ Audrey,</p>
<p>You might also consider looking at the built in torch operation “L1Loss”, which can be set up as averaged or not. The average takes place over all elements. Not sure if this will serve your purpose. Please see the following:</p>
<p><a href="https://pytorch.org/docs/stable/generated/torch.nn.L1Loss.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.L1Loss.html</a></p>
<p>Sincerely, mb1996</p>

username: Audrey
cooked: <p>Thanks <a class="mention" href="/u/mb1996">@mb1996</a> ! Just woke up and saw your reply.</p>
<p>I tried your idea about “L1Loss” below, but the <code>input</code> and <code>target</code> in its parameters are required to be the same shape, which unfortunately doesn’t work in my situation.</p>
<pre><code class="lang-auto">loss = nn.L1Loss()
input = torch.randn(2, 2, requires_grad=False)
target = torch.randn(3, 2)
output = loss(input, target)
</code></pre>
<p>The error I got:</p>
<pre><code class="lang-auto">RuntimeError: The size of tensor a (3) must match the size of tensor b (2) at non-singleton dimension 0
</code></pre>
<p>Anyway, thanks for your reply.</p>

username: mb1996
cooked: <p>Hi <a class="mention" href="/u/audrey">@Audrey</a>,</p>
<p>No problem. That is true. I didn’t think of that initially but you are right, that dimension has to match. I hope that the solution you found works out to be fast enough.</p>
<p>Sincerely, mb1996</p>
Reproducible CNN model with data transforms?

username: Z_Huang
cooked: <p>Hi community,</p>
<p>I tried to create my reproducible CNN model, but it seems not to happen on my end. I set the seeds with the following train_loader. Each re-train (train/valid/test loss and acc for each epoch) always looks slightly different. May I know why?</p>
<pre><code class="lang-auto">seed = 1
os.environ["PL_GLOBAL_SEED"] = str(seed)
random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed_all(seed)

data_transform = transforms.Compose([
    transforms.Resize((32, 32)),
    transforms.RandomHorizontalFlip(),
    transforms.RandomCrop((32, 32), padding = 4),
    transforms.RandomRotation(10),
    transforms.ColorJitter(),
    transforms.RandomAffine(10),
    transforms.ToTensor(),
    transforms.Normalize((0.4941, 0.4851, 0.4504), (0.2467, 0.2430, 0.2617)),
])

data = datasets.CIFAR10('data', train=True, download=True, transform=data_transform)

train_loader  = DataLoader(data,  batch_size=batch_size, num_workers=num_workers, shuffle=True)
</code></pre>

username: Z_Huang
cooked: <p>Find a good answer from another post: <a href="https://discuss.pytorch.org/t/reproducibility-with-all-the-bells-and-whistles/81097">Reproducibility with all the bells and whistles</a></p>
Question about a special case of right padding

username: lvkd84
cooked: <p>Hello!</p>
<p>I want to invert the padding of the last dimension of my input tensor from the left to the right.</p>
<p>For example, the last dimension of my tensor may look like this:</p>
<p>[0,0,0,1,2,3]<br>
[0,1,2,3,4,5]<br>
[0,0,1,2,3,4]<br>
[1,2,3,4,5,6]</p>
<p>I want to transform such input into this:</p>
<p>[1,2,3,0,0,0]<br>
[1,2,3,4,5,0]<br>
[1,2,3,4,0,0]<br>
[1,2,3,4,5,6]</p>
<p>I wonder if there is a clean way to achieve this? My tensor is quite large so I’d like to find a solution that does not require iterating through all rows/columns.</p>
<p>Thank you!</p>
Difference between TCP and ENV init method

username: Frank_Leeeee
cooked: <p>Hi, I am wondering if the init_method to initialise the process group indeed causes any diff in performance? The first init_method is to use <code>tcp://&lt;host&gt;:&lt;port&gt;</code>, the second is to use the default env.</p>

username: H-Huang
cooked: <p>No, there aren’t performance differences between the two init methods. Default env initialization creates a TCPStore instance (same as <code>tcp://</code> init method) but uses environment variables MASTER_ADDR and MASTER_PORT as the <code>&lt;host&gt;:&lt;port&gt;</code> see <a href="https://github.com/pytorch/pytorch/blob/master/torch/distributed/rendezvous.py" class="inline-onebox" rel="noopener nofollow ugc">pytorch/rendezvous.py at master · pytorch/pytorch · GitHub</a></p>
What are the default initialised matrix values of the convolution kernels in PyTorch?

username: CCL
cooked: <p>Different kernel values are useful for different tasks, e.g. edge detection. What kernel matrix values does PyTorch use by default?</p>

username: Nikronic
cooked: <p>This might help</p><aside class="quote quote-modified" data-post="1" data-topic="84696">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/taylor_webb/48/16355_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/clarity-on-default-initialization-in-pytorch/84696">Clarity on default initialization in pytorch</a> 
  </div>
  <blockquote>
    According to the <a href="https://pytorch.org/docs/stable/nn.html#linear-layers" rel="noopener nofollow ugc">documentation</a> for torch.nn, the default initialization uses a uniform distribution bounded by 1/sqrt(in_features), but <a href="https://github.com/pytorch/pytorch/blob/3b1c3996e1c82ca8f43af9efa196b33e36efee37/torch/nn/modules/linear.py#L79" rel="noopener nofollow ugc">this code</a> appears to show the default initialization as Kaiming uniform. Am I correct in thinking these are not the same thing? And if so, perhaps the documentation can be updated? 
Does anyone know the motivation for this choice of default? In particular, the code appears to define a negative slope for the subsequent nonlinearity of sqrt(5), whereas the default…
  </blockquote>
</aside>

<p>Bests</p>

username: dkleine
cooked: <blockquote>
<p>And if so, perhaps the documentation can be updated? Does anyone know the motivation for this choice of default?</p>
</blockquote>
<p>I absolutely agree with that statement in the linked post. It took me quite a lot of time to find a comprehensive answer to the question “what are the default values for the kernel used in a convolutional layer in a CNN?”.</p>
<p>I think the documentation needs some improvements here. Some suggestions:</p>
<ul>
<li>default initializations (torch.nn.init) need to be linked to relevant pages, e.g. Conv2D</li>
<li>the web page to default initializations (torch.nn.init) needs more documentation, e.g. that “Conv{1,2,3}D” refers to values (weights) used in the convolution kernel initialized by default. There is also no information that it uses Kaiming uniform distribution (for me as a beginner in CV, this was not that obvious).</li>
</ul>

username: ptrblck
cooked: <p>These sound like great suggestions! Would you be interesting in adding these to the documentation? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

username: dkleine
cooked: <p>Sure, I would be happy to contribute! <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=12" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:" loading="lazy" width="20" height="20"></p>
Calculating the cross attention between the bottleneck and the output of the decoder

username: ZSheikhb
cooked: <p>I would like to compute cross attention between the bottleneck layer of an autoencoder (with the size of <code>(2x128x6x6)</code>) and the output of the decoder layer <code>(2x25x48x48)</code>. I’d like to keep the positional information and finally I would like to weigh the bottleneck layer output with the cross-attention values. I don’t have a clear idea how it should be down. I will appreciate if someone can advise me whether it is possible or not and refer me to an example code or paper. Thanks</p>
Model Parameter as Function of another parameter

username: sesale
cooked: <p>I am trying to implement an algorithm where I have randomly initialized model parameters θ and a weight matrix W. I have different tasks and my loss function is split into separate losses (one loss for each sample within each task). the weight matrix contains the corresponding weights per sample loss. I first have to perform gradient descent w.r.t. to θ. Then I have to write θ as of W and then take the gradient w.r.t. W (I think something like e.g. autograd.grad(loss,W)). My question is: In order to take the gradient w.r.t. W, do I first have to register W as model parameters? Otherwise, the gradient of the loss will of course be None w.r.t. to W, but since the parameters are weights for the losses and not directly model parameters, I am somehow confused. If so, I would assume that when taking the gradient w.r.t. to θ, I cannot do something like</p>
<pre><code class="lang-auto">SGD(model.parameters(), lr=lr) 
</code></pre>
<p>but rather have to exclude the weights? Something like</p>
<pre><code class="lang-auto">SGD(model.parameters()[:-1], lr=lr)?
</code></pre>

username: suraj.pt
cooked: <p>Can you share a minimal code snippet of your problem and expected solution? When you call <code>backward()</code> on the loss, it will compute gradients in the <code>W.grad</code> attribute. Registering in the optimizer enables it to update every time you call <code>optimizer.step()</code>.</p>
How to save audio as .wav from waveform and sample rate?

username: ADONAI_TZEVAOT
cooked: <p>I have a speech1.wav . I have applied some effects on the waveform of this audio file and I have a new waveform and sample rate. How do I create a new speech2.wav file using this new waveform and sample rate? Please help!</p>

username: suraj.pt
cooked: <p>From the torchaudio tutorial <a href="https://pytorch.org/audio/stable/tutorials/audio_io_tutorial.html#saving-audio-to-file:" class="inline-onebox">Audio I/O — Torchaudio 2.1.1 documentation</a></p>
<blockquote>
<p>To save audio data in formats interpretable by common applications, you can use <a href="https://pytorch.org/audio/stable/torchaudio.html#torchaudio.save"><code>torchaudio.save()</code></a>.</p>
</blockquote>
Weighted binary cross entropy and weighted sampler

username: aleemsidra
cooked: <p>I am working with a highly imbalanced dataset. I have read a few papers using weighted cross-entropy loss for class imbalance. I have also seen some examples using a weighted sampler. However, using a weighted sampler, the model ‘miss’ a large portion of the majority class during each epoch, since the minority class is now overrepresented in the training batches. What is the common practice to deal with imbalanced datasets? Both weighted samplers and weighted cross-entropy used together? Or only one of them is used?</p>

username: JuanFMontesinos
cooked: <p>You use one of them only.<br>
The purpose is compensate the over-representation of one class over the rest. To do so you can either modify the sampler (classes are loaded in a balanced way, thus the optimization problem converges towards a joint solution) or penalty the values of the loss (so there is an imbalance in the data but you force the network to learn “more” from those samples which are under represented).</p>
<p>If you use both you end up in a non-balanced scenario again.</p>
Can't log in with my regular account to pytorch forum "App not set up"

username: Damien_Menigaux
cooked: <p>Hi. When I try to log to Pytorch forum with my Facebook account, I end up on a Facebook bug page where I can read :<br>
“App not set up<br>
This app is still in development mode, and you don’t have access to it. Switch to a registered test user or ask an app admin for permissions.”</p>
<p>I lost access to my email account, so I can’t ask for a new password. I can only connect to Facebook. So I’m effectively locked out of Pytorch forum. How can I retrieve my account, please ?</p>

username: ptrblck
cooked: <p>I’m not sure which app you are referring to, but based on <a href="https://stackoverflow.com/questions/30085246/app-not-setup-this-app-is-still-in-development-mode">this</a> post it seems that this particular app might need to change some settings.</p>

username: Damien_Menigaux
cooked: <p>That’s the point. I’m not using an app. I’m simply trying to log in with my browser. I’m guessing that the hosting of the pytorch forum itself and its interaction with facebook might have an issue.</p>

username: Damien_Menigaux
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Hi I’m still very interested in being able to log in to my account. Do you know anything I could try ? This is not using any app, I’m simply trying to log in to the forum through facebook auth on my web browser.</p>
TypeError: tuple is not a Module subclass

username: Tuna_can
cooked: <p>Hi all, I’m new to pytorch.<br>
I try to implement Resnet currently, but I have a problem about nn.sequential</p>
<pre><code class="lang-auto">class PlainBlock(nn.Module):
  def __init__(self, Cin, Cout, downsample=False):
    super().__init__()
    self.net = None

    padding_size1 = 1
    kernel_size1 = 3
    padding_size2 = 1
    kernel_size2 = 3
    
    self.net = nn.Sequential(
      ('Spatial Batch normalization1', nn.BatchNorm2d(Cin)),
      ('ReLU1', nn.ReLU()),
      ('Conv1', nn.Conv2d(Cin, Cout, kernel_size = kernel_size1, stride = stride_1, padding = padding_size1, bias = False)),
      ('Spatial Batch normalization2', nn.BatchNorm2d(Cout)),
      ('ReLU2', nn.ReLU()),
      ('Conv2', nn.Conv2d(Cout, Cout, kernel_size = kernel_size2, stride = 1, padding = padding_size2, bias = False))
    )
   
    ############################################################################

  def forward(self, x):
    return self.net(x)
</code></pre>
<p>this is the error i got</p>
<pre><code class="lang-auto"> TypeError: tuple is not a Module subclass
</code></pre>
<p>Here is the traceback</p>
<pre><code class="lang-auto">TypeError                                 Traceback (most recent call last)

&lt;ipython-input-32-ccdd2c3e2e78&gt; in &lt;module&gt;()
      1 data = torch.zeros(2, 3, 5, 6)
      2 # YOUR_TURN: Impelement PlainBlock.__init__
----&gt; 3 model = PlainBlock(3, 10)
      4 if list(model(data).shape) == [2, 10, 5, 6]:
      5   print('The output of PlainBlock without downsampling has a *correct* dimension!')

2 frames

/content/drive/My Drive/HW9/pytorch_autograd_and_nn.py in __init__(self, Cin, Cout, downsample)
    357       ('Spatial Batch normalization2', nn.BatchNorm2d(Cout)),
    358       ('ReLU2', nn.ReLU()),
--&gt; 359       ('Conv2', nn.Conv2d(Cout, Cout, kernel_size = kernel_size2, stride = 1, padding = padding_size2, bias = False))
    360     )
    361 

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/container.py in __init__(self, *args)
     89         else:
     90             for idx, module in enumerate(args):
---&gt; 91                 self.add_module(str(idx), module)
     92 
     93     def _get_item_by_idx(self, iterator, idx) -&gt; T:

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in add_module(self, name, module)
    376         if not isinstance(module, Module) and module is not None:
    377             raise TypeError("{} is not a Module subclass".format(
--&gt; 378                 torch.typename(module)))
    379         elif not isinstance(name, torch._six.string_classes):
    380             raise TypeError("module name should be a string. Got {}".format(


</code></pre>
<p>Hope someone can help<br>
Sincerely!</p>

username: Tuna_can
cooked: <p>I got the answer<br>
thanks all !</p>
<p>I forget to use orderdict in sequential</p>