akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Is there a cost-sensitive loss function implementation in PyTorch?

username: Horacio_Screen
cooked: <p>I would like to implement a cost-sensitive loss function in PyTorch. My two-class training dataset is heavily imbalanced, where 75% of the data are label ‘0’ and only 25% of the data are label ‘1’.</p>
<p>I am new to PyTorch but my supervisor is adamant that I use it (they have more experience with it). I found some implementations in Keras, but I am not that strong in coding to be able to port it over to PyTorch.</p>
<p>I have read around to find some resources to create a cost-sensitive loss function. This paper uses something which I think might work (<a href="https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9417097" class="inline-onebox" rel="noopener nofollow ugc">IEEE Xplore Full-Text PDF:</a>), but I do not understand how the code is implemented despite having access to it here (<a href="https://github.com/emadeldeen24/AttnSleep/blob/f993511426900f9fca20594a738bf8bee1116381/utils/util.py" class="inline-onebox" rel="noopener nofollow ugc">AttnSleep/util.py at f993511426900f9fca20594a738bf8bee1116381 · emadeldeen24/AttnSleep · GitHub</a>).</p>
<p>This website describes the math very detailedly but I do not understand it: <a href="https://medium.com/rv-data/how-to-do-cost-sensitive-learning-61848bf4f5e7" class="inline-onebox" rel="noopener nofollow ugc">How to do Cost-Sensitive Learning | by Joe Tenini, PhD | Red Ventures Data Science &amp; Engineering | Medium</a></p>
<p>Here is an implementation in Keras which I have trouble with converting to PyTorch: <a href="https://towardsdatascience.com/fraud-detection-with-cost-sensitive-machine-learning-24b8760d35d9" rel="noopener nofollow ugc">https://towardsdatascience.com/fraud-detection-with-cost-sensitive-machine-learning-24b8760d35d9</a></p>
<p>I also found this implementation in PyTorch, but have trouble with understanding it: <a href="https://discuss.pytorch.org/t/dealing-with-imbalanced-datasets-in-pytorch/22596/21" class="inline-onebox">Dealing with imbalanced datasets in pytorch - #21 by snakers41</a></p>
<p>Could you please help me to understand the last link’s implementation of the cost-sensitive loss function?</p>
<p>Thank you.</p>
Torch.randn_like - generator argument?

username: vgoklani
cooked: <p>Curious, why doesn’t the torch.randn_like() function take a generator or a random seed as an argument?</p>
<p>How should we reproduce results?</p>
<p><a href="https://pytorch.org/docs/stable/generated/torch.randn_like.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.randn_like.html</a></p>
<p>It’s odd, because this function does indeed accept a generator as an argument:</p>
<p><a href="https://pytorch.org/docs/stable/generated/torch.randn.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.randn.html</a></p>
<p>Thanks!</p>

username: ptrblck
cooked: <p>It seems this argument wasn’t implemented yet, so would you be interested in adding it?</p>
Unexpected key(s) in state_dict

username: HK0930
cooked: <p>I get an RuntimeError: Error(s) in loading state_dict for DataParallel: I when I tried to load a model as follows;</p>
<pre><code class="lang-auto">model = ModelMark1()
model = torch.nn.DataParallel(model)
checkpoint = torch.load(checkpoint_path, map_location=device)
</code></pre>
<p>The details of error;</p>
<pre><code class="lang-auto">Unexpected key(s) in state_dict: "module.velocity_model.some_layer.gen_fc.linear.weight", "module.velocity_model.some_layer.gen_fc.linear.bias".
</code></pre>
<p>Here is the printed checkpoint dictionary;</p>
<pre><code class="lang-auto">odict_keys(['module.spectrogram_extractor.stft.conv_real.weight', 'module.spectrogram_extractor.stft.conv_imag.weight', 'module.logmel_extractor.melW', 'module.bn0.weight', 'module.bn0.bias', 'module.bn0.running_mean', 'module.bn0.running_var', 'module.bn0.num_batches_tracked', 'module.velocity_model.conv_block1.conv1.weight', 'module.velocity_model.conv_block1.conv2.weight', 'module.velocity_model.conv_block1.bn1.weight', 'module.velocity_model.conv_block1.bn1.bias', 'module.velocity_model.conv_block1.bn1.running_mean', 'module.velocity_model.conv_block1.bn1.running_var', 'module.velocity_model.conv_block1.bn1.num_batches_tracked', 'module.velocity_model.conv_block1.bn2.weight', 'module.velocity_model.conv_block1.bn2.bias', 'module.velocity_model.conv_block1.bn2.running_mean', 'module.velocity_model.conv_block1.bn2.running_var', 'module.velocity_model.conv_block1.bn2.num_batches_tracked',  'module.velocity_model.fc.weight', 'module.velocity_model.fc.bias', 'module.velocity_model.some_layer.gen_fc.linear.weight', 'module.velocity_model.some_layer.gen_fc.linear.bias'])
</code></pre>
<p>I am not too sure what are the problem here since the module.velocity_model. keys are loaded.<br>
Also the model structure, names are not changed in ModelMark1.<br>
Please let me know if you have advice/clue/solution on this issue.<br>
Thank you so much for your help in advance!</p>

username: lfsalasg
cooked: <p>Check for typos or provide the state dictionary of the new model you are trying to set.</p>

username: HK0930
cooked: <p>There is no typo I think since it is generated by pytorch and I just loaded them.<br>
The model I trained is one which have those unexpected keys and is the same as ModelMark1. I do not known why there is model does not recognize the 2 keys.<br>
Is there any way to check all expected keys by the ModelMark1?</p>

username: HK0930
cooked: <p>I think the problem is that the layer contains these 2 unexpected keys are not initialized in constructor but in a forward method. Does this cause not creating keys when instantiating the ModelMark1?<br>
If so is there any way to add “mock keys” of the 2 keys in the state_dict and run the model for evaluation or testing purposes?</p>
Lstm input dimension

username: NoWay2Guess
cooked: <p>I am working on a project of time series data using lstm. Both inputs and targets are sinusoidal signals, with the same size of (20,250), 20 is the n_samples, and 250 is the input_features. I am confused about the input dimension of nn.LSTM(). For parameters, my settings are nn.LSTM(input_size = 1,hidden_size = 10,num_layers = 2, batch_first=True). As the for the input_size, does this always need to be one, I use one because I saw an example using LSTMcell() use 1 as the input_size. For inputs of nn.LSTM(), if I use a customize dataset and dataloader, assume the batch size is 4, so the input of x has a shape of (4,250,1), with batch_first=True?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/4/f426bc5e91596af1f7b6828ac17e33fa0f548812.png" data-download-href="https://discuss.pytorch.org/uploads/default/f426bc5e91596af1f7b6828ac17e33fa0f548812" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/4/f426bc5e91596af1f7b6828ac17e33fa0f548812_2_690x197.png" alt="image" data-base62-sha1="yPRuCxrXRTu4wkEZ5oTadC7cYaC" width="690" height="197" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/4/f426bc5e91596af1f7b6828ac17e33fa0f548812_2_690x197.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/4/f426bc5e91596af1f7b6828ac17e33fa0f548812_2_1035x295.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/4/f426bc5e91596af1f7b6828ac17e33fa0f548812.png 2x" data-dominant-color="EEE9E5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1166×333 113 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: vdw
cooked: <p>I’m a bit confused. <code>input_size</code> refers to the number of features, which is 250 in your case. Not sure why you have <code>input_size=1</code></p>
<p>Or do you mean you have 250 data points for each of you 20 samples, and each data point is a single value, i.e., 1-dimensional. Then it has to be <code>input_size=1</code>, yeah. So if each individual data point in a sample sequence is not represented by a single (1-diminsional) value, then you would use larger <code>input_size</code> values. For example, in NLP, each word are often represented as, say, 300-dimensional vector. In this case, I would need <code>input_size=300</code>.</p>
Cholesky decomposition with a sparse matrix

username: nazareem
cooked: <p>Hi Guys,</p>
<p>Thanks for the great work!</p>
<p>Is there a way to perform the operations <code>torch.linalg.cholesky</code> and <code>torch.cholesky_solve</code> with sparse matrices?</p>
<p>Thank you in advance!</p>
Installing C++ extensions in pip's sandbox

username: Jegp
cooked: <p>When using the recent PEPs for pyproject.toml, pip will install and compile projects in a sandbox (<a href="https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/?highlight=isolation" class="inline-onebox" rel="noopener nofollow ugc">pyproject.toml - pip documentation v22.3</a>). Given that PyTorch cares deeply about which <em>variant</em> is installed (CUDA/CPU/…), any C++ extensions being compiled/linked in that environment might target the wrong PyTorch version.<br>
Are there ways to ensure that the PyTorch <em>outside</em> the environment is used during pip installations? Do you have a favorite/recommended method to get around this sandboxing to avoid double installations?</p>
<p>One solution that sprung to mind is to pass the <code>--extra-index-url</code> config to <code>pip</code> during install. But that still means re-installing PyTorch (along with all the other dependencies) and 2) putting a burden on the user to add the (verbose) flag.</p>
Fully connected layer in forward()

username: NoWay2Guess
cooked: <p>Hi, I am wondering if it is possible to place self.fc = nn.Linear(hidden_size, x.shape[1]) in the forward(self,x). Because I use Time Series cross validation to split the dataset, which means the seq_len for train_dataset changes over time.</p>
<p>class LSTM(nn.Module):<br>
def <strong>init</strong>(self, input_size, hidden_size, num_layers, p):<br>
super(LSTM,self).<strong>init</strong>()<br>
self.hidden_size = hidden_size<br>
self.num_layers = num_layers<br>
self.dropout = nn.Dropout(p)<br>
self.lstm = nn.LSTM(input_size = input_size,<br>
hidden_size = hidden_size,<br>
num_layers = num_layers,<br>
batch_first = True,<br>
dropout = p)</p>
<p>def forward(self,x):<br>
self.fc = nn.Linear(self.hidden_size,x.shape[1])<br>
h0 = torch.zeros(self.num_layers, x.shape[0], self.hidden_size)<br>
c0 = torch.zeros(self.num_layers, x.shape[0], self.hidden_size)<br>
output, (hn,cn) = self.lstm(x, (h0,c0))<br>
x = self.fc(hn[self.num_layers-1,:,:])<br>
return x</p>

username: ptrblck
cooked: <p>You could recreate a module in the <code>forward</code> method but would then use this randomly initialized layer which won’t be trained which sounds wrong.<br>
A common approach to allow for variable input shapes would be to use e.g. adaptive pooling layers which are creating a defined shape as their output.</p>

username: NoWay2Guess
cooked: <p>I figure out this problem in a different approach, but still thanks for the reply</p>
Adding an offset to qint8 tensor

username: Sairam954
cooked: <p>Hi,</p>
<p>I want to add certain offset or error value to a quantized tensor qint8, I want each value in quantized tensor to be updated by error times its value + old value. Below code shows this manipulation with normal tensor, can please anyone suggest how similar thing can be achieved for qint8.<br>
<a class="mention" href="/u/hdcharles">@HDCharles</a>  <a class="mention" href="/u/jerryzh168">@jerryzh168</a></p>
<pre><code class="lang-auto">import torch
t = torch.tensor([-1.0, 0.0, 1.0, 2.0])
error = 3
# add an error of 3%
t = t + (0.03)*t

qt = torch.quantize_per_tensor(torch.tensor([-1.0, 0.0, 1.0, 2.0]), 0, 10, torch.quint8)
# How can such error addition be done on quantized tensors
</code></pre>

username: HDCharles
cooked: <p>mathematically, this is the same as multipling the scale by (1+.03) or whatever your error is.</p>
<p>i.e. t.set_q_scale(t.q_scale()*(1.03)) but there’s no way to directly alter q_scale to my knowledge, set_q_scale doesn’t exist.</p>
<p>at worst you can do torch.quantize_per_tensor(t.dequantize(), t.q_scale()*(1+.03), t.q_zero_point(), t.dtype)</p>
LSTM batched input shape

username: NancyShehata
cooked: <p>my input shape of the data is (7, 2, 141) and I need to run the LSTM on Pytorch but I don’t know what should be the sequential length, input size, batch size, can someone please advise?</p>
<p>the code is below</p>
<p>features_test.shape<br>
(7, 2, 141)</p>
<p>def sliding_windows(data, seq_length):<br>
x = []<br>
y = []</p>
<pre><code>for i in range(len(data)-seq_length-1):
    _x = data[i:(i+seq_length)]
    _y = data[i+seq_length]
    x.append(_x)
    y.append(_y)

return np.array(x),np.array(y)
</code></pre>
<p>seq_length = 2<br>
x, y = sliding_windows(features_test, seq_length)<br>
train_size = int(len(y) * 0.80)<br>
test_size = len(y) - train_size<br>
trainX = Variable(torch.Tensor(np.array(x[0:train_size])))<br>
trainY = Variable(torch.Tensor(np.array(y[0:train_size])))</p>
<p>print(trainX.shape)<br>
print(trainY.shape)</p>
<p>testX = Variable(torch.Tensor(np.array(x[train_size:len(x)])))<br>
testY = Variable(torch.Tensor(np.array(y[train_size:len(y)])))<br>
class LSTM(nn.Module):</p>
<pre><code>def __init__(self, num_classes, input_size, hidden_size, num_layers):
    super(LSTM, self).__init__()
    
    self.num_classes = num_classes
    self.num_layers = num_layers
    self.input_size = input_size
    self.hidden_size = hidden_size
    self.seq_length = seq_length
    
    self.lstm = nn.LSTM(input_size=input_size, hidden_size=hidden_size,
                        num_layers=num_layers, batch_first=False)
    
    self.fc = nn.Linear(hidden_size, num_classes)

def forward(self, x):
    x = torch.randn(2, 2, 141)
    h_0 = Variable(torch.zeros(
        self.num_layers, x.size(0), self.hidden_size))
    
    c_0 = Variable(torch.zeros(
        self.num_layers, x.size(0), self.hidden_size))
    
    # Propagate input through LSTM
    ula, (h_out, _) = self.lstm(x, (h_0, c_0))
    
    h_out = h_out.view(-1, self.hidden_size)
    
    out = self.fc(h_out)
    
    return out
</code></pre>
<p>num_epochs = 5000<br>
learning_rate = 0.001</p>
<p>input_size = 141<br>
hidden_size = 100<br>
num_layers = 1<br>
num_classes = 1</p>
<p>lstm = LSTM(num_classes, input_size, hidden_size, num_layers)</p>
<p>criterion = torch.nn.MSELoss()    # mean-squared error for regression<br>
optimizer = torch.optim.Adam(lstm.parameters(), lr=learning_rate)<br>
<span class="hashtag">#optimizer</span> = torch.optim.SGD(lstm.parameters(), lr=learning_rate)</p>
<h1>
<a name="train-the-model-1" class="anchor" href="#train-the-model-1"></a>Train the model</h1>
<p>for epoch in range(num_epochs):<br>
outputs = lstm(trainX)<br>
optimizer.zero_grad()</p>
<pre><code># obtain the loss function
loss = criterion(outputs, trainY)

loss.backward()

optimizer.step()
if epoch % 100 == 0:
    print("Epoch: %d, loss: %1.5f" % (epoch, loss.item()))
</code></pre>
BatchNorm after ConvTranspose?

username: Omroth
cooked: <p>I’m inexperienced in this area.  I’m adding BatchNorms after my Convs in my U-net, should I also add after my ConvTranspose layers?</p>

username: ptrblck
cooked: <p>Run a few experiments with this change and compare the training as well as validation loss to the model without batchnorm layers. Based on these findings, pick the best approach for your use case.</p>
Using from_blob to create a tensor

username: Shobhit_Narayanan
cooked: <p>I am getting the following error using from_blob. I just want to initiate a tensor with fixed values on GPU. How can I do it?</p>
<pre><code class="lang-auto">float _x[4] = {0.5,-1.2,3.4,2.2};
torch::Tensor x = torch::from_blob(_x, {4}).clone().to(torch::kCUDA);

</code></pre>
<p>But I end up receiving the following error</p>
<blockquote>
<p>IndexError: select(): index 4 out of range for tensor of size [4] at dimension 0</p>
</blockquote>
<p>I have also tried using this method to create a tensor on the GPU.</p>
<pre><code class="lang-auto">auto options = torch::TensorOptions().device(torch::kCUDA,0);```</code></pre>

username: ptrblck
cooked: <p>I cannot reproduce the issue and your code runs fine:</p>
<pre><code class="lang-auto">#include &lt;torch/script.h&gt; // One-stop header.

#include &lt;iostream&gt;
#include &lt;memory&gt;

int main(int argc, const char* argv[]) {
  float _x[4] = {0.5,-1.2,3.4,2.2};
  torch::Tensor x = torch::from_blob(_x, {4}).clone().to(torch::kCUDA);

  std::cout &lt;&lt; x &lt;&lt; std::endl;
  return 0;
}
</code></pre>
<p>Output:</p>
<pre><code class="lang-auto"> 0.5000
-1.2000
 3.4000
 2.2000
[ CUDAFloatType{4} ]
</code></pre>
Loss. backward() takes longer and longer

username: xuxuxu0701
cooked: <p>At the begining of the training, the code “loss.backward()” only cost about 40s(I have a big model about optical system), but as the training goes on,  “loss.backward()” 's running time is longer and longer.In my mind, during the training different epoch should have the same time, there must be something wrong, but I can’t find it.So where the problem may occur？</p>

username: ptrblck
cooked: <p>Check if you are extending the computation graph and if the backward call then computes gradients for multiple iterations. Also, how did you narrow down the slowdown is coming from the <code>backward</code> call?</p>
Are optimimizer's Learning rate and RL's Discount factor related [newbie]

username: Kallinteris-Andreas
cooked: <p>They are both represented by the \gamma icon</p>
<p>Should I set the optimizer’s learning rate to RL’s discount factor?</p>
<p>Or is it simply a coincidence, that they are both represented by \gamma character</p>
<p>e. g.<br>
<a href="https://pytorch.org/docs/stable/generated/torch.optim.Adam.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.optim.Adam.html</a></p>
Loss becomes NAN or Inf when using MPS

username: xavi0007
cooked: <p>I’m running some experiments on Pytorch, with very simple settings, and simple I.I.D data of MNIST</p>
<p>criterion: torch.nn.CrossEntropyLoss<br>
optimizer: torch.optim.SGD</p>
<p>on the model:</p>
<pre><code class="lang-auto">class MNIST_Net(nn.Module):
    def __init__(self, name, in_channels, hidden_channels, num_hiddens, num_classes):
        super(MNIST_Net, self).__init__()
        self.name = name
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return F.log_softmax(x, dim = 1)
</code></pre>
<p>and the model update snipper</p>
<pre><code class="lang-auto">for _ in range(self.local_epoch):
            for data, labels in self.dataloader:
                data, labels = data.float().to(self.device, non_blocking=True), labels.long().to(self.device,non_blocking=True)
                
                optimizer.zero_grad()
                outputs = self.model(data)
                loss = eval(self.criterion)()(outputs, labels)
                
                loss.backward()
                torch.nn.utils.clip_grad_norm_(self.model.parameters(), 5)
                optimizer.step() 

                if self.device == "cuda": torch.cuda.empty_cache()        
</code></pre>
<p>The code works properly when I used <code>device = torch.device('cpu')</code><br>
but when I switch it to <code>device = torch.device("mps")</code><br>
My loss would suddenly become NaN or Inf after a few iterations, like in the screenshot.</p>
<p>This problem arises after I upgrade my OS to MacOS Ventura. Already added non blocker: <code>self.model.to(self.device, non_blocking=True)</code></p>
<p>Not sure what’s the issue here.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/3/4322fb94557660b1a5a7df57735ab9ab7aec1940.png" alt="Screenshot 2022-10-31 at 1.02.00 PM" data-base62-sha1="9zUWjXUMZpMQIj69KqwhVhuiZGM" width="630" height="460"></p>

username: spiegelball
cooked: <p>Are you using the latest 1.13 version? Many mps issues have been fixed there.</p>

username: xavi0007
cooked: <p>It fixed when I update it.<br>
Just a caution, for anyone reading this.<br>
<code>conda install pytorch torchvision torchaudio -c pytorch-nightly</code> for some reason still installs the old 1.12 version.</p>
<p>Using <code>pip3 install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu</code> solved it for me.</p>
<p>Thanks!</p>
Why LSTM register_full_backward_hook not working

username: BruceDai003
cooked: <p>I’m trying to dump all intermediate modules’ input and output tensor values both during forward and backward. So I just simply iterate through the named_modules, and <code>register_forward_hook</code> and <code>register_full_backward_hook</code> as usual. But to my surprise, in my model everything works fine except for LSTM module. I only see the forward hook gets called for this LSTM module, the backward hook function never gets called at all for it.<br>
But if I change <code>register_full_backward_hook</code> to <code>register_backward_hook</code>, I can see the hook function gets called. I thought <code>register_full_backward_hook</code> is better than <code>register_backward_hook</code>, since the latter might just dump the intermediate results in a module. But why for LSTM, only the latter works here?<br>
Is there anything wrong with LSTM backward hooks?</p>

username: AlphaBetaGamma96
cooked: <p>Hi <a class="mention" href="/u/brucedai003">@BruceDai003</a>,</p>
<p>Can you share a minimal reproducible script? Also, <code>register_backward_hook</code> gives incorrect (in most cases) results so it’s best not to use it!</p>
Support for one vs all classification in pytorch

username: Saswat
cooked: <p>I have been clinging to a problem with multi-class classification with a large number of classes (500) and a one vs rest classifier seems to be a good approach to handle it. Is there any support for the same in PyTorch itself?</p>
<p>All that I could find is this <a href="https://discuss.pytorch.org/t/one-vs-rest-multi-class-classification-with-fully-connected-network-and-sigmoid-pytorch/85311">post</a> which doesn’t have answers either. I understand that scikit-learn does offer this feature but I wish to stick to pytorch because of the numerous familiar customizations that can be tweaked within this framework. I’ve tried exploring <a href="https://github.com/skorch-dev/skorch" rel="noopener nofollow ugc">skorch</a> but even it’s overly simplified and doesn’t have good documentation to allow modifications which my use case needs</p>

username: Saswat
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> can you share any pointers here? Thanks in advance</p>
Inference time mismatch between JAVA and C++ API

username: vjayd
cooked: <p>Hi PyTorch Community,</p>
<p>I have used pytorch_lite:12.2 for inferencing on a Android Phone i.e., Java version (arm64-v8a)  and found out that it took 700 msec for inferencing whereas, if I use C++ API in Android  through JNI it is taking 3 seconds to infer a single instance. Can somebody help me regarding the same?</p>
How to implement forward pass for a quantized linear?

username: stephen
cooked: <p>Hi, I have a quantized model and I want to extract model’s parameters and implement a forward pass of a quantized linear layer manually. But I don’t know how quantized model do forward pass.<br>
All I get is this code from pytorch source.</p>
<pre><code>def forward(self, x: torch.Tensor) -&gt; torch.Tensor:
    return torch.ops.quantized.linear(
        x, self._packed_params._packed_params, self.scale, self.zero_point)
</code></pre>
<p>But I cannot find where torch.ops.quantized.linear is defined.</p>
<p>Can someone give me a hint on how to implement the forward of a quantized linear layer?</p>

username: suraj.pt
cooked: <p>Hi, this calls the low-level (C++) quantized linear kernel. You can find its implementation in <a href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/quantized/cpu/qlinear.cpp" class="inline-onebox">pytorch/qlinear.cpp at master · pytorch/pytorch · GitHub</a></p>

username: cornman
cooked: <p>hi，I also want to implement forward pass for a quantized linear, but in the model_state_dict only has weight parameters. From the source code, I think that we need parameters named  output_scale and output_zero_point. How can I get this?</p>

username: stephenwang016
cooked: <p>For a single layer, it’s simply layer.scale and layer.zero_point</p>
How to pass input to 3DCNN GRU architecture

username: talhaanwarch
cooked: <p>I want to pass video data to 3DCNN-GRU architecture.<br>
This is shape of data<br>
<code>Batch,Channels,Num_Frames, Height, Width</code><br>
I split the num_frames to windows<br>
Now the shape is<br>
<code>Batch,Channels,Windows,Length, Height, Width</code><br>
After that i reshape the data<br>
<code>Batch,Windows,Channels,Length, Height, Width</code><br>
Now i combine first two axis and pass it to CNN<br>
<code>Batch*Windows,Channels,Length, Height, Width</code><br>
The 3D CNN outputs<br>
<code>Batch*Windows,Features</code><br>
I reshape the data again<br>
<code>Batch,Windows,Features</code><br>
Permute the the shape before passing it to GRU, the new shape is<br>
<code>Batch,Features,Windows</code><br>
the GRU output shape is<br>
<code>Batch,Features,Windows</code><br>
the GRU hidden state shape is<br>
<code>1,Batch,Windows</code><br>
This is GRU layer<br>
torch.nn.GRU(input_size=8,hidden_size=8,batch_first=True)</p>
<p>Can any one help me is this flow is correct or not.</p>
How to vectorize a custom autograd function?

username: pylearn
cooked: <p>My loss function involves evaluations of a custom autograd function.<br>
Currently, I am using for loop to compute the loss for every item in a batch with this custom autograd function, which is slow.</p>
<pre><code class="lang-auto">loss = torch.stack([func(xi, some_data) for xi in x]).mean()
</code></pre>
<p>where <code>func</code> is a custom autograd function, <code>x</code> is a batch of tensors returned from a netwrok.</p>
<p>In principle, this can be vectorized or computed in parallel since each iteration is independent.<br>
I tried to use <code>torch.vmap</code> to vectorized this custom autograd function. However, <code>torch.vmap</code> does not support autograd function at the moment.</p>
<p>(I did not write a vectorized version of autograd function directly since this function involves an iterative method to find the root of a function. For different inputs, the numbers of iterations required are different, which makes the vectorization difficult.)</p>
<p>Is there a way to speed up this for loop by vectorization or parallel computation?</p>
Question about Dealing with pad token when computing loss

username: sentinel8b
cooked: <p>Good morning.</p>
<p>I’m training image captioning model and just wondered if there’s any different between those two code.</p>
<pre><code class="lang-python"># index 0 is for pad token
criterion = nn.CrossEntropyLoss(ignore_index=0)
'computing loss'
loss = criterion(pred, target)
loss.backward()
optimizer.step()
</code></pre>
<pre><code class="lang-python">criterion = nn.CrossEntropyLoss()
'computing loss'
pad_location = torch.ne(target, 0)
loss = criterion(pred, target)
loss *= pad_location
loss.backward()
optimizer.step()
</code></pre>
<p>Thanks</p>
Output of nn.Module

username: timosy
cooked: <p>I put a simple model below</p>
<pre><code class="lang-auto">class NetA(nn.Module):
    def __init__(self):
        super(NetA, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, (5, 5)) 
        self.conv2 = nn.Conv2d(6, 16, (5, 5))
        self.drop1 = nn.Dropout(p=0.5)
        self.fc1 = nn.Linear(16 * 5 * 5, 64) 
        self.fc2 = nn.Linear(64, 3) 
    def forward(self, x):
        x = F.max_pool2d(F.relu(self.conv1(x)), 2) 
        x = self.drop1(x)
        x = F.max_pool2d(F.relu(self.conv2(x)), 2) 
        x = x.view(-1, 16 * 5 * 5) 
        x = F.relu(self.fc1(x))

        if 1: # pattern1
            x = self.fc2(x)
        else: #pattern2
            x = F.softmax(self.fc2(x), dim=1)

        return x
</code></pre>
<p>If I input certain random value to the model, an output is [[0. 0. 0.]].<br>
But, when activating the last line, and inputting certain random value, the output is, for instance, [[0.2243069 0.3878466 0.3878466]].</p>
<p>My question is why “x = F.softmax(self.fc2(x), dim=1)” gives me some values although x = self.fc2(x), the previous function in the model, gives exsactly 0,0,0 ??</p>

username: srishti-git1110
cooked: <p>Hi, can you please specify the exact input size you are working with?<br>
Also, are you sure it’s exactly 0, 0, 0 and not some values that are displaying zero due to limited precision?</p>

username: timosy
cooked: <p>thank you for your comment,</p>
<p>the input is Input [input] shape: (1, 3, 32, 32)<br>
with np.random.random(input_shape)</p>
<p>and output values are<br>
[[0.000000000000000000000000<br>
0.000000000000000000000000<br>
0.000000000000000000000000]]</p>
<p>There was my wrong explanation.<br>
The pattern1 is “x = self.fc2(x); return x”<br>
This gives 0,0,0.</p>
<p>The pattern2 is “x = F.softmax(self.fc2(x), dim=1); return x”<br>
This gives some values although self.fc2(x) gives 0,0,0</p>
How to convert coordinate to image

username: wxystudio
cooked: <p>I have n points as tensor(n, 2), the second dimension is x and y coordinate on an image, I want to convert them to tensor(n, W, H), the W and H are   x and y coordinate on the image.<br>
How can I do it <strong>with no for loop?</strong></p>
<p>thanks</p>

username: ptrblck
cooked: <p>If you want to set a specific value for these coordinates into the image, this code should work:</p>
<pre><code class="lang-python">img = torch.zeros(4, 5, 5)
x = torch.randint(0, 5, (4, 2))
img[torch.arange(img.size(0)), x[:, 0], x[:, 1]] = 1.

print(img)
</code></pre>

username: Weedsinml
cooked: <p>If the points tensor is [B, N, 2], how to convert it to [B, H, W] without a loop?</p>

username: ptrblck
cooked: <p>I’m unsure what your exact use case is so could you describe it in more details, please?</p>

username: Weedsinml
cooked: <p>I have the projected points, [B, N, 2](batch size, points numbers, x/y), and I want to convert them to images [B, H, W]. First, I set images = torch.zeros([B,H,W]). Then, images[points]=1. The points are like an index, for these indexed pixels, values=1, others are 0.</p>

username: ptrblck
cooked: <p>Indexing should work:</p>
<pre><code class="lang-python">B, N = 3, 4
H, W = 5, 5

x = torch.zeros(B, H, W)
idx = torch.randint(0, H, (B, N, 2))

x[torch.arange(B).unsqueeze(1), idx[:, :, 0], idx[:, :, 1]] = 1.
</code></pre>

username: Weedsinml
cooked: <p>Thank you very much!<br>
I tested it. After several iterations, " x[torch.arange(B).unsqueeze(1), idx[:, :, 0], idx[:, :, 1]] = 1." causes an error: “CUDA error: device-side assert triggered”. My environment is Cuda 11.4, torch 1.12.1. And using os.environ[“CUDA_LAUNCH_BLOCKING”] = “1” has the same error log.</p>

username: ptrblck
cooked: <p>I guess you might be running into an indexing error and the stacktrace with blocking launches would also show it. Check the shape of <code>x</code> as well as the values in <code>idx</code> and make sure they are still valid after a few iterations.</p>

username: Weedsinml
cooked: <p>Yes, it’s because the idx values are out of bounds. Thank you!</p>
Erf implementation under the hood

username: SHREYA_SHARMA
cooked: <p>I am trying to find how <a href="https://pytorch.org/docs/stable/special.html#torch.special.erf" rel="noopener nofollow ugc">erf</a> function is implemented in pytorch but cannot find a code pointer. I am assuming being an integral it must have been approximated but there is no references on how so.<br>
I found <a href="https://github.com/pytorch/pytorch/pull/2799/files" rel="noopener nofollow ugc">this commit</a> which contains erf inverse being approximated and then corrected using newton raphson, is there something similar being done for erf or does pytorch use existing implementations from std?</p>

username: ptrblck
cooked: <p>Yes, it seems the standard definition is used <a href="https://github.com/pytorch/pytorch/blob/85ef1a1cd104033143cfa9a3f19fc3ab326d737a/aten/src/ATen/native/cuda/UnarySpecialOpsKernel.cu#L257">here</a>.</p>
Does PyTorch detection models don’t support batching?

username: arunprabhu_vijayan
cooked: <p>Does PyTorch detection models doesn’t support batching ? I was trying to vision detection models</p>

username: ptrblck
cooked: <p>Detection models accept different batch sizes as seen in <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html#testing-forward-method-optional">this tutorial</a>.</p>
Connecting the models

username: mfatih7
cooked: <p>Hello</p>
<p>I want to connect 2 models and run training.</p>
<p>My first model A generates one-dimensional output.<br>
While batches are being processed, in the training loop, I want to collect 1024 of these and give them to Model B as input.</p>
<p>What is the best way to connect Model A and Model B and backpropagate in a suitable fashion?</p>
Two networks with one loss. How is it possible?

username: S_M
cooked: <p>Hi all, I have two distinct network with same loss and two optimizers for each network. How can I backprop the loss to the networks?<br>
Any help would be appreciated.</p>
<pre><code>        optimizer_A.zero_grad()
        optimizer_B.zero_grad()
        loss.backward()
        optimizer_A.step()
        optimizer_B.step()
</code></pre>

username: srishti-git1110
cooked: <p>Hi,<br>
This probably means the common loss is a function of the parameters of both the models.</p>
<p>You could then just create a single optimizer containing the parameters of both the models, like so:</p>
<pre><code class="lang-auto">optimizer = torch.optim.Adam([modelA.parameters()] + [modelB.parameteres()])
</code></pre>
<p>Let me know if this gives any errors.</p>

username: S_M
cooked: <p>Hi <a class="mention" href="/u/srishti-git1110">@srishti-git1110</a>, thanks for your tip, but my optimizers are two distinct optimizers with different hyperparameters.</p>

username: srishti-git1110
cooked: <aside class="quote no-group" data-username="S_M" data-post="1" data-topic="164736">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/s_m/48/19733_2.png" class="avatar"> S_M:</div>
<blockquote>
<pre><code class="lang-auto">optimizer_A.zero_grad()
        optimizer_B.zero_grad()
        loss.backward()
        optimizer_A.step()
        optimizer_B.step()
</code></pre>
</blockquote>
</aside>
<p>In that case, is this code not working?</p>
<p>There’s also an option to create parameter groups within the same optimizer but I think that is equivalent to creating two different optimizers with different learning rates etc. (even for optimization algorithms like Adam that calculate running stats.)</p>

username: S_M
cooked: <p>Is it true? I mean what is the acceptable code for this problem</p>

username: srishti-git1110
cooked: <p>Your code should work fine. Please post if you face any errors.</p>
How to read a batched output tensor?

username: arunprabhu_vijayan
cooked: <p>Hi,</p>
<p>I’m using c++, my input is an image tensor batched(b,c,h,w). Can you please tell me the shape of the output tensor?</p>

username: srishti-git1110
cooked: <p>Could you please specify the task and maybe the loss function etc.?</p>
Parallel GRUCell computation

username: Ayush_Chakravarthy
cooked: <p>Say I have a tensor A:  [B, n, 64, 64]. Now I want to have n GRUCell units process this tensor across dim=1.</p>
<p>That is GRUCell_0 gets A[:, 0] and so on and so forth.</p>
<p>Is there a parallel way to achieve this behavior instead of a for loop?</p>
<p>Thanks!</p>
What is the default sharing pattern? Temporal or spatial?

username: Tianyu9748
cooked: <p>If I run this command</p>
<pre><code class="lang-auto">python model1.py &amp; python model2.py
</code></pre>
<p>How these two models are running on the device?<br>
Temporal sharing, model1 runs, then model2 runs, and repeat until the end?<br>
Or spatial sharing, simultaneous kernels?</p>
<p>Best<br>
Max</p>
Gradient descent with nested functions

username: espresso
cooked: <p>Edit: I spent a few minutes replacing the LaTeX with images but it was rejected because I am a new user. Sorry if it is difficult to read.</p>
<p>This is a code question where I am curious how to expand a simple example I am using to a more complex one using the autograd system in PyTorch. I am able to perform gradient descent with the following simple function I have written:</p>
<pre><code class="lang-auto">import torch

def gradient_descent(alpha, iterations, w, nll):
    print(f"Initializing at {w}.")
    
    w = torch.tensor(w, requires_grad=True)

    weights = [w.tolist()]
    costs = [nll(w).tolist()[0]]
    
    for i in range(iterations):
        nll_w = nll(w)
        nll_w.backward()

        with torch.no_grad():
            w -= (alpha * w.grad)
            w.grad.zero_()
        
        weights.append(w.tolist())
        costs.append(nll(w).tolist()[0])
        
    return weights, costs
</code></pre>
<p>You just define a cost function (in this case, negative log-likelihood) with an unknown quantity you wish to estimate and with the appropriate learning rate you can find a weight estimate given some sample data.</p>
<p>I am trying to do something similar for a cost function with nested variable and I am not sure how to do it. I have the following cost function (fictitious):</p>
<p>$C(\theta, a, b) = \ln(5) - a^2 - \sin(5) - b^3$</p>
<p>where $a = \frac{\theta^2-5}{3}$ and $b = \frac{\theta^3-6}{7}$. Thus my cost function actually has variables I wish to find minimum values for that themselves can be expanded into a fraction.</p>
<p>I want to find perform gradient descent where autograd determines the partial derivatives for $\frac{\partial C}{\partial \theta}$, $\frac{\partial C}{\partial a}$, and $\frac{\partial C}{\partial b}$ and performs gradient descent to determine the minima.</p>
<p>The issue is that if I define my cost function in terms of the three variables as shown above, then the cost function has no knowledge that $a$ and $b$ are actually fractions that contain $\theta$. But if I expand out my cost function, replacing $a$ and $b$ with their actual values, then the cost function is no longer using the inputs $a$ and $b$ anywhere and the gradients cannot be calculated.</p>
<p>So my question is about how to modify my code to achieve this.</p>
Output shape issue

username: Hemanth_Yernagula
cooked: <p>I’m trying to make a simple model where given a number it gives output with +2 number<br>
Ex1:- input - 5, output - 7<br>
Ex2:- input - 1, output - 3 so on…</p>
<ol>
<li>The output features of the model is clearly given 1, still I’m getting output with shape of 1X2</li>
<li>I could not use the batch normalization here, can anyone help me in understanding the issue here.</li>
</ol>
<pre><code class="lang-auto">class NN(nn.Module):
    def __init__(self):
        super(NN, self).__init__()
        
        self.ff = nn.Linear(in_features=1*BATCH_SIZE, out_features=15,)
        self.batch_norm = nn.BatchNorm1d(15 )
        self.ff1 = nn.Linear(in_features=15, out_features=1)
        
    def forward(self, x):
        print(f"x : {x}")
        print(f"x shape : {x.shape}")
#         x = x.view(-1, 1)
        ff = self.ff(x)
        
        print(f"ff value : {ff}")
        print(f"ff : {ff.shape}")
        
#         ff = ff.view(1, -1)
        
#         ff = ff.squeeze()
        
        print(f"ff view shape : {ff.shape}")
        ff = self.batch_norm(ff)
        
        out = self.ff1(ff)
        
#         out = ff
        
        print(f"returning output as : {out}, {out.shape}")
        out = out.squeeze()
        
        print(f"returning output as : {out}, {out.shape}")
        return out
        
</code></pre>
<p>output:</p>
<pre><code class="lang-auto">returning output as : tensor([[-0.4479],
        [ 0.4698]], device='cuda:0', grad_fn=&lt;AddmmBackward0&gt;), torch.Size([2, 1])
returning output as : tensor([-0.4479,  0.4698], device='cuda:0', grad_fn=&lt;SqueezeBackward0&gt;), torch.Size([2])
----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Linear-1                   [-1, 15]             975
       BatchNorm1d-2                   [-1, 15]              30
            Linear-3                    [-1, 1]              16
================================================================
Total params: 1,021
Trainable params: 1,021
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.00
Params size (MB): 0.00
Estimated Total Size (MB): 0.00
----------------------------------------------------------------
</code></pre>
<p>output while training:</p>
<pre><code class="lang-auto">ValueError: Expected more than 1 value per channel when training, got input size torch. Size([1, 15])

</code></pre>

username: nassim
cooked: <p>Hello,<br>
When you defined your first layers in the in_features you must just pass the input features of just one example (in this case in_features=1 if i understand correctly), PyTorch will take care of the batch dimension.</p>
Modifying Mask R_CNN Architecture

username: papoo13
cooked: <p>I am working on a research project and as part of it, I need to modify the Mask R_CNN architecture to have multiple inputs and outputs. I want to choose a base code for this, but various repos are available, including detecron2 and Facebook Mask_R_CNN. I found detecron2 complicated to start with. Do you suggest using PyTorch implementation of Mask R CNN?</p>

username: KFrank
cooked: <p>Hi Samin!</p>
<aside class="quote no-group" data-username="papoo13" data-post="1" data-topic="164629" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/f19dbf/48.png" class="avatar"> papoo13:</div>
<blockquote>
<p>Do you suggest using PyTorch implementation of Mask R CNN?</p>
</blockquote>
</aside>
<p>I have used pytorch’s Mask R-CNN.  It works and seems sensible and<br>
solid, but I don’t have any useful comparisons with other implementations.</p>
<p>One advantage of using the pytorch version is that you might be a little<br>
more likely to get answers about it on this forum, rather than about other<br>
versions.</p>
<p>Best.</p>
<p>K. Frank</p>
Torch::from_blob using different memory address

username: perf
cooked: <p>Hi,</p>
<p><a href="https://pytorch.org/cppdocs/api/function_namespacetorch_1ad7fb2a7759ef8c9443b489ddde494787.html" rel="noopener nofollow ugc">torch::from_blob</a> takes a pointer to the data that you want to turn into tensor.</p>
<p>However, is there a way to collect multiple pointers at different location in the memory to create one contiguous tensor? (in another word, how can I combine multiple arrays in a single call?)<br>
Thanks!</p>