akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How to store and load custom PyTorch 'Dataset'?

rishikeshF: <p>I have a dataframe with only one column named ‘address’. It consists of strings of addresses of different places. I am going to feed this data as input to RoBERTa for pretraining on mask language modelling task. So, I am trying to convert the dataset into PyTorch’s Dataset object. It is as follows :</p>
<pre><code class="lang-auto">from torch.utils.data.dataset import Dataset
class CustomDataset(Dataset):
    def __init__(self, df, tokenizer):
        # or use the RobertaTokenizer from `transformers` directly.

        self.examples = []
        
        for example in df.values:
            x=tokenizer.encode_plus(example, max_length = MAX_LEN, truncation=True, padding=True)
            self.examples += [x.input_ids]

    def __len__(self):
        return len(self.examples)

    def __getitem__(self, i):
        # We’ll pad at the batch level.
        return torch.tensor(self.examples[i])

# Create the train and evaluation dataset
train_dataset = CustomDataset(train['address'], tokenizer)
eval_dataset = CustomDataset(test['address'], tokenizer)
</code></pre>
<p>Everything works fine upto this. Now, I want to store the ‘train_dataset’ and ‘eval_dataset’ so that I can use them later in GPU instance. I tried to do it using torch.save as follows :</p>
<pre><code class="lang-auto">torch.save(train_dataset, './train.pt')
torch.save(eval_dataset, './test.pt')
</code></pre>
<p>This too works fine. But, when I try to load this saved dataset into new variable in another instance, it throws attribute error. The code I used for loading this dataset is as follows :</p>
<pre><code class="lang-auto">train_dataset = torch.load('./train.pt')
</code></pre>
<p>It throws error as follows :</p>
<blockquote>
<p>AttributeError: Can’t get attribute ‘CustomDataset’ on '<strong>main</strong> '&gt;<br>
I think, this error is because of custom dataset that I created. I want to save those datasets to a file so that I can use them in another instance with GPU. Could someone help with the code required to load this data from a ‘.pt’ file ?</p>
</blockquote>

Matias_Vasquez: <p>What the error is telling you is that it does not know what to do with <code>CustomDataset</code>.</p>
<p>I think if you import this code into your main file it might work.</p>
<aside class="quote no-group quote-modified" data-username="rishikeshF" data-post="1" data-topic="150828">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/r/ecc23a/48.png" class="avatar"> rishikeshF:</div>
<blockquote>
<pre><code class="lang-auto">class CustomDataset(Dataset):
    def __init__(self, df, tokenizer):
        # or use the RobertaTokenizer from `transformers` directly.

       ... 
</code></pre>
</blockquote>
</aside>
<p>So it would look something like</p>
<pre data-code-wrap="python"><code class="lang-python">from YOUR_FILE_LOCATION import CustomDataset
</code></pre>
<p>And then you can load it.</p>
"TypeError: Invalid dimensions for image data" when plotting a channel in a 3D tensor image

danielmanu93: <p>Hi All, I am trying to plot a channel of a 3D tensor image from my test data. I used the Dataloader function to load my images so images are tensors. The images are in the shape <code>torch.Size([1, 10, 60200])</code> but I have reshaped them into have reshaped the images <code>torch.Size([1, 10, 200, 301])</code> where <code>test batch size=1, channels=10, height=200, width=100</code>. I want to plot one channel of this 3D image so I reshape to torch.Size([200, 301, 10]) by moving the channels to the last dimension. However when I run I get <code>TypeError: Invalid dimensions for image data</code>. Any suggestions to resolve this issue and plot a channel of my 3D image are welcomed. My plot function and data loading code are below:</p>
<pre><code class="lang-auto">test = data_utils.TensorDataset(torch.from_numpy(test_set),torch.from_numpy(label_set))
test_loader = data_utils.DataLoader(test,batch_size=TestBatchSize,shuffle=True)
for i in range(TestBatchSize): 
    image, label = iter(test_loader).next()  #shape of images = [1, 10, 60200]
    image = image.view(TestBatchSize,Inchannels,data_dsp_dim[0],data_dsp_dim[1]) #reshaped to [1,10,200,301]

    for k in range(TestBatchSize):
        seis = images.view(data_dsp_dim[0], data_dsp_dim[1], Inchannels)  #reshaped to [200, 301, 10] --&gt; moved channel to the last dimension

### I want to plot just one channel out of the 10 channels in the data####

#Below is my plot function
def display_seismic(seis, font2, font3, SavePath):
        fig, ax = plt.subplots(figsize=(4, 4))
        seis = ax.imshow(seis, cmap='gray', aspect='auto', vmin=0.06, vmax=0.06)
        divider = make_axes_locatable(ax)
        cax    = divider.append_axes("right",size="5%",pad=0.05)
        plt.colorbar(seis,ax=ax,cax=cax).set_label('Amplitude (m/s)')
        plt.tick_params(labelsize=6)
        for label in  ax.get_xticklabels()+ax.get_yticklabels():
            label.set_fontsize(6)
        plt.savefig(SavePath+'Seismic',transparent=True)
        plt.close()
</code></pre>

Matias_Vasquez: <aside class="quote no-group" data-username="danielmanu93" data-post="1" data-topic="150818">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/a88e4f/48.png" class="avatar"> danielmanu93:</div>
<blockquote>
<pre><code class="lang-auto">for k in range(TestBatchSize):
        seis = images.view(data_dsp_dim[0], data_dsp_dim[1], Inchannels)  #reshaped to [200, 301, 10] --&gt; moved channel to the last dimension
</code></pre>
</blockquote>
</aside>
<p>Here you are not doing what you say you are doing. For this, you should use either <code>permute</code>, or even better something like <code>ToPILImage</code> to take care of this for you.</p>
<p>As you said, you only want to plot ONE channel, so you need to select it. Right now it looks as if you are trying to plot all 10 channels and pyplot does not know what to do with 10 channels.</p>
Placing modified regions of interest in the orginal location

subhashnerella: <p>Lets say i have an CNN intermediate layer output tensor call it X with shape (B,C,H,W) batch, channels, height and width. I extract the regions of interest (ROIs) from the tensor based on some manually chosen criteria. Assume all the ROIs have same shape (B,N,C,h,w). N is number of ROIS, h is height, and w is width of ROI respectively. Lets call the ROI tensor Y. Now i perform an operation on Y (assume convolution), this operation does not alter the dimension or shape of the ROIs. Lets call the modified ROI tensor as Y’(shape: B,N,C,h,w).</p>
<p>Now i want to replace the locations where Y are extracted from X with Y’. This modified X is further processed in the subsequent layers of the model. So essentially if i do the following things<br>
Y = X[location criteria]<br>
Y’ = some_operation(Y)<br>
X[location criteria] = Y’</p>
<p>Will i be able to train the model without any issues if i do the above?</p>

Andrei_Cristea: <p>Loosely speaking,. I think:</p>
<ul>
<li>the <code>location_criteria</code> indexer is not trainable – but you said that’s “manual” so you probably expected this</li>
<li>if <code>some_operation</code> is some differentiable function that torch knows how to compute gradients for, the model should train all the way through</li>
<li>if <code>some_operation</code> isn’t differentiable, then Y will not contribute to the training of upstream parameters, so training won’t work as you may expect it to</li>
</ul>
How can i enable GPU in those lines?

Sammo1: <p>i need to enable fast computations using single GPU.</p>
<pre><code>def load_points_cacl_distance(self, path):
    points = pd.read_csv(path, sep='\t', usecols=[0, 1])      
    max_id = len(points)
    device = torch.device("cuda")
    d = pd.DataFrame(np.zeros((max_id, max_id)))  
    tensor1 = torch.from_numpy(d.values)
    dis = sch.distance.pdist(points, 'euclidean') 
    tensor2 = torch.from_numpy(dis)
    print(dis)
    n = 0
    for i in range(max_id):
        print(i)
        for j in range(i + 1, max_id):    
            d.at[i, j] = tensor2[n].to(device)
            d.at[j, i] = tensor1[i, j].to(device)
            n += 1
</code></pre>
<p>but the output is still slow</p>

Andre_Amaral_IST: <p>Hey,</p>
<p>In the following situations you should put .cuda() our .to(device).<br>
Situation 1: tensors creation<br>
Situation 2: model definition (in case of neural networks models)</p>
<p>Since you define a device with <code>device = torch.device("cuda")</code> you should put tensor.to(device) in all the tensors you create.</p>
RuntimeError: expected scalar type Float but found Int? Why does torch.matmul support only float32?

Sairam954: <p>Hi,</p>
<p>I was wondering about the reason behind torch.matmul being only supported for float in PyTorch? I would really appreciate if someone can describe the best work around for it without losing performance.</p>
<p>Thanks</p>

eqy: <p>This is a known upstream issue: <a href="https://github.com/pytorch/pytorch/issues/29961" rel="noopener nofollow ugc">Add support for integer matrix multiplication (particularly for dtype = torch.int8 ) · Issue #29961 · pytorch/pytorch (github.com)</a></p>
<p>As for a workaround if you know the dynamic range of your integers you can do exact integer accumulation in float16 from to <code>-2**11</code> to <code>2**11</code> and float32 from <code>-2**24</code> to <code>2**24</code>.</p>
ONNX export warning: Constant folding not applied on opset 12

alwynmathew: <p>While converting pytorch model to onnx</p>
<pre><code class="lang-auto">torch.onnx.export(model, 
                  dummy_input,  
                  save_path,  
                  operator_export_type=torch.onnx.OperatorExportTypes.ONNX, 
                  export_params=True, 
                  opset_version=12, 
                  verbose=False)
</code></pre>
<p>I get multiple lines of warning as below</p>
<pre><code class="lang-auto">Warning: Constant folding - Only steps=1 can be constant folded for opset &gt;= 10 onnx::Slice op. Constant folding not applied.
Warning: Constant folding - Only steps=1 can be constant folded for opset &gt;= 10 onnx::Slice op. Constant folding not applied.
.
.
</code></pre>
<h3>
<a name="system-information-1" class="anchor" href="#system-information-1"></a>System information</h3>
<ul>
<li>OS Platform and Distribution: Ubuntu 20.04</li>
<li>Python version: 3.7.11</li>
<li>Pytorch version: 1.10.2</li>
</ul>
<p>How can I remove this warning? Does this affect prediction values?</p>

alwynmathew: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> could you please comment on this question. Thank you.</p>
How to use torch.nn.Unfold in order to get a right output tensor size

ZSheikhb: <p>I am trying to adapt <a href="https://github.com/moothes/SALOD/blob/1bd0c1d586ace7b8cfe344058fe8aef60a8c753c/methods/picanet/model.py" rel="noopener nofollow ugc">this code</a> to the Camvid dataset.</p>
<pre><code class="lang-auto">class PicanetG(nn.Module):
    def __init__(self, size, in_channel):
        super(PicanetG, self).__init__()
        self.renet = Renet(size, in_channel, 100)
        self.in_channel = in_channel

    def forward(self, *input):
        x = input[0]
        size = x.size()
        kernel = self.renet(x)
        kernel = F.softmax(kernel, 1)
        print(x.size(), kernel.size())
        kernel = kernel.reshape(size[0], 100, -1)  # [20, 100, 4, 4]
        x = F.unfold(x, kernel_size=(5, 5), padding=[1,1], dilation=[2, 2],stride=5)  # (x, [10, 10], dilation=[3, 3])
        print(x.size(), kernel.size())
        x = x.reshape(size[0], size[1], 10 * 10)
        x = torch.matmul(x, kernel)
        x = x.reshape(size[0], size[1], size[2], size[3])
        return x
</code></pre>
<p>The error message from this class is</p>
<pre><code class="lang-auto">torch.Size([20, 1024, 18, 24]) torch.Size([20, 100, 4, 4])
torch.Size([20, 25600, 6]) torch.Size([20, 100, 16])
  0%|                                                                                                                                         | 0/19 [00:01&lt;?, ?it/s]
Traceback (most recent call last):
  File "train.py", line 81, in &lt;module&gt;
    pred, loss = model(img, mask)
  File "/home/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/VAE/model.py", line 46, in forward
    dec, _pred = self.decoder[i](en_out[5 - i], dec)
  File "/home/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/VAE/model.py", line 146, in forward
    fmap_att = self.picanet(fmap)  # F_att
  File "/home/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/VAE/model.py", line 176, in forward
    x = x.reshape(size[0], size[1], 10 * 10)
RuntimeError: shape '[20, 1024, 100]' is invalid for input of size 3072000
</code></pre>
<p>I have problem to change different variables of <code>torch.nn.Unfold</code> in order to get exactly the correct output size. I will appreciate if someone can help.</p>

Sairam954: <p>What parameters are you willing to change in unfold and can you also specify the output shape you require?<br>
Here is a video which explains the unfold operation very nicely [<a href="https://www.youtube.com/watch?v=zVDDITt4XEA" class="inline-onebox" rel="noopener nofollow ugc">PyTorch - Convolution under the hood (Unfolding/Folding) - YouTube</a>]</p>
Can not display image using plot in spider

PAVANI_KATAKAM: <p>import os</p>
<p>import numpy as np</p>
<p>import <a href="http://skimage.io" rel="noopener nofollow ugc">skimage.io</a> as skio</p>
<p>vol=skio.imread(‘file:///C:/Users/paka874c/Desktop/diatom_late_1_040nm_recApp1_2.labels.tif’)<br>
print(vol.shape)  # vol size (1753,584,1780)</p>
<p>subvol=vol[301:800,300:800,300:1200]</p>
<p>subvol=vol[300:800,300:600,300:1200]</p>
<p>subvol=vol[300:800,300:550,300:1200]</p>
<p>import matplotlib.pyplot as plt</p>
<p>plt.imshow(subvol[250])</p>
<p>error:Invalid shape (500, 250, 900) for image data and no image on plots its empty</p>
<p>can anybody solve my problem please ? I cant find any solution</p>

Matias_Vasquez: <p><a href="https://matplotlib.org/3.5.0/api/_as_gen/matplotlib.pyplot.imshow.html" rel="noopener nofollow ugc">Here</a> is the documentation for <code>pyplot.imshow</code>. As you can see there, the expected input image has to have a certain shape.</p>
<pre><code class="lang-auto">The image data. Supported array shapes are:

* (M, N): an image with scalar data. The values are mapped to colors using normalization and a colormap. See parameters *norm*, *cmap*, *vmin*, *vmax*.
* (M, N, 3): an image with RGB values (0-1 float or 0-255 int).
* (M, N, 4): an image with RGBA values (0-1 float or 0-255 int), i.e. including transparency.
</code></pre>
<p>However, the “image” that you want to plot has the shape <code>500 x 250 x 900</code>. This means that you have 900 in the third dimension, but only 3 or 4 are supported.</p>
<p>Maybe you can take a look at how the images are stored in order to determine where is one valid image, select it and then you can plot it.</p>
Reference to 'optional' is ambiguous

Lotte: <p>I want to use Pytorch in a project using openCV. I get similar issues as described in <a href="https://discuss.pytorch.org/t/linker-failed-with-ambiguous-references/36255/5">this topic</a>.<br>
![image|614x293]. I do need the namespace std, so I can not use the trick they used to solve the problem. I also get some addtional errors in from_blob.h <em>uninitialized reference member in ‘class at::TensorMaker&amp;’ [-fpermissive]</em> and <em>expected ‘)’ before ‘&lt;’ token</em> . These are all the errors I get when building:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/3/a/3a5f64f01aea75072176ac1b7e172040f0c2f88c.png" alt="image" data-base62-sha1="8ko4CMWmjOrBA4GM9F0xT8gFotK" width="617" height="295"></p>

ptrblck: <p>I would try to solve the previous errors, especially the apparently missing <code>;</code> in line 29, as the downstream failures might be caused by the initial error.</p>

Lotte: <p>I have not changed anything in the file from_blob.h and line 29 does not seem to need a ;<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/d/1dd2547c65708b1bd35a5678ab776d9a1efa34cd.png" data-download-href="https://discuss.pytorch.org/uploads/default/1dd2547c65708b1bd35a5678ab776d9a1efa34cd" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/d/1dd2547c65708b1bd35a5678ab776d9a1efa34cd_2_556x500.png" alt="image" data-base62-sha1="4fOtycuzt1VXuK47WgGyDJN3QbX" width="556" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/d/1dd2547c65708b1bd35a5678ab776d9a1efa34cd_2_556x500.png, https://discuss.pytorch.org/uploads/default/original/3X/1/d/1dd2547c65708b1bd35a5678ab776d9a1efa34cd.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/d/1dd2547c65708b1bd35a5678ab776d9a1efa34cd.png 2x" data-dominant-color="F7F9F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">793×713 65.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Decoration or redefining torch.mm(...) to be used inside torch

Ali_Nezhadi: <p>Hi, I was curious, Is it possible to define a custom function like</p>
<pre><code class="lang-auto">def myMM(...):
  return something
</code></pre>
<p>and assign its pointer to <code>torch.mm()</code> or some of other implicitly defined functions in <code>torch</code>(like <code>torch.dot</code>,<code> torch.matmul</code> or …), then instead of defining new Function for <code>nn.linear</code> or …; make <code>nn.linear</code> to <strong>perform matrix multiplication</strong> using <code>myMM</code> function inside its source code?</p>
<p>I just want to give a new definition to <strong>dot product</strong> in pytorch but I don’t want to redefine all <code>nn</code> modules like (conv, linear, lstm, …) as a customized function for my simulations.</p>
<p>I would be appreciated it if you could help me<br>
TNX</p>

ptrblck: <p>Maybe the <a href="https://pytorch.org/docs/stable/notes/extending.html#extending-torch">Extending torch docs</a> might be useful as they describe how to create a custom tensor class and make it work via overriding built-in methods.</p>

Ali_Nezhadi: <p>Thank you for your answer, I have already studied them. But they are not what I am looking for.</p>
<p>let me be precise, how can I edit <strong>torch.mm()</strong> or <strong>torch.matmul()</strong> source code inside pytorch? what files should I look into that? I already looked at ATen source file for cuda,… also torch directory in github  but I couldn’t figure out where is the definition of torch.mm() to edit</p>
Modifying the gradients after backward pass in DDP

anshul.singh: <p>Hello,<br>
I’m trying to implement an optimizer algorithm that involves modifications in the gradients after the backward pass. When I replace the old gradients with the modified ones after the backward pass, Is it correct to assume that the values will get updated on the all the processes running across all the GPUs under DDP or will the updates take place only for one process?</p>

cbalioglu: <p>Hey <a class="mention" href="/u/anshul.singh">@anshul.singh</a>, it depends on what you mean by “backward pass”? If you mean once all layers of your model have gone through the backprop, then your optimizer should be run on all ranks since gradients are gradually synced among ranks during backprop. Having said that I would suggest checking out our DDP communication hooks, which might be a better fit for your use case:<br>
<a href="https://pytorch.org/docs/stable/ddp_comm_hooks.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/ddp_comm_hooks.html</a></p>

anshul.singh: <p>Thank you <a class="mention" href="/u/cbalioglu">@cbalioglu</a> for your response. I’m still in need of some more clarification, I’ll rephrase my question to convey it better.</p>
<p>I’m running the DDP training on 3GPUs, with a batch size of 8. What I understand is that when I do a forward pass and calculate loss, each of the three process on GPUs will have their own losses. And when I do backward pass (all layers), the gradients on each processes are calculated using their corresponding losses but they all get synced by the end of the backward pass.</p>
<p>Now all GPUs have the same gradients. I make a copy of the gradients and make some modifications and replace the original values with the modified ones. Will this change only happen on one GPU or on all the GPUs?</p>
Is there a way to visualize the network structure with TensorBoard like the graph in TorchStudio?

songyuc: <p>Hi, guys,<br>
I am wondering whether there is a way to visualize the network structure with TensorBoard like the graph in TorchStudio.<br>
And here is a screenshot of the graph from <a href="https://www.torchstudio.ai/getstarted/" rel="noopener nofollow ugc">TorchStudio</a>,<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/1/2155a0f00c1ac795c0d82e175136bf878e8e62fe.png" data-download-href="https://discuss.pytorch.org/uploads/default/2155a0f00c1ac795c0d82e175136bf878e8e62fe" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/1/2155a0f00c1ac795c0d82e175136bf878e8e62fe_2_442x500.png" alt="image" data-base62-sha1="4KTdkNGQQlDBlmmvmoRkWKOuake" width="442" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/1/2155a0f00c1ac795c0d82e175136bf878e8e62fe_2_442x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/1/2155a0f00c1ac795c0d82e175136bf878e8e62fe_2_663x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/1/2155a0f00c1ac795c0d82e175136bf878e8e62fe_2_884x1000.png 2x" data-dominant-color="222220"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1368×1545 142 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Your answer and guide will be appreciated!</p>
Call ExponentialLR schedule per step or per epoch?

caesar025: <p>What is considered best practice when using an ExponentialLR schedule? Should the learning rate be updated every epoch (as the <a href="https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.ExponentialLR.html" rel="noopener nofollow ugc">documentation</a> suggests) or every optimization step?</p>
How to know the parameter for an effect in sox_effects?

ADONAI_TZEVAOT: <p><code>torchaudio.sox_effects.effect_names()</code> gives me the list of all the sox effects in torchaudio. I want to use these effects on my audio files but I don’t know what parameters some of these effects need. For example the effect <code>pitch</code> can be applied by using an integer number that specifies the pitch whereas lowpass requires 2 numbers, whereas reverb needs a character value …as follows:</p>
<pre><code class="lang-auto">waveform1, sample_rate1 = torchaudio.load(WAV_PATH)

effects = [
           ['pitch', '1.1'],
  ["lowpass", "-1", "300"], # apply single-pole lowpass filter
  ["speed", "0.8"],  # reduce the speed
                     # This only changes sample rate, so it is necessary to
                     # add `rate` effect with original sample rate after this.
  ["rate", f"{sample_rate1}"],
  ["reverb", "-w"],  # Reverbration gives some dramatic feeling
]



waveform2, sample_rate2 = torchaudio.sox_effects.apply_effects_tensor(
    waveform1, sample_rate1, effects)

</code></pre>
<p>How do I know what values to specify for other sox effects such as 'allpass, chorus, divide, …" Is there any documentation available. Please help</p>

ptrblck: <p>I’m not entirely sure, but would assume that the effect parameters are the same as when calling <code>sox</code> directly and are defined in <a href="http://sox.sourceforge.net/sox.html">their docs</a>.</p>
Sum at each position on boolean mask without for loop

Fst_Msn: <p>Hi there,</p>
<p>This is my first question, I hope that I’m in the right category!</p>
<p>Briefly, I have a 2d boolean mask, and I want to obtain the sum at each position.</p>
<p>Example:</p>
<blockquote>
<p>mask = torch.Tensor([[1, 0, 0, 1, 1, 0, 1],[0, 1, 0, 1, 0, 1, 0]])</p>
</blockquote>
<p>A normal sum on axis 1 will be:</p>
<blockquote>
<p>torch.sum(mask, dim=1)<br>
tensor([4., 3.])</p>
</blockquote>
<p>But I  would like to obtain the sum at each position, where the result has the same shape as mask:</p>
<blockquote>
<p>sum_pos(mask, dim=1)<br>
tensor([[1, 1, 1, 2, 3, 3, 4],[0, 1, 1, 2, 2, 3, 3]])</p>
</blockquote>
<p>Is there any way to do that, <strong>without a for loop on dim=1</strong> ?</p>
<p>Thanks in advance!</p>

InnovArul: <aside class="quote no-group" data-username="Fst_Msn" data-post="1" data-topic="150745">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fst_msn/48/31467_2.png" class="avatar"> Fst_Msn:</div>
<blockquote>
<p>mask = torch.Tensor([[1, 0, 0, 1, 1, 0, 1],[0, 1, 0, 1, 0, 1, 0]])</p>
</blockquote>
</aside>
<p>You could use <code>tensor.cumsum()</code> to achieve this.</p>
<pre><code class="lang-auto">mask = torch.Tensor([[1, 0, 0, 1, 1, 0, 1],[0, 1, 0, 1, 0, 1, 0]])
mask.cumsum(dim=1)
</code></pre>
Biased logistic regression

John_Grace: <p>I am building a binary classifier in python.<br>
My model has decently high AUC=90%, but is biased, underestimating the probability $y=1$. This is systematic across some of the input features as well. How can I nudge the bias term, or otherwise address this issue? I am surprised that the model ends up being biased, despite it having an intercept (bias term).<br>
Perhaps there’s a way to tell it after training “now scale up the bias term so as to match the sum of predicted probabilities (y_hat) with the sum of the actual (real) ys”<br>
Or how would I address a bias issue like this?<br>
My dataset is very unbalanced, only 3% positives (1s) vs 97% 0s. But in y_hat, the number of 1s is closer to 2.5%.</p>

eqy: <p>Are you weighting the loss function in some way? e.g., via the weight parameter? <a href="https://pytorch.org/docs/stable/generated/torch.nn.BCELoss.html" class="inline-onebox" rel="noopener nofollow ugc">BCELoss — PyTorch 1.11.0 documentation</a></p>
Stop inference on GPU

tominator: <p>Is there a way to stop the process of the inference on the GPU?</p>
<p>Description: I load a object detection model onto the GPU. After that I want to run the inferences on the GPU. Is there a possibility to stop the inference process on the GPU (for example after a specified time)? But the model should remain on the GPU and a reload of the model onto the GPU should be prevented.</p>

marksaroufim: <p>CUDA is asynchronous so I’m not sure if there’s a way to stop an execution from the host.</p>
<p>Could you elaborate on your use case, are you trying to vary batch size based on throughput while a model is running? I generally profile offline and then keep the batch size fixed to a performance threshold I know.</p>
Registering a new 'NPU' device type to PyTorch

lichao0613: <p>Recently, we added a merge request about our ‘NPU’ device type registering to PyTorch(<a href="https://github.com/pytorch/pytorch/pull/75863" class="inline-onebox" rel="noopener nofollow ugc">Add new keys for Ascend NPU by lichao0613 · Pull Request #75863 · pytorch/pytorch · GitHub</a>).<br>
NPU is device abstraction of Ascend Neural Networks Process Units, which works as a normal torch device like CPU, GPU and XPU.<br>
Registering of ‘NPU’ is very important for us to enable our extension(<a href="https://gitee.com/ascend/pytorch" rel="noopener nofollow ugc">https://gitee.com/ascend/pytorch</a>) , it’s glad to see our PR can be reviewed as soon as possible.  Any questions or suggestions will be answered seriously and promptly.</p>
Can I collectivize models learning?

isdriai: <p>Hi,</p>
<p><a href="https://discuss.pytorch.org/t/giving-multiple-parameters-in-optimizer/869">I saw some topics like this one</a></p>
<p>My case is little more complex, I simplify it:</p>
<p>I have data, some of them have common parts but not all. There is not common part for all data</p>
<p>For example,</p>
<p>I have data of type A with features F0 … F6<br>
data of type B with features F0…F3 and F7…F8<br>
data of type C with features F6…F9</p>
<p>An illustration:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/f/f/fff38e89004d34cb4dae58c1941b879fca969385.png" alt="tab_type" data-base62-sha1="Awfzwr9nzWJKE3lea9KGOMpjPWR" width="648" height="196"></p>
<p>We can see all data have the same size. My idea is to do little neural networks for all subsets of features (S0 = F0…F3, S1 = F4…F6, S2 = F7…F8) and globals neurals networks (one by combinaison of two subsets Sn, in my little example, it means we have 3 globals networks) , this one takes the outputs of two little NN</p>
<p>An illustration:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/8/b81bc2ed502b1aa1eea82e84b9fb94582b908287.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/b81bc2ed502b1aa1eea82e84b9fb94582b908287" title="Neurals"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b81bc2ed502b1aa1eea82e84b9fb94582b908287_2_345x227.jpeg" alt="Neurals" data-base62-sha1="qgHkEFjwKFV5GaSq3fUiF1h00Xd" width="345" height="227" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b81bc2ed502b1aa1eea82e84b9fb94582b908287_2_345x227.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b81bc2ed502b1aa1eea82e84b9fb94582b908287_2_517x340.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b81bc2ed502b1aa1eea82e84b9fb94582b908287_2_690x454.jpeg 2x" data-dominant-color="F4F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Neurals</span><span class="informations">1660×1097 50.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>As we can see on the topic on top, we can do</p>
<pre><code>fc1 = nn.Linear(784, 500)
fc2 = nn.Linear(500, 10)
optimizer = torch.optim.SGD([fc1.parameters(), fc2.parameters()], lr=0.01)
</code></pre>
<p>but if we do that</p>
<pre><code>S_one = get_network_subset(num_subset = 0)
S_two = get_network_subset(num_subset = 1)
Global = get_network_global(nums_subsets = (0,1))
optimizer = torch.optim.SGD([S_one.parameters(), S_two.parameters(), Global.parameters()], lr=0.01)
</code></pre>
<p>Will Pytorch understand how to do the backpropagation (global network has the “deepest” weights) or I have to respect a specific order to pass parameters ?</p>
<p>At the end I want to be able to collectivize learning between models and I don’t find resources about that</p>
Torch.unique derivative not implemented

yong1990: <p>Recently, I need to use torch.unique to choose the unique elements in a 2D tensor as part of my graph computation. But the runtime error popped up <strong>derivative for _unique2 is not implemented</strong>. I used a similar function called <strong>tf.raw_ops.uniquev2</strong> in TensorFlow, which fulfills same functioning as torch.unique and it is differentiable in my case, I think. Anyone would know how to make this <strong>torch.unique</strong> differentiable? Or is there any suggestion someone would make? I will really appreciate it.</p>
<p>I also attached part of code for simulating my case:</p>
<p>import torch<br>
a = torch.tensor([1.0,1.0,2.1,3.1,3.5,2.0,2.0])<br>
w1 = torch.tensor(5.0, requires_grad=True)<br>
d = w1 * a<br>
u = torch.unique(d)<br>
w2 = torch.tensor(.5, requires_grad=True)<br>
g = w2 * u<br>
g.sum().backward()</p>

Jason_Liu1: <p>I met the same issue. So sad <img src="https://discuss.pytorch.org/images/emoji/apple/pleading_face.png?v=12" title=":pleading_face:" class="emoji" alt=":pleading_face:" loading="lazy" width="20" height="20"></p>

AlphaBetaGamma96: <aside class="quote no-group" data-username="yong1990" data-post="1" data-topic="118467">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yong1990/48/37220_2.png" class="avatar"> yong1990:</div>
<blockquote>
<p>tf.raw_ops.uniquev2</p>
</blockquote>
</aside>
<p>According to <a href="https://www.tensorflow.org/api_docs/python/tf/raw_ops#UniqueV2" class="inline-onebox" rel="noopener nofollow ugc">tf.raw_ops &nbsp;|&nbsp; TensorFlow v2.14.0</a>, that function isn’t differentiable. Are you sure you backprop through that method, <a class="mention" href="/u/yong1990">@yong1990</a>?</p>
<p>You could try using <code>torch.autograd.Function</code> and manually define a derivative, but how exactly do you differentiate a unique function with respect to its inputs? There could be a relation to dirac-deltas but who knows.</p>
<p>If you can get the same behavior as <code>torch.unique</code> with standard torch ops then you might be able to get a derivative that way.</p>
Using DDP on slurm cluster with NCCL_IB_DISABLE=1 is very slow

Jacfger: <p>I was trying to use DDP on a slurm cluster, but I got error and had to export NCCL_IB_DISABLE=1 to run the program.</p>
<p>I’m not sure if it has anything to do with the flag, the training progress seemed to be very slow. I wonder if it is normal or what. I have tried the same training procedure on two A100 (another machine) and four RTX 3090 (on the slurm cluster), I found that the four RTX 3090 were actually two times slower than the training scripts on the two A100.</p>
<p>If it’s the problem of this flag, is there anyway solve it? Because currently, I can ensure the program is actually running on the same machine (node), and from my understanding, NCCL is trying to use internet interface which leads to error while it’s trying to communicate with IB/RoCE protocol as there’re no suitable hardware. Since the all the processes will be on the same machine, is there anyway to configure slurm cluster, or PyTorch to configure NCCL such that it’s using the interconnects (probably PCIe) to communicate instead of ethernet?</p>

cbalioglu: <p><a class="mention" href="/u/kwen2501">@kwen2501</a> do you have any insights regarding the <code>NCCL_IB_DISABLE</code> flag?</p>

kwen2501: <p>Hi, if your training is on a single node, NCCL will likely not use any network interconnect for data communication in the first place, whether your network is IB/RoCE or traditional TCP/IP. For intranode communication, NCCL would try to use NVLink, PCI-e and system shared memory, roughly in this order.</p>
<p>If your training does run across multiple nodes, <code>NCCL_IB_DISABLE=1</code> would disable use of IB/RoCE for internode communication and fall back to socket. If your nodes do not have any IB devices installed, NCCL should be able to detect it and disable IB automatically.</p>
<p>To fully understand the problem, you can set <code>NCCL_DEBUG=INFO</code> and <code>NCCL_DEBUG_SUBSYS=INIT,GRAPH,ENV</code> to see your system settings. If you have more questions, you can also post to the <a href="https://github.com/NVIDIA/nccl/issues" rel="noopener nofollow ugc">NCCL project</a>.</p>