akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How do I run inference in parallel on the same model?

yaphel_lee: <p>Hello,</p>
<p>I have a model as follows, where I have multiple inputs (<em>x1, x2, x3</em>) which are needed to be fed to the same network <em>model1</em>. The basic idea is to feed them one by one and get the results separately.</p>
<p>My question is that is there a way to make this inference faster by inference in parallel?</p>
<p>Thank you in advance.</p>
<pre><code class="lang-auto">class MyModel(nn.Module):
    def __init__(self, model1):
        super().__init__()
        self.model1 = model1
    def forward(self, x1, x2, x3):
        out1 = self.model1(x1)
        out2 = self.model1(x2)
        out3 = self.model1(x3)
        return out1, out2, out3
</code></pre>

basingse: <ol>
<li>What you can do is stack <code>x1, x2 and x3</code> like having a batch size of 3 then use only a forward pass once as<pre><code class="lang-python"># assuming x1, x2 and x3 first dimension represent batch
b = x1.shape[0] 
inputs = torch.vstack((x1, x2, x3), dim=0)
out = self.model1(inputs)
out1 = out[:b]
out2 = out[b: 2*b]
out3 = out[2*b:]
</code></pre>
</li>
</ol>
Backpropagation for complex tensors in Pytorch

zerozy: <p>Hello,</p>
<p>I’m doing a project that aims to investigate complex NNs. I see that Pytorch has some implementations for complex NN layers, and would like to know how Pytorch handles the backpropagation for complex data (e.g. calculate derivatives, update parameters). It would be helpful if I could get some pointers to the codebase. Thank you!</p>

basingse: <p>Hope this helps</p>
<ul>
<li><a href="https://pytorch.org/docs/stable/complex_numbers.html" class="inline-onebox" rel="noopener nofollow ugc">Complex Numbers — PyTorch 1.11.0 documentation</a></li>
<li><a href="https://pytorch.org/docs/stable/notes/autograd.html#complex-autograd-doc" class="inline-onebox" rel="noopener nofollow ugc">Autograd mechanics — PyTorch 1.11.0 documentation</a></li>
</ul>
CNN Produces same output for every Image after initialization

Malte: <p>I’ve implemented a version off the VGG16-Net for regression (so it predicts one output value) and noticed that after random initialization of the network produces the same output value for every image. During training this starts to change as the loss decreases. I wonder if this behavior is due to the random initialization or if there is actually something wrong with my network?</p>

eqy: <p>Is this happening after a few training steps (e.g., due to bias in the dataset) or immediately after initialization? I wouldn’t expect this to be the case after initialization:</p>
<pre><code class="lang-auto"># cat vggtest.py
import torch
import torchvision

model = torchvision.models.vgg16(init_weights=True, num_classes=1)
model = model.cuda()

for i in range(10):
    inp = torch.randn(1, 3, 224, 224, device='cuda')
    out = model(inp)
    print(out)
</code></pre>
<pre><code class="lang-auto"># python3 vggtest.py
tensor([[0.2661]], device='cuda:0', grad_fn=&lt;AddmmBackward0&gt;)
tensor([[0.1230]], device='cuda:0', grad_fn=&lt;AddmmBackward0&gt;)
tensor([[0.1168]], device='cuda:0', grad_fn=&lt;AddmmBackward0&gt;)
tensor([[-0.0185]], device='cuda:0', grad_fn=&lt;AddmmBackward0&gt;)
tensor([[-0.0407]], device='cuda:0', grad_fn=&lt;AddmmBackward0&gt;)
tensor([[-0.0701]], device='cuda:0', grad_fn=&lt;AddmmBackward0&gt;)
tensor([[0.0390]], device='cuda:0', grad_fn=&lt;AddmmBackward0&gt;)
tensor([[0.0971]], device='cuda:0', grad_fn=&lt;AddmmBackward0&gt;)
tensor([[0.2076]], device='cuda:0', grad_fn=&lt;AddmmBackward0&gt;)
tensor([[0.1138]], device='cuda:0', grad_fn=&lt;AddmmBackward0&gt;)
</code></pre>
Where can I find patch-GAN discriminator impelementation in pytorch

Dalal: <p>Hello,</p>
<p>I am looking for patch-GAN discriminator implementation in pytorch. Where can I find it?</p>
<p>thanks in advance</p>

mxahan: <aside class="quote no-group" data-username="Dalal" data-post="1" data-topic="148954">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dalal/48/44954_2.png" class="avatar"> Dalal:</div>
<blockquote>
<p>patch-GAN discriminator implementation in pytorch</p>
</blockquote>
</aside>
<p>You can use the discriminator as <a href="https://github.com/eriklindernoren/PyTorch-GAN/blob/master/implementations/pix2pix/pix2pix.py" rel="noopener nofollow ugc">patchgan</a>. Described in line 56.</p>
Passing another parameter to LSTM

nkn002: <p>Hi,</p>
<p>I am implementing a model from a paper, in which they modify the hidden state of LSTMCell by multiplying with a matrix d_c so that for a sequence, this would become the previous state for the next timestep h_{t+1}.The equation is:<br>
h_t = [r_t ⊗ Φ(c_t)] ⊗ d_c</p>
<p>I understand can implement a custom LSTMCell to perform this task, but when I tried to make a custom LSTM, its speed is much slower than that implemented by Pytorch. Is there a way to still keep the speed of the Pytorch’s model while incorporating this new change?</p>
Grad Cam error in final two layers of network

hpsauce37: <p>I am attempting Binary classification on lung cancer tumours. The network inputs are “masks” of the tumours, with everything outside of the mask being set to 0 as a background. I have been having several issues with the last layers of the network. For the first 3 layers, the grad cams are activating as expected, but for the final two they arent. The activations on the last layer are appearing nowhere near the actual tumours, but instead in the zero background as circles, or the layers arent activating at all. This is possibly reflected in the testing phase of the program, which always returns an accuracy near 50%, despite the validation accuracy being much higher consistently. For this I am using a Resnet architecture found here <a href="https://github.com/kenshohara/3D-ResNets-PyTorch/blob/master/models/resnet.py" class="inline-onebox" rel="noopener nofollow ugc">3D-ResNets-PyTorch/resnet.py at master · kenshohara/3D-ResNets-PyTorch · GitHub</a> but I have replaced the final fc layer with another conv layer and a global average pool.</p>
DDP and using GPU in the dataloader

jprobichaud: <p>Hi everyone,</p>
<p>I have an original training pipeline that works well with DistributedDataParallel, running on a single machine with 8 GPUs.  So far, I haven’t used torchrun, I’m using DDP as-is, spawining one process per GPU in bash and specifying the ddp method/params manually on the command line.</p>
<p>Now I would like to use another model than the one I’m training as part of the dataloader pipeline, using multiple workers.  I would like that model to also use the GPU.  This is where problems start.  CUDA isn’t happy to be initialized again, asking me to set the mp.set_start_medhod to ‘spawn’, but when I do that, I get ask to call ‘freeze()’ and complains again about the context already being set.</p>
<p>I’ve tried to move to torchrun, in case some magic would happen, but that also doesn’t work in similar ways.</p>
<p>I’ll try to create a toy example to share here, but I wonder if there’s something obvious I’m missing here?</p>

kumpera: <p>A small example showing your issue would be great.</p>
<p>One thing you should try is to switch to single process data loading: <a href="https://pytorch.org/docs/stable/data.html#single-and-multi-process-data-loading" class="inline-onebox" rel="noopener nofollow ugc">torch.utils.data — PyTorch 1.11.0 documentation</a></p>
<p><a class="mention" href="/u/vitalyfedyunin">@VitalyFedyunin</a> do you have further recommendations here?</p>

jprobichaud: <p>Thanks for your quick reply.  Using a single worker process to generate the data works, but it is a lot slower for reasons related to the nature of the data and data pipeline requirements.</p>
<p>Rewriting the training loop from scratch a bit, I’m getting something out of the pipeline now, but the processing hangs at some point and I notice that only one GPU ever gets used, as if only one worker process is effectively used.</p>
<p>debugging continues…</p>

kumpera: <p>Maybe you can structure your training loop around pipeline parallelism with the loader-time model as the first stage.</p>
<p><a href="https://pytorch.org/docs/stable/pipeline.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/pipeline.html</a></p>
Conv2d/Maxpool2d and Conv3d/Maxpool3d

jhoanmartinez: <p>In CIFAR 10 tutorial on pytorch(<a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Training a Classifier — PyTorch Tutorials 1.11.0+cu102 documentation</a>) why use Conv2d and Maxpool2d if images are in 3d shape?</p>
<pre><code>import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):

def __init__(self):
    super().__init__()
    self.conv1 = nn.Conv2d(3, 6, 5)
    self.pool = nn.MaxPool2d(2, 2)
    self.conv2 = nn.Conv2d(6, 16, 5)
    self.fc1 = nn.Linear(16 * 5 * 5, 120)
    self.fc2 = nn.Linear(120, 84)
    self.fc3 = nn.Linear(84, 10)

def forward(self, x):
    x = self.pool(F.relu(self.conv1(x)))
    x = self.pool(F.relu(self.conv2(x)))
    x = torch.flatten(x, 1) # flatten all dimensions except batch
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = self.fc3(x)
    return x</code></pre>

AlphaBetaGamma96: <p>Images are 2D but color images have a corresponding ‘channel’ axis too. This is what the 3rd dimension in your <code>Conv2d</code> objects are. There’s more information in the documentation <a href="https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html" rel="noopener nofollow ugc">here</a>.</p>
Normalization values of UCF101

gyuko: <p>Hi,<br>
I’m working with 3D convolutional network and I have to train ( from scratch) the net on a subset of UCF101. Do you know what are the normalization values  ( mean, std) that I should to use with UCF101? Can I use the normalization values of ImageNet for UCF101? Thanks</p>

ptrblck: <p>The ImagNet stats might be a good starter, as the UCF101 dataset uses images from the same “domain” (natural images) and these stats are also used in tutorials such as <a href="https://cv.gluon.ai/build/examples_action_recognition/demo_tsn_ucf101.html">this one</a>.<br>
You could of course also compute the stats from the training set manually and compare them to the ImageNet ones.</p>

gyuko: <p>Thanks you for the reply !</p>
Differentiable way to assign an array/tensor to another tensor

HitM: <p>Hello,<br>
I’m not skillful in PyTorch so I’m sorry if I’m asking a weird question.</p>
<p>I have a tensor of zeros (say A) and an array that has a gradient (say B) and am trying to assign B to A while keeping the gradient.<br>
Then A is used as the input to the subsequent MLP (say MLP1) and loss is calculated.</p>
<p>Here’s a sample code</p>
<pre><code class="lang-auto">import torch
A = torch.zeros(3, 4, 5)
B = torch.arange(10, dtype=float, requires_grad=True).reshape(2, 5)
A[0, :2, :] = B # Does this operation keep gradient?
# A is used as an input to a subsequent network, e.g., MLP, and loss is calculated
</code></pre>
<p>where B is actually outputted from another MLP (say MLP2) and thus I’m expecting the back-propagation eventually update the weights of both the MLP1 and MLP2.</p>
<p>Does this operation work as I expect?</p>
<p>Thank you in advance!</p>

HitM: <p>I’m sorry, I have confirmed by myself that the model’s weight is updated.</p>
Optimization with pytorch - NN used in the objective function

Tomasz_Gutowski: <p>I am trying to use the Adam algorithm for optimization, but instead of getting a decrease in the loss function value it increases up to some value and then remains the same.<br>
The value of the objective function is calculated using a NN (<code>self.model</code> variable), this is how the value of the objective function i calculated:</p>
<pre><code class="lang-auto">self.time_steps = torch.arange(self.settings.t_u + self.settings.t_p, self.settings.t_d, self.settings.delta_t)
d_t = F.pad(var_list[0], (2, 0), mode='constant', value=0.0)
d_s = F.pad(var_list[1], (2, 0), mode='constant', value=0.0)
times = list(map(lambda x: generate_network_input_row(d_t, d_s, 2, x), self.time_steps))
scores = self.model(torch.stack(times))
return (F.relu(scores - self.settings.max_state) ** 2).mean()
</code></pre>
<p>The <code>generate_network_input_row</code> function is defined like that:</p>
<pre><code class="lang-auto">def generate_network_input_row(times, sizes, last_considered, current_time):
    indexes = (times &lt;= current_time).nonzero().flatten()[-last_considered:]
    return torch.cat((current_time - times[indexes], sizes[indexes]))
</code></pre>
<p>Even though gradients are calculated the optimization does not seem to go in the right direction (mostly the loss increases). Could you please point me in the right direction, what I am doing wrong?</p>
Batch norm layer in sequential stored as separate in state dict

Julian_Buchel: <p>Hi.</p>
<p>When defining a network as such:</p>
<pre><code class="lang-auto">class test(torch.nn.Module):
    def __init__(self):
        super(test, self).__init__()
        self.bn1 = torch.nn.BatchNorm2d(num_features=100, eps=1e-5)
        self.conv = torch.nn.Sequential(
            torch.nn.Conv2d(in_channels=3,out_channels=128,kernel_size=3),
            self.bn1
        )

if __name__ == "__main__":

    t = test()
</code></pre>
<p>The batch norm parameters appear twice in the <code>state_dict</code>. Me as a user expected only the <code>bn1</code> to appear in the <code>state_dict</code>, but it appears twice. Once with the <code>bn1</code> and once with the <code>conv</code> (from <code>Sequential</code>) prefix.<br>
Is this desired? I would say no, since the object at heart is the same, but could be wrong.</p>
Does pinning memory works when there are random augmentations?

Trump: <p>When <code>Dataset</code> has transforms like <code>torchvision.transforms.RandomErasing</code>, the resulted data tensor will have randomly modified value. In this case, does <code>pin_momory</code> in <code>DataLoader</code> class have any practical advantages?</p>

eqy: <p>As I understand it, pinned memory is used to make host-to-device memory transfers (e.g., CPU to GPU) more efficient, so it is orthogonal to an augmentation such as <code>RandomErasing</code>.</p>

Trump: <p><a class="mention" href="/u/eqy">@eqy</a> ok. I thought that the contents of memory matter when allocating page-locked memory… because the contents are consumed only single time.</p>
Writing output of neural net to specific buffer

jamied157: <p>If I have preallocated some buffers on my GPU using CudaMalloc(). Can I force my neural net to write the output to that buffer? I know I’m able to create a Tensor object using <code>torch::from_blob</code> using those buffers but then it’s not clear how I can force the neural net to write to those buffers without avoiding a redundant copy.</p>
<p>E.g:</p>
<pre><code class="lang-auto">void run(void* input_buffer, std::vector&lt;int&gt; input_shape, void* output_buffer, std::vector&lt;int&gt; output_shape){
    auto model = torch::jit::load(model_path)
    auto input_tensor = torch::from_blob(input_buffer, input_shape)
    auto output_tensor = torch::from_blob(output_buffer, output_shape)
    model.forward(input_tensor, output_tensor) // model writes to output buffer
}
</code></pre>
<p>I could definitely copy the data over from some temporary tensor but I’m interested in finding a way to avoid that overhead (or potentially an explanation why what I’m trying to do doesn’t make sense).</p>
<p>Thanks!</p>

ptrblck: <p>Some PyTorch operators provide the <code>out</code> argument, which can be specified to write the result directly to this tensor. Depending on your last operation you might be able to use this approach.</p>

jamied157: <p>Thanks for your response!</p>
<p>I think I’m looking for something a bit more generic, but maybe it really does just depend on what the last operator I’m using is doing.</p>
(KNN) when get Train accuracy, got an error

tjalstjr: <p>from sklearn.model_selection import train_test_split</p>
<p>X = df.drop(‘HeartDisease’, axis=1)</p>
<p>y = df[‘HeartDisease’]</p>
<p>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=1)</p>
<p>X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=1/9, random_state=1)</p>
<p>def euclidean_distance(a, b):</p>
<pre><code>distance = np.sqrt(np.sum(a - b)**2)

return distance
</code></pre>
<p>def manhattan_distance(a, b):</p>
<pre><code>distance = 0

for a_i,b_i in zip(a,b):

    distance += abs(a_i - b_i)  

return distance
</code></pre>
<p>def KNN_euclidean(X_train, X_test, y_train, y_test, K):</p>
<pre><code>y_list = []

for test_knn in X_test.to_numpy():

  distances = []

for i in range(len(X_train)):

  distances.append(euclidean_distance((np.array(X_train.iloc[i])), test_knn))

distance_data = pd.DataFrame(data = distances, columns = ['distance'], index = y_train.index)

k_neighbors_list = distance_data.sort_values(by=['distance'], axis = 0)[:K]

labels = y_train.loc[k_neighbors_list.index]

voting = mode(labels).mode[0]

y_list.append(voting)

return y_list
</code></pre>
<p>def KNN_manhattan(X_train, X_test, y_train, y_test, K):</p>
<pre><code>y_list = []

for test_knn in X_test.to_numpy():

 distances = []

for i in range(len(X_train)):

  distances.append(manhattan_distance((np.array(X_train.iloc[i])), test_knn))

distance_data = pd.DataFrame(data = distances, columns = ['distance'], index = y_train.index)

k_neighbors_list = distance_data.sort_values(by=['distance'], axis = 0)[:K]

labels = y_train.loc[k_neighbors_list.index]

voting = mode(labels).mode[0]

y_list.append(voting)

return y_list



euclidean_acc_trains = []

for i in range(1, 10):

**f = KNN_euclidean(X_train, X_test, y_train, y_test, K=i).fit(X_train, y_train)**

result = clf.predict(X_train)

euclidean_acc_trains.append(accuracy_score(y_train, result))

plt.plot(range(1, 10), euclidean_acc_trains, color='b', marker='o')
</code></pre>
<p>****AttributeError: ‘list’ object has no attribute ‘fit’<br>
i wanna get train accuracy with this code, but it doesnt work. how can i solve this prob?</p>
P.grad.grad_fn is None and p.grad.requires_grad = True

Glory_Chen: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> <a class="mention" href="/u/alband">@albanD</a> et al.,</p>
<p>May I ask in which case it will go into this <code>else</code> branch, i.e., <code>p.grad.grad_fn is None and p.grad.requires_grad = True</code>? I understand <code>requires_grad</code> will be <code>True</code> if <code>grad_fn</code> is not <code>None</code>, e.g., <code>create_graph</code> turned on in <code>autograd.grad</code>.</p>
<p>The source snippet is <a href="https://pytorch.org/docs/master/_modules/torch/optim/optimizer.html#Optimizer.zero_grad" rel="noopener nofollow ugc">optim.optimizer.zero_grad</a>.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/e/1e2a2fe82aa075e1c486109313e4b7a414180286.png" alt="image" data-base62-sha1="4iQHZQaZRN3tmIATt0tjgXB8iFw" width="274" height="86"></p>
<p>Thanks.</p>
Running PyTorch Lightning API with multiple Dataloaders because DataFrame too large

eTuDpy: <p>This seems like an easy question to solve, however, I did not find a working solution for it yet.</p>
<p>I have a very big time series dataset, which if put into one single dataloader will exceed the GPUs memory by far!</p>
<p>My general idea is to have a double for loop. First loop over the DataFrame, take a part of it, transform it into a dataloader and pass it into the second loop to run through as set set of epochs (and then a third for the batches), training and validating one and the same model.</p>
<p>If this works, the question occurs how to train the model. First I thought calling trainer.fit() multiple times, however, this seems not to work. Then I thought about writing a custom optimization loop in the classical PyTorch approach. However, the model I am using is from PyTroch Forecasting and runs on PyTorch Lightning, so I need to use the Lightning API. How do I do that?</p>
<p>Can I solve this by writing a custom training_step(batch, batch_idx, data_loader_idx)? But if doing so, how can I correctly implement it to run on multiple dataloaders?</p>
<p>FYI: My DataFrame is so large, that manually splitting it into <code>dataloader_i</code> does not work. The For-Loop over the DataFrame is manditory.</p>

nivek: <p>Are you using the standard <code>Dataset</code>? Have you tried using <code>IterableDataset</code> or the new <a href="https://pytorch.org/data/beta/torchdata.datapipes.iter.html" rel="noopener nofollow ugc"><code>IterDataPipe</code></a>?</p>
<p>One possible solution is to have your data on disk, then lazily load each sample as needed within an <code>IterDataPipe</code>, this should not require having your entire dataset in memory.</p>

eTuDpy: <p>I am using the <a href="https://pytorch-forecasting.readthedocs.io/en/stable/api/pytorch_forecasting.data.timeseries.TimeSeriesDataSet.html#pytorch_forecasting.data.timeseries.TimeSeriesDataSet" rel="noopener nofollow ugc">TimeSeriesDataSet</a> from PyTorch <a href="https://pytorch-forecasting.readthedocs.io/en/stable/index.html" rel="noopener nofollow ugc">Forecasting</a>.</p>
<p>So from DataFrame to TimeSeriesDataSet to DataLoader.</p>
<p>I am not quite sure if it will work with IterDataPipe. I’ll look into it.</p>
Pytorch installing error on jhub just appeared today and not before

Cecilia_s: <p>I have been installing pytorch with this command and worked perfectly fine before until today<br>
Has there been an update and I should be installing the package that I want another way? Thank you!<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/3/73929421b33cb031ce7c3d09bf6c7d69dd645117.png" data-download-href="https://discuss.pytorch.org/uploads/default/73929421b33cb031ce7c3d09bf6c7d69dd645117" title="Screen Shot 2022-04-11 at 1.12.44 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/3/73929421b33cb031ce7c3d09bf6c7d69dd645117_2_690x130.png" alt="Screen Shot 2022-04-11 at 1.12.44 PM" data-base62-sha1="guoX2YGvogK2lXJy0zujepLMNkH" width="690" height="130" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/3/73929421b33cb031ce7c3d09bf6c7d69dd645117_2_690x130.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/3/73929421b33cb031ce7c3d09bf6c7d69dd645117_2_1035x195.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/3/73929421b33cb031ce7c3d09bf6c7d69dd645117_2_1380x260.png 2x" data-dominant-color="F2F3F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2022-04-11 at 1.12.44 PM</span><span class="informations">2508×474 106 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Cecilia_s: <p>my python version is 3.9.7</p>

AlphaBetaGamma96: <p>Did you follow the install guide on <a href="https://pytorch.org/get-started/locally/" class="inline-onebox" rel="noopener nofollow ugc">Start Locally | PyTorch</a> to get the right <code>pip</code> command? It seems that URL doesn’t work. Or are you trying to install a specific version of PyTorch?</p>
<p>Installing 1.11 with CUDA 11.3 via pip is given by</p>
<pre><code class="lang-auto">pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113
</code></pre>

ptrblck: <p>Based on your output you are seeing a warning due to a <code>pip</code> update and the <code>Collecting torch==1.9.0+cu111</code> message shows that PyTorch was apparently installed. Do you see any errors or re you only concerned about the warning?</p>
Slowdown when running multiple processes doing inference in parallel

Prashant: <p>Hi I have a torchscripted transformer model and I’m trying to evaluate the performance of my C++ inference code. To this end, I’m running on a server with 32 CPUs  (and doing inference on CPU). I have set OMP_NUM_THREADS=1 and MKL_NUM_THREADS=1 but I still notice that running 30 inference processes in parallel causes each forward step to be significantly slower (up to 2x slower) than running inference jobs one at a time.</p>
<p>Is there something else I’m missing?</p>

hoyden: <p><a class="mention" href="/u/prashant">@Prashant</a> I met the same question.Did you solve it?</p>
Resize Image Dimensions of a 4D tensor

danielmanu93: <p>Hi All,</p>
<p>I have an 4D image tensor of dimension (10, 10, 256, 256) which I want to resize the image height and width to 100 x 100 such that the resulting 4D tensor is of the dimension (10, 10, 100, 100). I have tried several options but not getting the required dimensions. Any suggestions to resolved this are welcomed.</p>

eqy: <p>Have you tried <a href="https://pytorch.org/docs/stable/generated/torch.nn.AdaptiveAvgPool2d.html" rel="noopener nofollow ugc">AdaptiveAvgPool2d — PyTorch 1.11.0 documentation</a>?</p>
Getting non-tensor members from C++ code

daver: <p>Hey folks. Suppose I have the following model in python:</p>
<pre><code class="lang-auto">@torch.jit.script
class ModelProperties:
    def __init__( self, state: int, time: float ):
        self.state = state
        self.time = time

class MyModel(nn.Module):
    def __init__self, props: ModelProperties):
        self.props_ = props
</code></pre>
<p>Now, I can save it to torchscript using <code>torch.jit.script</code>, and load it as a <code>torch::jit::script::Module</code>. Is there away to access that <code>self.props_</code> field from the C++ code? I see that there is API for getting “parameters”, but I don’t see a way to get or set the non-parameter members. Is that a thing that can be done, or do I have to use parameter objects for everything I might want to change?</p>
Post processing

einrone: <p>Hi!</p>
<p>I am doing a lesion segmentation for multiple sclerosis (MS), and at the moment I am using a attention unet for my thesis. The best validation dice score I have recieved is 0.771 and train 0.84. I am thinking of doing some post processing for removing some FP and FN in order to enhance the predictions. Any advice? currently I am using opening and closing, and I am not sure if this is the right approach.</p>
Hyper parameter tuning tips

NearsightedCV: <p>Hi</p>
<p>I am looking for beginner tips on hyper parameter tuning in Pytorch. Is there anything within the library that can do this? I have googled a bit and most refer to other packages. Also what would one target first if your application is semantic segmentation. I am getting decent results with Cross Entropy loss between 2-5% on my training set, though it oscillates a bit. I think I would need to tune stuff like learning rate and weights to get it better than this. Not sure where to start.</p>
<p>Thanks in advance.</p>
How to cite DDP in scientific projects?

mlearn: <p>Hello everyone!</p>
<p>I found the general Pytorch reference. But how should I reference the DDP package if I want to explicitely highlight it?</p>
<p>Many thanks!</p>

agu: <aside class="onebox allowlistedgeneric" data-onebox-src="https://arxiv.org/abs/2006.15704">
  <header class="source">
      <img src="https://static.arxiv.org/static/browse/0.3.4/images/icons/favicon.ico" class="site-icon" width="32" height="32">

      <a href="https://arxiv.org/abs/2006.15704" target="_blank" rel="noopener nofollow ugc">arXiv.org</a>
  </header>

  <article class="onebox-body">
    <img src="https://static.arxiv.org/icons/twitter/arxiv-logo-twitter-square.png" class="thumbnail onebox-avatar" width="500" height="500">

<h3><a href="https://arxiv.org/abs/2006.15704" target="_blank" rel="noopener nofollow ugc">PyTorch Distributed: Experiences on Accelerating Data Parallel Training</a></h3>

  <p>This paper presents the design, implementation, and evaluation of the PyTorch
distributed data parallel module. PyTorch is a widely-adopted scientific
computing package used in deep learning research and applications. Recent
advances in deep learning...</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
Device errors despite explictly placing tensors?

neel_g: <p>It seems that whenever I try to perform inference in plain Pytorch style (Using Pytorch <code>Lightning</code>)</p>
<pre data-code-wrap="py"><code class="lang-nohighlight">inf_device = torch.device('cpu')

dummy, _ = next(iter(validation_loader))
#move dummy to inf_device
dummy = dummy[0].reshape(1, 3, 256, 256).to(inf_device)
dummy.to(inf_device)

# print which device dummy is on
print('\n\nDUMMY:',dummy.device)
print(dummy.shape)

Predator_model.to(inf_device)
Predator_model.eval()

with torch.no_grad():
    out = Predator_model(dummy)
</code></pre>
<p>I obtain errors complaining the input being on <code>CUDA</code>, but the weights succesfully placed on <code>CPU</code>. From the print statements placed, I can confirm it shows device as <code>CPU</code></p>
<p>This is a weird bug I’ve faced many times. Any idea how to solve this?</p>

albanD: <p>The <code>.to()</code> function on Tensors is NOT inplace. You should do <code>dummy = dummy.to(inf_device)</code>.</p>
Generalized broadcasting

Santosh_Manicka: <p>Are there any in-built methods for implementing generalized broadcasting during tensor operations? I’m specifically looking for a sort of ‘block’ broadcasting that could enable, for example, the multiplication of a tensor of shape (m,n) with another tensor of shape (3m,2n) where every cell of the first tensor would be multiplied by a corresponding (3,2) block of the other tensor.</p>

Santosh_Manicka: <p>I just found a solution here: <a href="https://stackoverflow.com/questions/56601115/partitioned-matrix-multiplication-in-tensorflow-or-pytorch" class="inline-onebox" rel="noopener nofollow ugc">Partitioned matrix multiplication in tensorflow or pytorch - Stack Overflow</a> but other suggestions are welcome.</p>
Randomness in weight init is not guaranteed on linux

Alon: <p>Following this closed issue: <a href="https://github.com/pytorch/pytorch/issues/5059" rel="noopener nofollow ugc">np.random generates the same random numbers for each data batch · Issue #5059 · pytorch/pytorch (github.com)</a></p>
<p>We know that NumPy on Linux (in contrary to Windows) uses the same seed every time unless manually set. The issue was closed by adding a re-seeding to the DataLoader<br>
(see <a href="https://github.com/pytorch/pytorch/pull/56488" rel="noopener nofollow ugc">[DataLoader] Add Numpy seeding to worker of DataLoader by ejguan · Pull Request #56488 · pytorch/pytorch (github.com)</a>)</p>
<p>However, as far as I can tell, the issue still exists in the weight initialization when creating a model or when calling reset_parameters() to reset the weights of a module.</p>
<p>Would it be possible to implement the seeding in the reset_parameters() methods?<br>
Otherwise to ensure randomness across repeats on Linux, I think that a manual NumPy random seed should be forced.</p>

ptrblck: <p>The <code>.reset_parameters()</code> methods should not depend on numpy (and would break if numpy is not installed as an optional dependency). Seeding PyTorch should be enough to get reproducible results.<br>
Could you post a code snippet showing the issue you are hitting, please?</p>

Alon: <p>Upon writing a minimized code snippet to reproduce this, I found that I had an import in my code to an old (and unused) python file that had <code>torch.manual_seed(2021)</code> under the imports lines (and not inside a function) which subsequently caused all following torch functions to use this seed <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=12" title=":blush:" class="emoji" alt=":blush:" loading="lazy" width="20" height="20"><br>
I think it’s a worthwhile cautionary tale so it may be of some value if anyone ever reads this.</p>
Train Test Split using SubsetRandomSampler

nisharaichur: <p><a href="https://stackoverflow.com/questions/50544730/how-do-i-split-a-custom-dataset-into-training-and-test-datasets" rel="noopener nofollow ugc">https://stackoverflow.com/questions/50544730/how-do-i-split-a-custom-dataset-into-training-and-test-datasets</a></p>
<p>I am trying to split the dataset into train-test datset using the SubsetRandomSampler.</p>
<p>I want to achieve this:<br>
Shuffle the train dataset and not shuffle the validation dataset.</p>
<p>I also need first 80% of the data for training and last 20% for validation.</p>
<p>DOes this makes sense?</p>
<pre><code class="lang-auto">from torch.utils.data.sampler import SubsetRandomSampler, SequentialSampler 

train_split = 0.8
random_seed = 42

dataset_size = len(dataset)
validation_split = .2
random_seed= 42
indices = list(range(dataset_size))
split = int(np.floor(validation_split * dataset_size))
train_indices, val_indices = indices[split:], indices[:split]

train_sampler = SubsetRandomSampler(train_indices)
valid_sampler =  SequentialSampler(val_indices)

train_loader = torch.utils.data.DataLoader(dataset, batch_size=64, 
                                           sampler=train_sampler)
validation_loader = torch.utils.data.DataLoader(dataset, batch_size=64,
                                                sampler=valid_sampler)
</code></pre>

ptrblck: <p><code>SequentialSampler</code> expects a <code>Dataset</code> as the input, so you could use <code>Subset</code> for the validation set instead:</p>
<pre><code class="lang-python">val_dataset = Subset(dataset, val_indices)
validation_loader = DataLoader(val_dataset, batch_size=64, shuffle=False)
</code></pre>
<p>Beside that the code looks alright.</p>

nisharaichur: <p>Thank you works fine now <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

reza_shahabian: <p>Thanks for your information, but there is a big problem when your dataset data does not benefited from random distributed data. In my case, I had a range on data which each class’s data stacked back to back. meaning that after each class I have another class. so this code just split some class data, then just chose rest of data which belongs to another classes as test data.<br>
tried below code and it works for me to randomized data.<br>
<code>train_size = int(0.8 * len(full_dataset)) test_size = len(full_dataset) - train_size train_dataset, test_dataset = torch.utils.data.random_split(full_dataset, [train_size, test_size])</code></p>
Keeping all BERT embeddings of input sequence instead of CLS

elsharko: <p>Hi there,</p>
<p>I’m currently implementing a model in which I need to compute a cross-attention between sequences from different modalities. For the textual sequence, I thus want to use BERT to embed my raw sentences into textual embeddings.<br>
I use the <code>transformers</code> package this way to initialize my encoder:</p>
<pre><code class="lang-auto">bertconfig = BertConfig.from_pretrained('bert-base-uncased',
output_hidden_states=True)
self.bertmodel = BertModel.from_pretrained('bert-base-uncased', config=bertconfig)
</code></pre>
<p>As I want to keep the cross-attention between modalities on the whole sequence (and observe the words triggering the most the attention weights), I currently keep the embeddings of all input words, rather than only the one corresponding to the CLS token. Is it OK to proceed this way ?</p>
<p>Plus, is it OK if I then slice my input sequence (composed of all BERT embeddings) into different smaller sequences and work locally on each of these subsequences ? I mean that in the sense of the meaningfullness of the BERT embeddings of the different subsequences, as the initial embeddings are computed contextually, on the whole initial sequence, and with adequate positional encodings.</p>
<p>I ask these questions because my model seems not to learn properly with this approach, whereas it successfully learns when I use GloVe embeddings.</p>
<p>Thank you in advance for your answer !</p>
Generator not converging?

ljeonjko: <p>Hi guys!</p>
<p>I’m trying to implement Contex Conditional GAN (<a href="https://arxiv.org/abs/1611.06430" rel="noopener nofollow ugc">paper</a>, <a href="https://github.com/TomislavZupanovic/Data-Reconstruction/blob/main/source/ccgan.py" rel="noopener nofollow ugc">github</a>) as a means of data inpainting. It is my understanding of GANs that the losses of discriminator and generator should converge to approximately same value at one point. However, in my case the discriminator loss is converging to 0, while generator loss is converging to 1, as shown in the graph.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/3/4317545e1ec5462a3d2459aa68f23abf1a4d8dbc.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/4317545e1ec5462a3d2459aa68f23abf1a4d8dbc" title="convergence"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/3/4317545e1ec5462a3d2459aa68f23abf1a4d8dbc_2_690x460.jpeg" alt="convergence" data-base62-sha1="9zvYnzwDKpwweUDsOGNSqWAu3h2" width="690" height="460" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/3/4317545e1ec5462a3d2459aa68f23abf1a4d8dbc_2_690x460.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/4/3/4317545e1ec5462a3d2459aa68f23abf1a4d8dbc_2_1035x690.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/3/4317545e1ec5462a3d2459aa68f23abf1a4d8dbc_2_1380x920.jpeg 2x" data-dominant-color="FBF6EE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">convergence</span><span class="informations">1920×1280 116 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Am I doing something wrong or am I just misunderstanding how to gauge whether my network is fully trained or not? Thanks!</p>
Number of data points of DataLoader with SubsetRandomSampler

cerebrou: <p>We have a DataLoader that is using SubsetRandomSampler. Is it possible to tell the number of datapoints in the loader (which are sampled) from the DataLoader?</p>

ptrblck: <p>You can print the <code>len</code> of the internal <code>.sampler</code> as seen here:</p>
<pre><code class="lang-python">N = 100
dataset = torch.utils.data.TensorDataset(torch.randn(N, 1))
sampler = torch.utils.data.sampler.SubsetRandomSampler(indices=torch.arange(N//2))
loader = torch.utils.data.DataLoader(dataset, sampler=sampler, batch_size=7)

print(len(loader))
# 8
# 7 full batches and 1 batch with a single element: 7*7+1*1 = 50

print(len(loader.sampler))
# 50 number of samples
</code></pre>

cerebrou: <p>Perfect, it seems that it’s even working when there is no sampler and return the value of the whole dataset.</p>
<p>You’re the best, <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
LSTM inference slower in Litorch

hoyden: <p>I have wrote my own LSTMCell as below, because I need to get some parameters of the model from somewhere else.</p>
<blockquote>
<p>class MyLSTMCell(nn.LSTMCell):<br>
def <strong>init</strong>(self, input_size, hidden_size, bias=True, device=None, dtype=None):<br>
┆   super(MyLSTMCell, self).<strong>init</strong>(input_size, hidden_size, bias, device, dtype)<br>
def forward(self, inputs, hidden, cell, weight_hh, bias_hh):<br>
┆   return torch._VF.lstm_cell(<br>
┆   ┆   ┆   inputs, (hidden, cell), self.weight_ih,<br>
┆   ┆   ┆   weight_hh, self.bias_ih, bias_hh)</p>
</blockquote>
<p>But when I transform it to torchscript, and run in libtorch C++, it’s slower than original LSTMCell in pytorch. So what happened? And Is there any way to make it run faster?</p>