akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Breakout Sequential

MaKaNu: <p>I have builded a model of darknet using alot of nn.Sequential modules. I choosed to do so, because I was thinking: putting the Sequential parts together in my nn.Module initializer method benefits later speed instead iterating over a nn.ModuleList in the forward method.</p>
<p>This works fine and I tested a dozen times the output shapes. My Problem is now, that if I wont to use this model for yolov3 implementation I need to access the repeat 8 time Residual Blocks. Identifying these blocks would not be an issue with self.backbone.modules().</p>
<p>This generates two problems:</p>
<ol>
<li>Using self.backbone.modules() to iterate over them in the forward method destroys kind of my idea</li>
<li>For some reason</li>
</ol>
<pre><code class="lang-python">for layer in self.backbone.modules():
    x = layer(x)
</code></pre>
<p>Results in an error and is absolute difficult at which position in the network this error occurs (it is a shape problem). While</p>
<pre><code class="lang-python">x = self.backbone(x)
</code></pre>
<p>works perfectly fine.</p>
<p><strong>What is best practice to solve both issues?</strong></p>
<p>Some Ideas I have:</p>
<ol>
<li>Instead of deploying the backbone all at once I could deploy it in three steps and output three different outputs (at the end and after both 8-time-repeats).</li>
<li>Working somehow with naming the layers/outputs and access them somehow (I have no clue how to achieve this, but sounds like the most reasonable approach)</li>
</ol>

ptrblck: <aside class="quote no-group" data-username="MaKaNu" data-post="1" data-topic="147967">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/makanu/48/48166_2.png" class="avatar"> MaKaNu:</div>
<blockquote>
<p>I choosed to do so, because I was thinking: putting the Sequential parts together in my nn.Module initializer method benefits later speed instead iterating over a nn.ModuleList in the forward method.</p>
</blockquote>
</aside>
<p>I don’t think this is true, since <code>nn.Sequential</code> is also using a for loop internally as seen <a href="https://github.com/pytorch/pytorch/blob/2bfa0184623c4bdc6caea0332cc1ef6d019892d1/torch/nn/modules/container.py#L137-L140">here</a>.<br>
If your submodule workflow is easier I would go for it.</p>

MaKaNu: <p>Well that might change my approach. What about optimized code for cuda processing? Does Sequential performs better in any circumstance?</p>

ptrblck: <p>No, I doubt you would see any difference between using <code>nn.Sequential</code> and e.g. calling the modules line-by-line manually.<br>
If you are concerned about the launch overhead for each CUDA kernel, check <a href="https://pytorch.org/docs/stable/notes/cuda.html#cuda-graph-semantics">CUDA graphs</a> but note the limitations of using it.</p>
Randomness during batching or bucketing in torchdata for Map-style pipes

leo19941227: <p>After reading the doc and a few source code, it comes to me that, despite there are Batcher and Shuffler as the built-in pipes, should we really use them instead of relying on DataLoader and Sampler?<br>
Since it seems that Shuffler is not dynamically shuffling across epochs, which produce deterministic batches, and there is no API to know the current epoch when calling <strong>getitem</strong> (compared to the set_epoch in traditional samplers).<br>
The former is simpler to support, while the later seems to involve some API change.<br>
I am wondering will there be solutions for “dynamically sort the items/batches with the epoch argument” across epochs in the future?</p>
<p>Sorry for bothering if there is already an existing way to do this.<br>
Thanks very much!</p>

nivek: <p>Hi,</p>
<p>Thanks for your feedback. We are aware of the first issue that you mentioned, it is being <a href="https://github.com/pytorch/data/issues/295" rel="noopener nofollow ugc">tracked and worked on here</a>.</p>
<p>As for your second point, I believe before <code>torchdata</code>, the solution is to create a new Sampler for each epoch. You can write a custom DataPipe that takes in an epoch argument and generate a new DataPipe for each epoch. Let me know if what your usage example looks like and how the current API is insufficient or inefficient. We are open to improving our API for additional common use cases.</p>

leo19941227: <p>Thanks!</p>
<p>I think the solution of new Sampler with epoch argument works for me. I did not realize I can create new data pipes inside the training loop (right before starting a new epoch). Thanks!</p>
How to generate a dataloader with file paths in pytorch?

theProcrastinatr: <p>Hi,<br>
I have a image dataset which consists of a <code>csv</code> file and two folders containing images. The <code>csv</code> file also contains all correct paths to the images in two folders.</p>
<p>Now I’m trying to generate a <code>DataLoader</code> by creating a <code>Dataset</code> object with the file paths present in the csv file. But, the path being either <code>str</code> or <code>pathlib.PosixPath</code> type wouldn’t work with dataloader as it expects <code>Tensors</code> and similar data.</p>
<p>So, is there any way to make use of the paths present in the <code>DataFrame</code>? I ask this because the data is quite large and won’t fit if I try to load it all at once.</p>

Matias_Vasquez: <p>This example taken from <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html#dataset-class" rel="noopener nofollow ugc">here</a> shows you how to use a <code>csv</code> file to create your own <code>Dataset</code>.</p>
<p>In the <code>__getitem__()</code> method you can define how you want to load the images. This means you can take the <code>str</code> defining where your image is and read it from there.</p>
<pre><code class="lang-python">class FaceLandmarksDataset(Dataset):
    """Face Landmarks dataset."""

    def __init__(self, csv_file, root_dir, transform=None):
        """
        Args:
            csv_file (string): Path to the csv file with annotations.
            root_dir (string): Directory with all the images.
            transform (callable, optional): Optional transform to be applied
                on a sample.
        """
        self.landmarks_frame = pd.read_csv(csv_file)
        self.root_dir = root_dir
        self.transform = transform

    def __len__(self):
        return len(self.landmarks_frame)

    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()

        img_name = os.path.join(self.root_dir,
                                self.landmarks_frame.iloc[idx, 0])
        image = io.imread(img_name)
        landmarks = self.landmarks_frame.iloc[idx, 1:]
        landmarks = np.array([landmarks])
        landmarks = landmarks.astype('float').reshape(-1, 2)
        sample = {'image': image, 'landmarks': landmarks}

        if self.transform:
            sample = self.transform(sample)

        return sample
</code></pre>

theProcrastinatr: <p>I too ended up doing just that. Thanks <a class="mention" href="/u/matias_vasquez">@Matias_Vasquez</a>  for confirming that its the way to go <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=12" title=":smile:" class="emoji" alt=":smile:" loading="lazy" width="20" height="20"></p>
How to remove some layers in Resnet18?

joseph7: <p>I am using the pre-trained model of resnet18 through torchvision. Now I only want to use its backbone to do feature extraction, which means I don’t need the last few layers as the image shows:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/0/70e12e52e6ee06271958e80bf689e8a266db90bf.png" data-download-href="https://discuss.pytorch.org/uploads/default/70e12e52e6ee06271958e80bf689e8a266db90bf" title="9452198a48cd94957a0c24f55118025"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/70e12e52e6ee06271958e80bf689e8a266db90bf_2_690x314.png" alt="9452198a48cd94957a0c24f55118025" data-base62-sha1="g6zVknpfWUTxbpbns6USnVcBH3x" width="690" height="314" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/70e12e52e6ee06271958e80bf689e8a266db90bf_2_690x314.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/70e12e52e6ee06271958e80bf689e8a266db90bf_2_1035x471.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/0/70e12e52e6ee06271958e80bf689e8a266db90bf.png 2x" data-dominant-color="EFEDED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">9452198a48cd94957a0c24f55118025</span><span class="informations">1126×513 116 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
How should I remove it?<br>
I try to use: layers = list(enc.children())[:-2]<br>
but it only can remove avgpool and fc layers. I don’t know how to remove the last bn layer.</p>

mxahan: <p>You can use the <a href="https://discuss.pytorch.org/t/how-can-l-load-my-best-model-as-a-feature-extractor-evaluator/17254/5">hook</a> to pull the Conv1 output and ignore the rest.</p>
<p>Better Solution in this <a href="https://discuss.pytorch.org/t/how-to-delete-layer-in-pretrained-model/17648/4">post</a></p>
Lightning flash pretraining produces empty string while doing prediction

Pracheta_Sahoo: <p>Hi all,<br>
I have finetuned the python-lightning-flash module on custom dataset. I followed the instructions from <a href="https://devblog.pytorchlightning.ai/fine-tuning-wav2vec-for-speech-recognition-with-lightning-flash-bf4b75cad99a" rel="noopener nofollow ugc">this link</a>. When I am loading the checkpoint and starting to predict it produces empty string. Has someone faced this issue? Need urgent help.</p>
Pytorch 1.8 and above with no CUDA on Mac OS?

maria.solyanik: <p>Hi everyone! I am a beginner. I am trying to instal pytorch 1.8 (at least) with no CUDA on Mac OS Big Sur. But no matter what I do, I keep on getting the version 1.7.1. I use conda. The problem is that this version seems to have outdated tensor algebra modules, like for instance fft doesn’t have fftfreq. Thanks in advance!</p>
Template code for a local video file for PyTorch Live

kmitsunami: <p>Hi, I am currently thinking of an app that runs an ML model frame-by-frame on a video file, is there a template code that would be suitable for this? Sample code for a single image or camera stream exist in PyTorch Live code, but it would be helpful if you could provide a sample code that processes against a local video file, or even pointers for implementation.<br>
Thank you in advance.</p>

Andrei_Cristea: <p>Are you looking to do this for model training or for inference of a pre-trained model?</p>

kmitsunami: <p>I’m looking to do this fo inference of pre-trained models and custom models (specifically, object detection and santic segmentation). I wanted to try PyTorch Live quickly, but I’m new to react native. So any input would be appreciated.</p>
Faster index the tensor

yiminchi: <p>Hi,</p>
<p>Due to some reason, I need to do the operation below.</p>
<pre><code class="lang-python">
labels = torch.tensor([[1, 0, 0, 1], [1,0, 1, 1]])
counts = torch.tensor([[2, 2], [1, 3]])
F = torch.zeros(2, 2, 4)
for i in range(2):
    for j in range(2):
        F[i, j, labels[i]==j] = counts[i][j]
print(F)
</code></pre>
<p>Out:</p>
<pre><code class="lang-python">tensor([[[0., 2., 2., 0.],
         [2., 0., 0., 2.]],

        [[0., 1., 0., 0.],
         [3., 0., 3., 3.]]])
</code></pre>
<p>The double for loop lead to the slow operation. Is there any torch function can make it faster?<br>
Thanks!</p>

yiminchi: <p>I solved this problem by writing a custom cuda extension. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

chchannn: <p>Hi there, I ran into the same problem. Do you mind sharing your solution?</p>
Speeding up autograd for very complex pointwise tensor operations

magnusross: <p>Hi!</p>
<p>I have a question related to speeding up a model I have written involving broadcasting complicated pointwise functions over large tensors with many (~5) dimensions.</p>
<p>In the model I have functions like this (there are 4 different functions of equivalent complexity),</p>
<pre><code class="lang-auto">@torch.jit.script
def I_k_phi(t, alpha, p1, z1, thet2, beta2):
    pi = math.pi
    coeff = torch.sqrt(pi / (alpha + p1))
    ea1 = torch.exp(-(4 * alpha * p1 * z1 ** 2 + thet2 ** 2) / (4 * (alpha + p1)))
    ea2 = torch.cos(beta2 + thet2 * (t - (p1 * z1) / (alpha + p1)))
    return coeff * ea1 * ea2
</code></pre>
<p>The tensors I am mapping over are of shape (S, B, D, N1, N2), where S is O(1), B is O(1000), and N1 and N2 are O(100). The problem is that for some datasets I require D to be ~8, but the model uses more memory than my GPU can handle for D&gt;6. After doing some profiling, the memory becomes a problem only for the backward pass. I have also tried replacing these complex functions with much simpler (but incorrect) functions, and the memory problems completely disappear.</p>
<p>An idea I had to help reduce the memory use is to implement the gradients of the complex functions by hand, using a custom <code>torch.autograd.Function</code>. Obviously this will take a lot of work, but I am happy to do it if I can be reasonably confident it will help. Is it likely this will help, or am I confused about how the autograd engine works?</p>
<p>Any help with this or alternative ideas for solutions would be much appreciated.</p>
<p>Thanks in advance!</p>
Binary Classification with Subclasses/Subcategories

talachem: <p>Dear PyTorch Forum,<br>
is it possible to have a binary classification with multiply categories? The idea is basically that I have multiple subcategories, which can categorised into two groups. For example I have the subcategories:</p>
<ul>
<li>A, B, 1 and 2<br>
which than can be:</li>
<li>letters or numbers</li>
</ul>
<p>additionally I am interested which of these four categories causes the most errors. Hope I explained well enough what I want/need. Thank you for your help in advance.<br>
Cheers,<br>
talachem</p>
Quantization aware training for parameterized model

Dominique_Kosters: <p>I have a parameterized linear layer in my model and would like to do quantization aware training with the model. However, if I prepare my model for quantization aware training I get the following error: “RuntimeError: Serialization of parametrized models is only supported through state_dict().”. It also provides me with a link to a guide about saving and loading models, which does not help me.<br>
How can I do quantization aware training with a parameterized layer?</p>
Feeding the CNN with Specific Features

okuryapar: <p>Hi,</p>
<p>I want to feed a CNN with vectors or matrices based on the result of some feature extraction methods like GLCM, Haralick, etc. How can I configure it?</p>

Andrei_Cristea: <p>Have a look at <a href="https://discuss.pytorch.org/t/setting-custom-kernel-for-cnn-in-pytorch/27176/3">this post</a>.</p>
<p>Briefly, as a mini-example:</p>
<pre><code class="lang-auto">import torch
conv = torch.nn.Conv2d(3, 16, 1)
desired_weight = np.random.randn(16, 3, 1, 1)  # initiating with something random for illustration
conv.weight = torch.nn.Parameter(torch.Tensor(desired_weight))
</code></pre>
<p>This will set the weights (but not the biases <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> exercise left to the reader) to your custom <code>desired_weight</code>.</p>
<p>If you’d like to freeze these layers during subsequent training (for example you want to use these precise features but you do want to train a downstream classifier) you can do this:</p>
<pre><code class="lang-auto">for param in conv.parameters():
    param.requires_grad = False
</code></pre>
<p>Hope this helps!<br>
Andrei</p>
Can any one tell , how to evaluate the log-mel-filter bank energy?

krishna511: <p>I want to find this feature —log mel filter bank energy for a speech data, and eventually feed it to the deep learning model.<br>
Thats a different thing, as of now I am confused is it log of MFCC as we are doing in librosa or there is any other way?<br>
any help will do</p>
<p>Regards</p>

NehaT: <p>Log mel filterbank energy is not log of MFCC. This is computed prior to MFCC computation.<br>
You can find these features implemented in python speech features package in Python:</p>
<p>import python_speech_features as psf<br>
psf.logfbank()</p>
Partially initialized module 'torchaudio' has no attribute 'pipelines'

Gorgen: <p>hello everyone.<br>
when  run the command as follows, it reporeted partially initialized module ‘torchaudio’ has no attribute ‘pipelines’ (most likely due to a circular import).</p>
<p>import torchaudio<br>
bundle = torchaudio.pipelines.HUBERT_BASE<br>
model = bundle.get_model()</p>
<p>here are packages.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/3/5/35c6c7b80c5465c81a9a524a850410951120d6e6.png" alt="_SK57N90H4BM2TJ@8RDVTS" data-base62-sha1="7FJbO4uZNZZLY3D5ftK22b0qzFs" width="638" height="362"></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/8/58f75f7f039961a431821a55b0aa0f9a78273c6b.png" data-download-href="https://discuss.pytorch.org/uploads/default/58f75f7f039961a431821a55b0aa0f9a78273c6b" title="A8@G`K$L$AFIG59LAZS{Q95"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/8/58f75f7f039961a431821a55b0aa0f9a78273c6b_2_690x209.png" alt="A8@G`K$L$AFIG59LAZS{Q95" data-base62-sha1="cH20ISH7ptTys1hr0ifgVKvTTED" width="690" height="209" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/8/58f75f7f039961a431821a55b0aa0f9a78273c6b_2_690x209.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/8/58f75f7f039961a431821a55b0aa0f9a78273c6b_2_1035x313.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/8/58f75f7f039961a431821a55b0aa0f9a78273c6b.png 2x" data-dominant-color="2F2E2E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">A8@G`K$L$AFIG59LAZS{Q95</span><span class="informations">1342×408 30 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
could you please give me some suggestions?<br>
Thanks, best wishes</p>

nateanl: <p>It seems related with your file name “torchaudio.py”. You can try to rename it and run it again.</p>

Gorgen: <p>Thank you for your suggestions.<br>
best wishes</p>
How to create a network with multipile inputs (and other modifications)?

Nimrod_Daniel: <p>Hi,</p>
<p>I want to create a network (inceptionv3 without the auxiliary, in this case) with multiple inputs, in which the number of inputs could be anywhere let’s say between 16-32 (I intend to feed it with a whole batch).<br>
It’s preferable to have an option to create a network that after it gets a variable (number of inputs, batch size, it’s the same in this case) it creates a network with that number of inputs.<br>
But if it’s easier to create one with a fixed input size (let’s say 16) then it’s ok too.<br>
Shared weights, of course.</p>
<p>Plus, after the last convolution layer, I need to concatenate all the outputs of the same batch and then feed it to a global average pooling layer (I guess I’d need to do mean over the 16 outputs and then feed it  Inception’s AdaptiveAveragePooling(1,1) layer (?)).<br>
The output of the global average pooling would be the new embeddings.<br>
From that point it should be as usual - FC layer (basically one output for regression, but a few outputs is optional for classification).</p>
<p>Notes:</p>
<ol>
<li>In training, each batch has one label (only the whole batch has a label, not each sample). That’s why I need to make these modifications</li>
<li>Basically same for inference. At inference all the images should get one prediction, even it’d be 100 batches, all would belong to one label. Later I’ll deal with slight modifications to the inference, now I’m focusing on the training (and specifically - the architecture.)</li>
<li>I know I’ll have to deal later with the sampler so it’d sample in accordance with what I want it to do. I have a custom dataset, and I think I know how to approach that.</li>
</ol>
<p>How do I do this modification to InceptionV3?</p>
<p>Thanks a lot!</p>
Input and output to the lstms

saran: <p>I want to implement lstms  with  CNN in pytorch   as my data is a time series data i.e. frames of  video for heart rate detection, I am struggling with the input and output  dimensions for lstms what and how i should properly configure  the dimensions/parameters/arguments  at input of lstms in pytorch as its quite confusing when considering time steps, hidden state etc.<br>
my output from CNN is “2 batches of 256 frames”, which is now the input to lstms<br>
batch is 2<br>
features =256</p>
Debugging Large Memory Usage

legoh: <p>Hi there, I’m currently working on a piece of code that had quite a few places of <code>torch.repeat_interleave()</code> to repeat data as part of the neural network. I’ve been trying to figure out where I can optimize the code, in terms of replacing these with <code>torch.expand</code>. However, I’m still facing the issue of out of memory. Is there a way to decipher which operations are using large amounts of memory? I’ve done my best in terms of using <code>torch.gather</code> to index and <code>torch.expand</code> to avoid allocating new memory for tensors. I suspect the issue might lie in the computational graph but I have no idea how to check which operations are causing the large memory use.</p>

ptrblck: <p>You could add print statements to your code and check the memory usage after some operations e.g. via <code>print(torch.cuda.memory_summary())</code> or <code>print(torch.cuda.memory_allocated())</code>.<br>
Using <code>expand</code> will not allocate new memory, but note that some operations might depend on contiguous data and could call <code>tensor.contiguous()</code> internally which will then allocate the memory.</p>
Module 'transforms' has no attribute 'RandomRotation'

Xiaoyu_Song: <p>I got this error using this version</p>
<p>Torch version 1.10.1<br>
Cuda 11.3</p>

ptrblck: <p>Which <code>torchvision</code> version did you install as the transformation is shipped in this package?<br>
You can check it with <code>torchvision.__version__</code>.</p>

Xiaoyu_Song: <p>HI <a class="mention" href="/u/ptrblck">@ptrblck</a> , torchvision: 0.11.2</p>

ptrblck: <p><code>torchvision==0.11.2</code> contains this transformation:</p>
<pre><code class="lang-python">&gt;&gt;&gt; import torchvision
&gt;&gt;&gt; torchvision.__version__
'0.11.2+cu102'
&gt;&gt;&gt; torchvision.transforms.RandomRotation
&lt;class 'torchvision.transforms.transforms.RandomRotation'&gt;
</code></pre>

Xiaoyu_Song: <p>This function is only useful to do image rotate. In pytorch exists rotation to bounding box as well?</p>
Storing batches form DataLoader in a Buffer/List for later training

qubehead: <p>Hello, I have a problem and I cannot figure out what I am doing wrong. I want to get samples from a pytorch DataLoader and store them in a buffer, so that I can use them for training later. The normal way to do a train-loop (without the buffer) would look something like this:</p>
<pre><code class="lang-python">for epoch in range(epochs):
  for data, target in data_loader:
      ... # loss, backprop etc
</code></pre>
<p>But instead, I want to do something like this</p>
<pre><code class="lang-python">buffer = []
for data, target in data_loader:
  buffer.append((data, target))

for epoch in range(epochs):
  for data, target in buffer:
      ... # loss, backprop etc
  random.shuffle(buffer)
</code></pre>
<p>As far as I understand, both methods should approximately produce about the same results. The second method converges, but it performs significantly worse than the first, without changing anything else in the code (except for adding the buffer, of course). I already experimented with cutting apart the batches and shuffling them around but nothing has worked so far. I would greatly appreciate any help, because I am struggling with this for a week. Thanks!</p>
<p>(PS: my real code is more complex, but it all boils down to this)</p>

ptrblck: <p>The main difference would be that the second approach would disable the data augmentation.<br>
I.e. if you are using random transformations in your <code>Dataset</code>, the first approach would apply them on each sample on-the-fly during the data loading and processing in <code>for data, target in data_loader</code>.<br>
In your second approach you’ve applied all (random) transformations once and stored this processed data. For the model, the samples are thus identical and no data augmentation is used, which can make the convergence worse.</p>
Installed version is 1.11.0 but torch.version is 1.7.0

leckofunny: <p>Hello everyone!</p>
<p>I installed PyTorch into a fresh Python 3.7 Anaconda environment using the provided command:</p>
<p><code>conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch</code></p>
<p>OS: Ubuntu 20.04<br>
nvidia Driver: 510.60.02<br>
CUDA: not installed</p>
<p>When I import torch and print <code>torch.__version__</code> it returns 1.7.0+cu101 and not 1.10.0.<br>
When I print torch.cuda.version it returns 10.1 and not 11.3.<br>
Running torch.cuda.is_avialable() returns True.</p>
<p>What’s going on with these version mismatches?<br>
Is this just a bug or am I reaching out to the wrong variables?<br>
I’m thankful for any clarifying insights!</p>

eqy: <p>You may want to see if there is another version of torch that is already installed and it is being selected first before the conda package (e.g., check the output of <code>torch.__file__</code> to see the location of the module that is being imported).</p>

leckofunny: <p>You are right! For some reason it referred to PyTorch to some python3.7 directory in home. Removing that directory resolved my issue. Thanks =)</p>
amp.GradScaler() and amp.autocast() question

Samuel_Bachorik: <p>What is amp.GradScaler() and amp.autocast() good for ?</p>
<p>When I use them in my training loop I don’t see any big difference. Can someone tell me when I should use these two ? Or how it should help.</p>

ptrblck: <p>The <a href="https://pytorch.org/tutorials/recipes/recipes/amp_recipe.html">docs on automatic mixed precision</a> are explaining both objects and their usage.<br>
TL;DR:</p>
<ul>
<li>
<code>autocast</code> will cast the data to <code>float16</code> (or <code>bfloat16</code> if specified) where possible to speed up your model and use TensorCores if available on your GPU</li>
<li>
<code>GradScaler</code> will prevent underflowing of small gradients, which could otherwise break your training</li>
</ul>
Converting from qF.conv2d to F.conv2d

sjlee1217: <p>Hello,<br>
I’m trying to implement quantized conv2d with F.conv2d function to port to embedded device. However, I couldn’t reproduce equal output when composed of F.conv2d. Is there any example to convert quantized conv2d to F.conv2d?</p>
<p>Dear best.</p>

andrewor: <p>Hi Sungjun,</p>
<p>Unfortunately I couldn’t find any tutorial with F.conv2d specifically, but you should be able to quantize it the same way as any other op. You can follow these tutorials (e.g. <a href="https://pytorch.org/tutorials/prototype/fx_graph_mode_ptq_static.html" rel="noopener nofollow ugc">fx</a>, <a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html" rel="noopener nofollow ugc">eager mode</a>) in general and replace the modules to quantize with F.conv2d. You can also check out <a>this test case</a> for an example of how to quantize a model with functional conv ops specifically.</p>
<p>Best,<br>
-Andrew</p>
Getting error by quantize_fx.prepare_fx

Rasoul_Zamani: <h1>
<a name="hi-for-quantization-of-yolov5-as-pytorch-doc-suggest-i-used-this-code-1" class="anchor" href="#hi-for-quantization-of-yolov5-as-pytorch-doc-suggest-i-used-this-code-1"></a>Hi, for quantization of YOLOv5 as pytorch doc suggest, I used this code:</h1>
<p>import torch<br>
model= torch.hub.load(‘ultralytics/yolov5’, ‘yolov5s’)</p>
<p>import torch.quantization.quantize_fx as quantize_fx<br>
import copy</p>
<h1>
<a name="post-training-static-quantization-2" class="anchor" href="#post-training-static-quantization-2"></a>post training static quantization</h1>
<p>model_to_quantize = copy.deepcopy(model)<br>
qconfig_dict = {"": torch.quantization.get_default_qconfig(‘qnnpack’)}<br>
model_to_quantize.eval()</p>
<p>model_prepared = quantize_fx.prepare_fx(model_to_quantize, qconfig_dict)</p>
<p>model_quantized = quantize_fx.convert_fx(model_prepared)</p>
<p>but this error was raised:</p>
<p>TraceError                                Traceback (most recent call last)<br>
 in ()<br>
7 model_to_quantize.eval()<br>
8 # prepare<br>
----&gt; 9 model_prepared = quantize_fx.prepare_fx(model_to_quantize, qconfig_dict)<br>
10 # calibrate (not shown)<br>
11 # quantize</p>
<p>6 frames<br>
/usr/local/lib/python3.7/dist-packages/torch/fx/proxy.py in to_bool(self, obj)<br>
150         information to the graph node using create_node and can choose to return a value.<br>
151         “”"<br>
 → 152         raise TraceError(‘symbolically traced variables cannot be used as inputs to control flow’)<br>
153<br>
154     <span class="mention">@compatibility</span>(is_backward_compatible=True)</p>
<p>TraceError: symbolically traced variables cannot be used as inputs to control flow</p>
<h1>
<a name="whats-the-problem-i-write-this-code-in-colab-3" class="anchor" href="#whats-the-problem-i-write-this-code-in-colab-3"></a>what’s the problem? (I write this code in colab)</h1>

andrewor: <p>Hi Rasoul,</p>
<p>It would seem that the model uses dynamic control flow, which is currently not supported in FX in general. For more detail, see <a href="https://pytorch.org/docs/stable/fx.html#dynamic-control-flow" class="inline-onebox" rel="noopener nofollow ugc">torch.fx — PyTorch 1.11.0 documentation</a>. Unfortunately, this means the only way to quantize this model is to use the eager mode API at the moment. Here is an example of how to do that: <a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">(beta) Static Quantization with Eager Mode in PyTorch — PyTorch Tutorials 1.11.0+cu102 documentation</a></p>
<p>Best,<br>
-Andrew</p>
Gradient of Image in PyTorch - for Gradient Penalty calculation in WGAN

Liz_Sampat: <p>I am following this <a href="https://github.com/amanchadha/coursera-gan-specialization/blob/main/C1%20-%20Build%20Basic%20Generative%20Adversarial%20Networks/Week%203/C1W3_WGAN_GP.ipynb" rel="noopener nofollow ugc">Github Repo</a> for the WGAN implementation with Gradient Penalty.</p>
<p>And I am trying to understand the following method, which does the job of unit-testing the gradient-penalty calulations.</p>
<pre data-code-wrap="py"><code class="lang-nohighlight">def test_gradient_penalty(image_shape):
    bad_gradient = torch.zeros(*image_shape)
    bad_gradient_penalty = gradient_penalty(bad_gradient)
    assert torch.isclose(bad_gradient_penalty, torch.tensor(1.))

    image_size = torch.prod(torch.Tensor(image_shape[1:]))
    good_gradient = torch.ones(*image_shape) / torch.sqrt(image_size)
    good_gradient_penalty = gradient_penalty(good_gradient)
    assert torch.isclose(good_gradient_penalty, torch.tensor(0.))

    random_gradient = test_get_gradient(image_shape)
    random_gradient_penalty = gradient_penalty(random_gradient)
    assert torch.abs(random_gradient_penalty - 1) &lt; 0.1

# Now pass tuple argument for image dimenstion of 
# (batch_size, channel, height, width)
test_gradient_penalty((256, 1, 28, 28))
</code></pre>
<h3>
<a name="i-dont-understand-the-below-line-1" class="anchor" href="#i-dont-understand-the-below-line-1"></a>I don’t understand the below line</h3>
<p><code>good_gradient = torch.ones(*image_shape) / torch.sqrt(image_size)</code></p>
<p>In above the <code>torch.ones(*image_shape)</code> is just filling a 4-D Tensor filled up with 1 and then<br>
<code>torch.sqrt(image_size)</code> is just representing the value of <code>tensor(28.)</code></p>
<h3>
<a name="so-what-i-am-trying-to-understand-why-i-need-to-divide-the-4-d-tensor-by-tensor28-to-get-the-good_gradient-2" class="anchor" href="#so-what-i-am-trying-to-understand-why-i-need-to-divide-the-4-d-tensor-by-tensor28-to-get-the-good_gradient-2"></a>So, what I am trying to understand why I need to divide the 4-D Tensor by <code>tensor(28.)</code> to get the <code>good_gradient</code>
</h3>
<p>If I print bad_gradient, it will be a 4-D Tensor as below</p>
<pre><code class="lang-auto">tensor([[[[0., 0., 0.,  ..., 0., 0., 0.],
          [0., 0., 0.,  ..., 0., 0., 0.],
          [0., 0., 0.,  ..., 0., 0., 0.],
          ...,
          [0., 0., 0.,  ..., 0., 0., 0.],
          [0., 0., 0.,  ..., 0., 0., 0.],
          [0., 0., 0.,  ..., 0., 0., 0.]]],

          ---
          ---
</code></pre>
<p>If I print <code>good_gradient</code>, the output will be</p>
<pre><code class="lang-auto">tensor([[[[0.0357, 0.0357, 0.0357,  ..., 0.0357, 0.0357, 0.0357],
          [0.0357, 0.0357, 0.0357,  ..., 0.0357, 0.0357, 0.0357],
          [0.0357, 0.0357, 0.0357,  ..., 0.0357, 0.0357, 0.0357],
          ...,
          [0.0357, 0.0357, 0.0357,  ..., 0.0357, 0.0357, 0.0357],
          [0.0357, 0.0357, 0.0357,  ..., 0.0357, 0.0357, 0.0357],
          [0.0357, 0.0357, 0.0357,  ..., 0.0357, 0.0357, 0.0357]]],

          ---
          ---
</code></pre>
How do i place my data when using DDP?

Mr.Crab: <p>i want to use DDP with multiple computers. And i know that the “torch.utils.data.distributed.DistributedSampler” can random load the dataset. The question is that i should palce the same dataset at each computers?  Can I assign a specified part of the dataset to each computer in order to release the storage pressure?</p>

fduwjj: <p>DistributedSampler should work well with DDP. And I believe split storage should be fine. Have you tried that?</p>

fduwjj: <p>cc Dataloader folks to confirm: <a class="mention" href="/u/vitalyfedyunin">@VitalyFedyunin</a></p>
Speeding up matrix multiplication like operations

sachinruk: <p>Suppose I have the following two matrices:</p>
<pre><code class="lang-python">x = torch.randint(0, 256, (100000, 48), dtype=torch.uint8)
x_new = torch.randint(0, 256, (1000, 48), dtype=torch.uint8)
</code></pre>
<p>I wish to do a matrix multiplication like operation where I compare the 48 dimensions and sum up all the elements that are equal. The following operation takes 7.81 seconds. Batching does not seem to help:</p>
<pre><code class="lang-auto">matrix = (x_new.unsqueeze(1) == x).sum(dim=-1)
</code></pre>
<p>However, doing a simple matrix multiplication (<code>matrix = x_new @ x.T</code>) takes 3.54 seconds. I understand this is most likely calling a deeper library that isn’t slowed down by python. However, the question is, <strong>is there a way to speed up the multiplication like operation?</strong> by using scripting, or any other way at all?</p>
<p>What is even stranger though is that if I do <code>matrix = x_new.float() @ x.float().T</code> this operation takes 214ms. This is more than 10x faster than the <code>uint8</code> multiplication. To add to the strangeness, uint8 is a single byte whereas float is 4 bytes, so I would have expected a speed up.</p>
<p>For context, I am trying to quantize vectors so that I can find the closest vector by comparing integers than directly doing dot products.</p>
Custom Dataset AttributeError:

ankita.buntolia: <p>I have created a custom Dataset but it is throwing AttributeError.</p>
<p><strong>Error Logs</strong></p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
&lt;ipython-input-20-d59eb5edbfee&gt; in &lt;module&gt;()
----&gt; 1 print(len(mydataset))
      2 print(mydataset[2])

1 frames
&lt;ipython-input-17-15b8607644ab&gt; in __getattr__(self, attribute_name)
     18         return function
     19     else:
---&gt; 20         raise AttributeError
     21 
     22   def __getitem__(self, idx):

AttributeError:
</code></pre>
<p><strong>My DataSet:</strong></p>
<pre><code class="lang-auto">
class OCTDataset(Dataset):
  "OCT Scan data set"
  def __init__(self, images, references, transform = None):
    super(OCTDataset, self).__init__()
    self.images = images
    self.references = references
    self.transform = transform

  def __len__(self):
      return len(self.img)


  def __getitem__(self, idx):
    img = self.images[idx]
    label = self.references[idx]

    data_dict = {'image': img, 'label': label}

    if self.transform:
        data_dict = self.transform(data_dict)   

    return data_dict
</code></pre>
<p><strong>Here I am initiating it, it does without any error.</strong></p>
<p><code>mydataset = OCTDataset(train_oct_images, train_labels)</code></p>
<p>But when I am running this code to test, it is throwing an error:</p>
<pre><code class="lang-auto">print(len(mydataset))
print(mydataset[2])
</code></pre>

nivek: <aside class="quote no-group" data-username="ankita.buntolia" data-post="1" data-topic="147854">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/bc79bd/48.png" class="avatar"> ankita.buntolia:</div>
<blockquote>
<pre><code class="lang-auto">  def __len__(self):
      return len(self.img)
</code></pre>
</blockquote>
</aside>
<p>In your <code>__len__</code> method, it should be <code>len(self.images)</code> instead of <code>len(self.img)</code>.</p>
How is topk selection implemented in torch?

ccoa: <p>I’m using torch.topk and I would like to find information about how top k selection is implemented in a differentiable way (with respect to the top k values).</p>
<p>Any information (also just a description in words or pseudocode) is most welcome.</p>

Varal7: <p>During forward, it will save the indices that form the top k values. Then for backward, it will pass the gradient through those indices only.</p>
<p>Code for forward:</p>
<pre><code class="lang-auto">std::tuple&lt;at::Tensor,at::Tensor&gt; topk(c10::DispatchKeySet ks, const at::Tensor &amp; self, int64_t k, int64_t dim, bool largest, bool sorted) {
  auto&amp; self_ = unpack(self, "self", 0);
  auto _any_requires_grad = compute_requires_grad( self );
  (void)_any_requires_grad;
  std::shared_ptr&lt;TopkBackward&gt; grad_fn;
  if (_any_requires_grad) {
    grad_fn = std::shared_ptr&lt;TopkBackward&gt;(new TopkBackward(), deleteNode);
    grad_fn-&gt;set_next_edges(collect_next_edges( self ));
    grad_fn-&gt;self_sizes = self.sizes().vec();
    grad_fn-&gt;dim = dim;
  }
  at::Tensor values;
  at::Tensor indices;
  auto _tmp = ([&amp;]() {
    at::AutoDispatchBelowADInplaceOrView guard;
    return at::redispatch::topk(ks &amp; c10::after_autograd_keyset, self_, k, dim, largest, sorted);
  })();
  std::tie(values, indices) = std::move(_tmp);
  if (grad_fn) {
      set_history(flatten_tensor_args( values ), grad_fn);
  }
  throw_error_for_complex_autograd(values, "topk");
  TORCH_CHECK(!(isFwGradDefined(self)), "Trying to use forward AD with topk that does not support it.");
  if (grad_fn) {
    grad_fn-&gt;indices_ = SavedVariable(indices, true);
  }
  return std::make_tuple(std::move(values), std::move(indices));
}
</code></pre>
<p>Code for backward</p>
<pre><code class="lang-auto">variable_list TopkBackward::apply(variable_list&amp;&amp; grads) {
  std::lock_guard&lt;std::mutex&gt; lock(mutex_);

  IndexRangeGenerator gen;
  auto self_ix = gen.range(1);
  variable_list grad_inputs(gen.size());
  auto&amp; grad = grads[0];
  auto indices = indices_.unpack(shared_from_this());
  bool any_grad_defined = any_variable_defined(grads);
  if (should_compute_output({ self_ix })) {
    auto grad_result = any_grad_defined ? (value_selecting_reduction_backward(grad, dim, indices, self_sizes, true)) : Tensor();
    copy_range(grad_inputs, self_ix, grad_result);
  }
  return grad_inputs;
}
</code></pre>
CUDNN_STATUS_MAPPING_ERROR runtime error when using multiple CUDA contexts

AmirFB: <p>When I use the cuda-python library to create multiple contexts, I get the CUDNN_STATUS_MAPPING_ERROR runtime error. Here is the code to reproduce this error:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from cuda import cuda

affinity = cuda.CUexecAffinityParam()
affinity.type = cuda.CUexecAffinityType.CU_EXEC_AFFINITY_TYPE_SM_COUNT
affinity.param.smCount.val = 10
ctx = cuda.cuCtxCreate_v3([affinity], 1, 0, 0)
in1 = torch.ones(5000, 30, 10, 5, device="cuda")#.share_memory_()
conv1 = nn.Conv2d(30, 5, 3, stride=1, padding=1, device='cuda')
mod1 = MyModule("mod1", in1).cuda()
th1 = ReusableThread(mod1, 2)
temp = conv1(in1)

affinity = cuda.CUexecAffinityParam()
affinity.type = cuda.CUexecAffinityType.CU_EXEC_AFFINITY_TYPE_SM_COUNT
affinity.param.smCount.val = 40
ctx = cuda.cuCtxCreate_v3([affinity], 1, 0, 0)
res = cuda.cuCtxSetCurrent(ctx[1])
in2 = torch.ones(5000, 30, 10, 5, device="cuda")#.share_memory_()
conv2 = nn.Conv2d(30, 5, 3, stride=1, padding=1, device='cuda')
mod2 = MyModule("mod2", in2).cuda()
th2 = ReusableThread(mod2, 68)
temp = conv2(in2)
</code></pre>
<p>I get the error on the last line. Using raw CUDA I can use multiple different contexts in a single process without any problems but I get this error with PyTorch. (Remember you need to enable MPS with <code>nvidia-cuda-mps-control -d</code> first and it’s only available in Linux)</p>
Question: Why module 'openimages' has no attribute 'download'?

defa: <p>Hello,<br>
this topic is just a question about something I want to understand, because I don’t know the reason for it.</p>
<p>So, I installed openimages with pip. When I try to use openimages like that:</p>
<pre><code class="lang-auto">import openimages 
openimages.download.download_dataset("example",['castle'])
</code></pre>
<p>I’m getting these error message:</p>
<pre><code class="lang-auto">AttributeError: module 'openimages' has no attribute 'download'
</code></pre>
<p>.<br>
But when I change the lines to:</p>
<pre><code class="lang-auto">from openimages.download import download_dataset
download_dataset(dest_dir="example", ['castle'])
</code></pre>
<p>then it works.</p>
<p>So can just someone explain, why?<br>
Thanks <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

Andrei_Cristea: <p>Hello defa,</p>
<p>It has to do with how Python deals with package imports. Have a look at <a href="https://realpython.com/python-modules-packages/#package-initialization" rel="noopener nofollow ugc">this link</a>.</p>
<p>In brief, I think it’s because the <code>__init__.py</code> file inside the <code>openimages</code> package does not include a command to import the <code>download</code> module.</p>
<p>Hope this helps!<br>
Andrei</p>
Apply a method to specific dimension while tensor is on CUDA

mkorycinski: <p>Hi,</p>
<p>I am playing around with creating a custom layer for training. At some point on this path, I ended up with a tensor of shape <code>(N x M x H)</code>. I would like to apply math operations to the last dimension of this tensor (for the sake of simplicity let’s say a sum of elements) to end up with a tensor of shape <code>(N x M)</code>. I found out that the <code>apply</code> function would do something like that but it will apparently not work on CUDA.</p>
<p>Is there a way to do something like this in an elegant and efficient way?</p>

suraj.pt: <p>Hi, how are you using <code>apply</code> for this? If available, I would just pass the <code>dim</code> arg in whatever math operation you’re calling on your tensor, e.g.: <code>a.sum(dim=2)</code></p>

mkorycinski: <p>That’s obviously an option, however a method I want to call on this dimension is more complex and written by me. It just requires a single dimension tensor with values and returns a single value as a result. Something like this:</p>
<pre><code class="lang-auto">def sth(values: torch.Tensor):
    return values.sum()
</code></pre>
<p>Best regards,<br>
Mateusz</p>
<p>Edit:</p>
<p>I think I managed to do it that way:</p>
<pre><code class="lang-python">def sth(values: torch.Tensor) -&gt; float:
    """ Exemplary function, I am aware of a pytorch built-in sum ;-) """
    return sum(values)

torch.stack(
    tuple(
        torch.stack(
          tuple(
              map(sth, elem.unbind(0)))
        ) 
        for elem in a
    )
)
</code></pre>
<p>However, it is super slow.</p>

suraj.pt: <p>Excuse my assumption <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"><br>
It’s hard to say without knowing more about your custom function; you might try using <code>torch.einsum</code> if it’s a matrix operation?</p>
<p>EDIT: <a class="mention" href="/u/mkorycinski">@mkorycinski</a> You also might be able to vectorize your custom function; that will certainly be more performant. Feel free to share the function if you’d like help with the vectorized implementation</p>
How to calculate precision for tensors of different sizes during detection?

parvector: <p>I train SSDLite320 from the detection module in pytorch and the model gives me predicted labels of length n, and the targetets have length k. But I get the following error:<br>
ValueError: Either <code>preds</code> and <code>target</code> should both have the (same) shape (N, …), or <code>target</code> should be (N, …) and <code>preds</code> should be (N, C, …).<br>
What to do to fix it? Thank you in advance. I am using torchmetrics.</p>
Suggession regarding a GPU to use for multi class sementic segmentation!

Alok1: <p>Hey guys!</p>
<p>I hope you all are doing well. I am currently stuck in a position where I have to make a decision on which GPU I must buy for a multiclass image segmentation problem. I never had to choose between a wide variety of GPUs until today for training a network. However, there is a GPU called <strong>Nvidia RTX A4000 16GB</strong> which perfectly fits my budget, but I am now unsure if this would handle the workload. Therefore, I would like to get a piece of advice from professionals or from people who might have used this GPU, if it is good for this problem statement?</p>
<p>CNN Networks that I will be exploring are UNet, ResNet and AlexNet.</p>
<p>Any suggestion/recommendation would be highly appreciated.</p>
<p>Thank you in advance and best regards,</p>