akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Are jit.script & prune.ln_structured incompatible?

username: Thytu
cooked: <p>Hello everyone,</p>
<p>I would like to prune my model and then script it but I receive the following error: <code>AttributeError: 'LnStructured' object has no attribute '__name__'</code></p>
<p>Can we prune a model and then script it?</p>
<p>I see that <code>LnStructured</code> actually add a forward pre_hook but <code>jit.script</code> can’t resolve its name.</p>
<p>cf:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "src/model_optimizer.py", line 226, in &lt;module&gt;
    script_module(pruned_model)
  File "src/model_optimizer.py", line 20, in script_module
    module = jit.script(module)
  File "/venv/lib/python3.8/site-packages/torch/jit/_script.py", line 1257, in script
    return torch.jit._recursive.create_script_module(
  File "/venv/lib/python3.8/site-packages/torch/jit/_recursive.py", line 451, in create_script_module
    return create_script_module_impl(nn_module, concrete_type, stubs_fn)
  File "/venv/lib/python3.8/site-packages/torch/jit/_recursive.py", line 513, in create_script_module_impl
    script_module = torch.jit.RecursiveScriptModule._construct(cpp_module, init_fn)
  File "/venv/lib/python3.8/site-packages/torch/jit/_script.py", line 587, in _construct
    init_fn(script_module)
  File "/venv/lib/python3.8/site-packages/torch/jit/_recursive.py", line 491, in init_fn
    scripted = create_script_module_impl(orig_value, sub_concrete_type, stubs_fn)
  File "/venv/lib/python3.8/site-packages/torch/jit/_recursive.py", line 513, in create_script_module_impl
    script_module = torch.jit.RecursiveScriptModule._construct(cpp_module, init_fn)
  File "/venv/lib/python3.8/site-packages/torch/jit/_script.py", line 587, in _construct
    init_fn(script_module)
  File "/venv/lib/python3.8/site-packages/torch/jit/_recursive.py", line 491, in init_fn
    scripted = create_script_module_impl(orig_value, sub_concrete_type, stubs_fn)
  File "/venv/lib/python3.8/site-packages/torch/jit/_recursive.py", line 465, in create_script_module_impl
    hook_stubs, pre_hook_stubs = get_hook_stubs(nn_module)
  File "/venv/lib/python3.8/site-packages/torch/jit/_recursive.py", line 758, in get_hook_stubs
    if pre_hook.__name__ in hook_map:
AttributeError: 'LnStructured' object has no attribute '__name__'
</code></pre>
<p>Here is the code to reproduce the error :</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.utils.prune as prune
import torch.jit as jit

class Model(nn.Module):
    def __init__(self):
        super().__init__()

        self.layers = nn.Sequential(
            nn.Linear(10, 10),
            nn.ReLU(),

            nn.Linear(10, 10),
            nn.Sigmoid(),
        )

    def forward(self, t):
        return self.layers(t)

model = Model()

for _, module in model.named_modules():
    if isinstance(module, torch.nn.Linear):
        prune.ln_structured(module, name='weight', amount=0.4, dim=1, n=float('-inf'))

jit.script(model)
</code></pre>
<p>Thanks,<br>
Thytu</p>
Summay library not reacting properly to Heteregeneous model definintion

username: yuri
cooked: <pre><code class="lang-auto">class Model(nn.Module):
    def __init__(self, channels):
        super(Model, self).__init__()
        self.conv11 = nn.Conv2d(in_channel, out_channel_1, kernel_size=(5, 5), bias=True, stride=2), padding='same')
        self.batch_Norm11 = nn.BatchNorm2d(out_channel_1, momentum=momentum)
        self.max_pooling1 = nn.MaxPool2d(2, stride=2)
        if some_condition:
            self.bn = nn.BatchNorm2d(n_joints+1, affine=True, momentum=0.05) 
            for joint in joint_names:
                for cond_joint in joint_dependence[joint]:
                    joint_key = joint + '_' + cond_joint
                    self.shapes = torch.nn.ModuleDict(
                    {f"{joint_key}": torch.nn.Linear(pairwise_distr_np[joint_key].shape[0], pairwise_distr_np[joint_key].shape[1], bias=False) for joint_key in joint_keys})
</code></pre>
<p>as you may notice here that we defined our model with two different approaches in the beginning sequentially and then with Module dict.<br>
<strong>once we run the summary of the model only the sequential layers were shown and not the nn.Linear layers inside the Module dict or even the Moduledict as a whole</strong>. here you can see the output of the summary function.<br>
<strong>does anyone also had experience with recursive as Param #</strong><br>
thank you<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/7/67557f93bec186852d28735b484481749c4fde98.png" data-download-href="https://discuss.pytorch.org/uploads/default/67557f93bec186852d28735b484481749c4fde98" title="Screenshot (11)"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/67557f93bec186852d28735b484481749c4fde98_2_365x500.png" alt="Screenshot (11)" data-base62-sha1="eK8mfjpYN70QvWyXUoNrZU57sCs" width="365" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/67557f93bec186852d28735b484481749c4fde98_2_365x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/67557f93bec186852d28735b484481749c4fde98_2_547x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/67557f93bec186852d28735b484481749c4fde98_2_730x1000.png 2x" data-dominant-color="323233"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot (11)</span><span class="informations">845×1157 136 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
How can I optimize this calculation?

username: Yosin
cooked: <p>I have the following toy code:</p>
<p>-modeField = torch.randn([10,64,64])<br>
-paras = torch.randn([batchSize, 10])<br>
-for bat in batchSize:<br>
-----result = 0<br>
-----for i in range(10):<br>
--------res += modeField[i] * paras[bat][i]<br>
-----loss = (res**2 - speckle)**2</p>
<p>This is taking really long. Is there any way, to do some batch multiplication to get the imediate result for all batches and paras, instead of doing it with for loops?</p>
<p>I’ve looked into matmul and bmm, but I dont really understand them and it seems like they dont multiplicate a whole array with only a single value.</p>
<p>I guess I could map the parameter value on an array of the same size as modeField but that would’nt be a very elegant solution.</p>

username: Eta_C
cooked: <pre><code class="lang-python">import torch 

batchSize = 3
modeFiled = torch.randn([10, 64, 64])
paras = torch.randn([batchSize, 10])

# your method
your_method_result = []
for bat in range(batchSize):
    res = 0 
    for i in range(10):
        res += modeField[i] * paras[bat][i]
    your_method_result.append(res)
your_method_result = torch.stack(your_method_result, dim=0)

# faster implementation
result = torch.matmul(paras, modeField.view(10, -1)).view(batchSize, 64, 64)

# show differents, it should be 1e-6
print((your_method_result - result).abs().max())
</code></pre>

username: Yosin
cooked: <p>It works perfectly. Thank you.</p>
Visualize misclassified and correct predicted samples together

username: Angelina_Robert
cooked: <p>Hi,<br>
Just wondering if anyone could guide, how to plot the  misclassified samples (outliers) along with the original feature space using tsne</p>

username: tom
cooked: <p>So T-SNE (just use .cpu().numpy() on the tensors and feed into sklearn’s TSNE) has the property that you need to provide all datapoints you want to project into a lower-dimensional space at fitting time.<br>
So you would need to concatenate the samples you get, call fit_transform to get the projected points. These will be in the same order, so you can take apart the array into the various types of points for visualzation.</p>
<p>More recent approaches such as Isomap from sklearn or <a href="https://umap.scikit-tda.org/transform.html" rel="noopener nofollow ugc">UMAP</a> allow you to fit a model first and then pass additional data through it, though in the case of outliers, this might not be as neat.</p>
<p>Best regards</p>
<p>Thomas</p>

username: Angelina_Robert
cooked: <p>Thanks very much, Thomas for the reply. I have plotted all the samples along with the misclassified samples, however, misclassified samples do not appear within the feature space, they are spread all around the feature space.</p>
Connecting Loss Based on Output of Other Function

username: J_Johnson
cooked: <p>This is a general AI question that I wanted to see if it has been addressed at any point.</p>
<p>Suppose we have some function A that has variables: x, y and z. Then we have a separate ML model B that takes as inputs the previous x, y, z and loss of function A, and then gives as output factors a, b, and c(i.e. 1.001, 0.998, 1.004) to multiply by x, y and z, with the goal of lowering the loss of function A. That new modified x, y and z then being inserted into function A to then get the new loss of function A.</p>
<p>Questions:</p>
<ol>
<li>How would you connect the new loss of function A to determine the loss of model B?</li>
<li>Would it be possible to just simply insert/replace the new loss of function A as the loss of model B? How would that be done?</li>
</ol>
How does pytorch model inference/training choose cudaStreams internally

username: BruceDai003
cooked: <p>As I use nvprof or pytorch profiler to profile my simple model, e.g., just resnet50. I can see that it uses multiple streams. However, the number of streams are different when I use data in fp32 mode and in fp16 mode:<br>
e.g.:</p>
<pre><code class="lang-auto">                for i in trange(len(dataloader)):
                    data, *_ = next(data_iter)
                # for data, *_ in dataloader:
                    if use_half:
                        data = data.type(torch.float16).to(args.device)
                    else:
                        data = data.to(args.device)

                    log_probs = model(data)
</code></pre>
<p>So, I am curious, if I build a simple model, like a sequential (Conv * 5 + MaxPool + Linear) model, or a resnet50 model. Without specifying any torch.cuda.Stream(), how did pytorch assigning the kernels to different cuda streams?<br>
I checked the cuda streams related C++ code, as in torch/csrc/cuda/Stream.cpp, c10/cuda/CUDAStream.cpp, c10/core/Stream.h. These are the interfaces to cudaStream_t and how to get the streams by fetching it from stream pools. But I’m missing the linking mechanism part of how pytorch use the cuda streams by default. Could you shed some light on this, thanks in advance?</p>
<p>[Edit]: I just checked, I think most cuda operators if not calling cudnn or cublas, that is to say, implemented in pytorch directly, as found under aten/src/ATen/native/cuda path, many of the operators when launching cuda kernels, will pass the current cuda stream as in</p>
<pre><code class="lang-auto">cudaStream_t stream = at::cuda::getCurrentCUDAStream();
</code></pre>
<p>For other operators call into cudnn or cublas, etc. I cannot find the cuda stream parameter passing into the calling interface, e.g. convolution as in aten/src/ATen/native/cudnn/Conv_v7.cpp:</p>
<pre><code class="lang-auto">AT_CUDNN_CHECK_WITH_SHAPES(cudnnConvolutionForward(

          args.handle,

          &amp;one, args.idesc.desc(), input.data_ptr(),

          args.wdesc.desc(), weight.data_ptr(),

          args.cdesc.desc(), fwdAlgPerf.algo, workspace.data_ptr(), fwdAlgPerf.memory,

          &amp;zero, args.odesc.desc(), output.data_ptr()),

        args, "Forward algorithm: ", static_cast&lt;int&gt;(fwdAlgPerf.algo), "\n");

      }
</code></pre>
<p>So, I guess cudnn library will manage the cudaStreams from there on(, but unfortunately we cannot see the source code <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=10" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"> ). Am I right?<br>
So, it might be that the libcudnn.so implemented fp16 which might use TensorCores in different algo than fp32, so it might use different cuda streams mechanisms.</p>
I am new here ,will i get some help from you?

username: YHJ
cooked: <p>i am new here ,please give me advices that help me better</p>

username: ptrblck
cooked: <p>Welcome!<br>
You can directly post your question in the forum and I’m sure users are happy to help you out.<br>
In case you are hitting an error using PyTorch, a minimal and executable code snippet would be great as it’s the best way to debug the issue and help you in a timely manner. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
RuntimeError: ones needs to be contiguous (even though I put contiguous on everything)

username: meowmix
cooked: <p>You get the error in the title if you run the below code, which I believe should not happen. The step that “causes” the error is y.transpose(1,2) – for some reason pytorch doesn’t like the 1st and 2nd dimension being transposed here. but the underlying issue appears to be with how the bias is being added (if I do bias = False the error goes away.)</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
k = 2
w,h,d = 1, 1, 1
x = torch.zeros((10, 1, w, h, d))
c = torch.nn.ConvTranspose3d(1, 2, k)
y = c(x).contiguous()
y = y.reshape(1, 10, 2, k*w, k*h, k*d).contiguous()
y = y.transpose(1, 2).contiguous()
hub = nn.SmoothL1Loss()
l = hub(y, torch.zeros(y.shape))
l.backward()
</code></pre>

username: tom
cooked: <p>Hi,</p>
<p>you found a bug, and one, has been around for a long time (credit to <a class="mention" href="/u/ptrblck">@ptrblck</a> who tested it all the way to PyTorch 1.4). Thank you so much for reporting it here with a minimal example, these are gold!<br>
The CPU conv3d transpose backward expects the gradient to be contiguous and as the transpose’s backward is the same transpose on the gradient, it is not.</p>
<p>Here is a workaround using an autograd function that makes the gradient contiguous:</p>
<pre><code class="lang-python">import torch
import torch.nn as nn

class ContiguousGrad(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x):
        return x
    @staticmethod
    def backward(ctx, grad_out):
        return grad_out.contiguous()


k = 2
w,h,d = 1, 1, 1
x = torch.zeros((10, 1, w, h, d))
c = torch.nn.ConvTranspose3d(1, 2, k)
y = c(x)
y = ContiguousGrad.apply(y)

y = y.reshape(1, 10, 2, k*w, k*h, k*d)
y = y.transpose(1, 2)
hub = nn.SmoothL1Loss()
l = hub(y, torch.zeros(y.shape))
l.backward()
</code></pre>
<p>The real fix will be adding contiguous to the backward, but that will be in PyTorch 1.11, likely.</p>
<p>Best regards</p>
<p>Thomas</p>

username: tom
cooked: <p>As a followup: I submitted a fix in <a href="https://github.com/pytorch/pytorch/pull/67829" class="inline-onebox" rel="noopener nofollow ugc">Fix conv_transpose3d backward with non-contiguous grad_out by t-vi · Pull Request #67829 · pytorch/pytorch · GitHub</a> . It should show up in the nightly builds shortly after it will be merged.</p>
<p>Again, thank you for submitting this with an easy test-case.</p>
<p>Best regards</p>
<p>Thomas</p>
Combining multiple predictions into one

username: person22
cooked: <p>I’m trying to predict if a player is cheating a FPS-game. The type of cheating I’m trying to catch is “aimbots” meaning a player that is using some external assistance to do the aiming for the player. My data consists of mouse movement information of shape (128,5), (128 timesteps and 5 features) of every kill the player got during the game.</p>
<p>So far, I’ve been using a GRU and feeding the individual kills into the model. The problem is that the model considers the kills independently of each other. If the data for one player, for the entire game, would be in the shape (number of kills, 128, 5), would it be preferable to concatenate the data into a big 2D array of shape 128,5*number of kills or keeping it and treating it like an image where the number of kills are the channels? On average players get around 50 kills per game.</p>
<p>Thank You!</p>

username: ptrblck
cooked: <p>Since the “number of kills” would be the variable dimension, I would probably not put it in the channel dimension which is usually fixed. I don’t know which approach would work the best, but I would probably try to treat each sequence as an individual sample and mix different players together first (as well as shuffling them) as this seems to be the easiest approach.<br>
Depending on the outcome you could (and should) also try out your other approach.</p>
How to save Pyrorch Model Configurations?

username: SalmanLeo93
cooked: <p>How can i save the pytorch model configuration in config.yml file?<br>
I need output as shown in attached photo.<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/2/b2a172caf67dedb88206d7a5dbe5f880596857e9.png" data-download-href="https://discuss.pytorch.org/uploads/default/b2a172caf67dedb88206d7a5dbe5f880596857e9" title="config"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/2/b2a172caf67dedb88206d7a5dbe5f880596857e9_2_493x500.png" alt="config" data-base62-sha1="pueTgovraZo68zzWaFXTQXkcS4p" width="493" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/2/b2a172caf67dedb88206d7a5dbe5f880596857e9_2_493x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/2/b2a172caf67dedb88206d7a5dbe5f880596857e9_2_739x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/2/b2a172caf67dedb88206d7a5dbe5f880596857e9.png 2x" data-dominant-color="FAF6F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">config</span><span class="informations">964×977 136 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Installing Pytorch on IBM Power 9 architecture

username: bing
cooked: <p>Hi,<br>
I am trying to install pytorch on IBM Power 9 architecture but it is taking obnoxiously a long time(~2 hours). It is stuck at below step -<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/f/5f72d7d4e6e7b6a09c13ff48029a855b14d7df41.png" alt="image" data-base62-sha1="dCnppAFqWFB4d5HguA6NA7PqkAV" width="522" height="96"></p>
<p>I am following <a href="https://gauravm.gitbook.io/about/blogs/installing-pytorch-and-transformers-on-ibm-powerpc-architecture" rel="noopener nofollow ugc">this</a> path steps.</p>
<p>Is it a normal time to get it install on power 9 architectures? Is there any other way?<br>
Below are further details of the system I am working on -<br>
<strong>Cuda version</strong><br>
lrwxrwxrwx.  1 root root    9 Jul 26  2019 cuda → cuda-10.1<br>
drwxr-xr-x. 14 root root  254 Jul 26  2019 cuda-10.1<br>
drwxr-xr-x   7 root root  131 May  4  2020 cuda-10.2</p>
<p>OS NAME=“Red Hat Enterprise Linux Server”<br>
VERSION=“7.6 (Maipo)”</p>
<p><code>uname -r</code></p>
<blockquote>
<p>4.14.0-115.el7a.ppc64le</p>
</blockquote>

username: ptrblck
cooked: <p>I don’t know what the current support of Power 9 in the prebuilt binaries is, but maybe a source build would be faster? Did you take a look at <a href="https://github.com/pytorch/pytorch#from-source">these build instructions</a>?</p>
AttributeError: 'torch.dtype' object has no attribute 'type'

username: dieptran149
cooked: <p>Hi all,<br>
I’ve run my code and I got the Error <code>AttributeError: 'torch.dtype' object has no attribute 'type'</code>. Through this following picture, I think np.mean can’t run on a list of tensor and that cause of my crashed code.<br>
Could anyone tell me how to tackle this error?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/0/70de278e6a09e40c9d14c12a2e9c3e26f58bd8a3.png" data-download-href="https://discuss.pytorch.org/uploads/default/70de278e6a09e40c9d14c12a2e9c3e26f58bd8a3" title="Screenshot from 2021-11-03 23-37-04"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/70de278e6a09e40c9d14c12a2e9c3e26f58bd8a3_2_690x316.png" alt="Screenshot from 2021-11-03 23-37-04" data-base62-sha1="g6trjsNecBv2RlVqJzHmTjDJnEL" width="690" height="316" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/70de278e6a09e40c9d14c12a2e9c3e26f58bd8a3_2_690x316.png, https://discuss.pytorch.org/uploads/default/original/3X/7/0/70de278e6a09e40c9d14c12a2e9c3e26f58bd8a3.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/0/70de278e6a09e40c9d14c12a2e9c3e26f58bd8a3.png 2x" data-dominant-color="632537"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2021-11-03 23-37-04</span><span class="informations">836×384 52.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: tom
cooked: <p>The easiest might be to provide use <code>torch.stack(batch_losses).mean().item()</code> or so to make a PyTorch tensor from a list of tensors and take the mean. .item() converts to a Python number.</p>
<p>Best regards</p>
<p>Thomas</p>

username: dieptran149
cooked: <p>I appreciate you! My problem was solved.</p>
Grayscale image plotted to have colours

username: asdas_casxc
cooked: <p>I am using PIL first to convert a saved image to grayscale then resize and convert to a tensor object using the following code:</p>
<pre><code class="lang-auto">gray_image = ImageOps.grayscale(img)

        resize = T.Compose([

                T.Resize((36,64))  # height, width

                ,T.ToTensor()

            ])
</code></pre>
<p>However, when i plot it using:</p>
<pre><code class="lang-auto">plt.figure()
plt.imshow(resize(gray_image).squeeze(0).permute(1, 2, 0).cpu(), interpolation='none')
plt.title('Non starting state example')
plt.show()
</code></pre>
<p>it shows me a plot like below that is not a grayscale.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/c/bc5207a47d7f84b389a80cd537ed723c6515d6ae.png" alt="img1" data-base62-sha1="qRXvMpPrzEnXIOTYIVo0FhYUxJA" width="640" height="480"></p>
<p>Is it because of the way the tensor is plotted?</p>

username: ptrblck
cooked: <p><code>matplotlib</code> uses the default colormap if you don’t specify another one. Use <code>plt.imshow(..., cmap='gray')</code> to use the grayscale colormap.</p>

username: asdas_casxc
cooked: <p>Thank you for your reply. Wouldn’t that be like manually setting the colors to be in grayscale? I want to see the tensor image as it is, which is supposed to be grayscaled. Not sure if that make sense.</p>

username: ptrblck
cooked: <p>If your tensor doesn’t contain a color channel dimension the colormap is undefined and matplotlib will use its default one.<br>
In case the tensor represents a real grayscale image, it would have a color channel size of 3 where all channels are equal.<br>
This code shows the behavior:</p>
<pre><code class="lang-python">x = torch.randint(0, 256, (100, 100, 1)).byte()
plt.imshow(x.numpy())

# create grayscale image
x = torch.cat((x, x, x), dim=2)
plt.imshow(x.numpy())
</code></pre>
<p>While single-channel images could be considered grayscale, matplotlib treats them as a matrix.</p>
Is there any other comprehensive model zoo like TIMM

username: bing
cooked: <p>Hi,<br>
I am trying to run transfer learning using various architecture.<br>
I came across (<a href="https://github.com/rwightman/pytorch-image-models" rel="noopener nofollow ugc">Pytorch Image Models(TIMM)</a>.<br>
The issue is they only support torch &gt;=1.4 while I am on 1.2 and the possibility of torch update is quite low because I am on power 9 architecture and still figuring out how to upgrade it(<a href="https://discuss.pytorch.org/t/installing-pytorch-on-ibm-power-9-architecture/135915">(Posted Question)</a>.<br>
So Is there any other comprehensive model zoo like TIMM?</p>

username: Mazhar_Shaikh
cooked: <p>I believe the <a href="https://pytorch.org/hub/" rel="noopener nofollow ugc">Pytorch Hub</a> is supported in Pytorch 1.2, though it is not as comprehensive as TIMM.</p>
How to calculate the product along a dimmension efficiently on a sparse matrix

username: user19
cooked: <p>For example, I have 400*800 tensor A and about 10 non-zero elements for each row.  Here’s how I do it:</p>
<pre><code class="lang-auto">A[A==0] = 1
result = torch.prod(A, dim=1)
</code></pre>
<p>I think it’s not very efficient because there are many redundant multiplications. Is there a faster way to do that on GPU?</p>
Model training on some machines but not others

username: MW47
cooked: <p>Hi there,</p>
<p>I have a model that trains on some machines but not others, I have tried running this model on 5 different machines,<br>
machine 1: trained before, not anymore<br>
machine 2: still trains<br>
machine 3: has never trained properly<br>
machine 4: still trains<br>
machine 5: still trains</p>
<p>If I could I would just use machines 1,2,or 5, but machines 1 and 3 don’t have enough ram and machine 5 does not support cuda.</p>
<p>Only machine 4 is windows, all others are linux. I am running in a venv and have ensured that the code across all the machines is identical except where I need to remove multiprocessing from windows.</p>
<p>Edit, I have now found out that changing the device to CPU on machine 1 and 3 allows it to run properly, but I still don’t know why it wont work with cuda.</p>

username: eqy
cooked: <p>It may be because there is an unsupported GPU (e.g., older compute capability). Could you provide some more details about the setup on the machine that has a GPU but won’t work with CUDA?</p>

username: MW47
cooked: <p>The machine has a 1660 super</p>

username: eqy
cooked: <p>OK, can you provide some more details about the problem? Is it running out of CUDA memory (e.g., even with the minimum possible batch size)?</p>

username: MW47
cooked: <p>thats the issue im having, as far as I can tell, the model doesn’t run out of memory, there are no errors being shown, since I have 12 possible outputs(one hot matrix of 12), my default accuracy should be ~8.33%, which is what I’m getting even after 100 epochs when I say the model wont train. but it trains properly on the CPU of all machines and on the GPU of other machines.</p>

username: MW47
cooked: <p>After removing all the non determinism caused by the GPU, and printing the gradient after each loss on machine 2 and 3 on both CPU and GPU, I can see that the loss on all is identical, and the gradients on machine 2 CPU and GPU along with machine 3 CPU are identical. For some reason machine 3 GPU doesn’t have identical gradients.</p>

username: eqy
cooked: <p>Unless the hardware is identical across the machines, I’m not sure that it would be expected that the gradients would be bitwise identical. For example, there might be subtle differences in rounding because different architectures might have kernels that do arithmetic operations (such as reductions) with a different ordering.</p>
Singular value decomposition (SVD)

username: Asad_Ullah
cooked: <p>Computing Singular Value Decomposition (SVD) in Pytorch always give me this error:<br>
U, _, V = torch.svd(torch.mm(src_vec, tgt_vec.t() ) )</p>
<p>Intel MKL ERROR: Parameter 4 was incorrect on entry to SLASCL.<br>
RuntimeError: svd_cuda: the updating process of SBDSDC did not converge (error: 23)<br>
Even I moved the Tensors to cpu but still getting the same issue.<br>
Any thoughts on this issue?</p>

username: zhangguanheng66
cooked: <p>If you really think there is an error with <code>torch.svd</code>, open an issue on pytorch github <a href="https://github.com/pytorch/pytorch/issues" rel="nofollow noopener">here</a></p>

username: Asad_Ullah
cooked: <p>I think my inputs contain nan, thats why I got an invalid svd.</p>

username: Brando_Miranda
cooked: <p>how did you resolve it? do you know why you got that?</p>
CPU training optimisation

username: bot
cooked: <p>I all! I have Intel i9-9980HK and running PyTorch on MacOS.</p>
<p>I am running a training script using custom Dataset and Dataloader, num_workers=0<br>
Before I updated the environment, my training script was utilizing 100% CPU - all cores in use efficiently. After I reinstalled PyTorch and some libraries, the utilization decreased. Now its 1/8 of what it used to be.</p>
<p><code>torch.__config__.parallel_info()</code> gives:</p>
<pre><code class="lang-auto">ATen/Parallel:
	at::get_num_threads() : 8
	at::get_num_interop_threads() : 8
OpenMP not found
Intel(R) Math Kernel Library Version 2020.0.1 Product Build 20200208 for Intel(R) 64 architecture applications
	mkl_get_max_threads() : 1
Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
std::thread::hardware_concurrency() : 16
Environment variables:
	OMP_NUM_THREADS : [not set]
	MKL_NUM_THREADS : [not set]
ATen parallel backend: native thread pool
</code></pre>
<p>Is it possible to make sense of this? Why mkl_get_max_threads gives 1? Is it possible to debug this and bring performance back? How does PyTorch adapt to CPU and RAM utilization? Shall I attempt to alter the training script to use all cores?</p>
Create 3D scatter matrix

username: thecho7
cooked: <p>I want to create a tensor having specific size like the below,</p>
<pre><code># a: (B, N, 2) = (2, 3, 2)
a = torch.LongTensor([[[2, 1], [2, 2], [3, 3]], [[2, 0], [3, 2], [1, 2]]])
K = 5

# The expected tensor has the size of (B, K, K) = (2, 5, 5)
# For the first batch,
result[0] = torch.LongTensor([[0, 0, 0, 0, 0],
                              [0, 0, 0, 0, 0],
                              [0, 1, 1, 0, 0],
                              [0, 0, 0, 1, 0],
                              [0, 0, 0, 0, 0]])
</code></pre>
<p>The result tensor can be a LongTensor or BoolTensor whatever. I need to use it as a mask.<br>
I want to avoid a loop operation since N might be scalable.</p>
<p>Thanks,</p>

username: ptrblck
cooked: <p>Directly indexing the result tensor should work:</p>
<pre><code class="lang-python">a = torch.LongTensor([[[2, 1], [2, 2], [3, 3]], [[2, 0], [3, 2], [1, 2]]])
K = 5

out = torch.zeros(a.size(0), K, K)
out[torch.arange(out.size(0)).unsqueeze(1), a[:, :, 0], a[:, :, 1]] = 1.
print(out)
&gt; tensor([[[0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.],
           [0., 1., 1., 0., 0.],
           [0., 0., 0., 1., 0.],
           [0., 0., 0., 0., 0.]],

          [[0., 0., 0., 0., 0.],
           [0., 0., 1., 0., 0.],
           [1., 0., 0., 0., 0.],
           [0., 0., 1., 0., 0.],
           [0., 0., 0., 0., 0.]]])
</code></pre>
Precision loss with torch::jit::freeze

username: louieh
cooked: <p>I’m seeing a strange behavior in <code>torch::jit::freeze</code> with precision loss that I’m not understanding, hope someone can clarify. The below code is pasted and ran with torch <code>1.9.1+cu111</code>. I have two models that are copies of each other. They are both frozen before running inference.</p>
<p>If I pass the same input to both, the exact same output is produced. However, if I pass the same input twice to the first model, and only once to the second model, the results are only equal up to some precision loss. This behavior does not reproduce if I do not freeze the models. The printed frozen code is the same for both models as well. Any ideas?</p>
<pre><code class="lang-auto">import argparse
import torch
from torch import nn
import copy


class BugModule(nn.Module):
    def __init__(self):
        super().__init__()
        self.filter = nn.Conv2d(4, 4, kernel_size=1, padding=0, bias=False, groups=4)
        
    def forward(self, x, y):
        cov_xy = self.filter(x*y) - self.filter(x) * self.filter(y)
        return cov_xy
    

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('--resolution', type=int, default=[4,4], nargs=2)
    args = parser.parse_args()

    model1 = BugModule()
    model2 = copy.deepcopy(model1)

    model1 = model1.to(device='cuda', dtype=torch.float32).eval()
    model1 = torch.jit.script(model1)
    model1 = torch.jit.freeze(model1)

    model2 = model2.to(device='cuda', dtype=torch.float32).eval()
    model2 = torch.jit.script(model2)
    model2 = torch.jit.freeze(model2)

    w, h = args.resolution
    x = torch.randn((1, 4, h, w), device='cuda', dtype=torch.float32)
    y = torch.randn((1, 4, h, w), device='cuda', dtype=torch.float32)

    with torch.no_grad():
      z1 = model1(x, y)
      # z1 = model1(x, y) # uncomment this to see precision loss
      z2 = model2(x, y)

      if not torch.equal(z1, z2):
        print('FAILED')
      else:
        print('SUCCESS')
</code></pre>

username: tom
cooked: <p>What you are seeing is a combination of two effects:</p>
<ul>
<li>
<p>Floating point arithmetic has it that it is only approximate. The difference you are seeing is within the range of accepted variation for single precision (1e-7ish), so neither of the two results is “incorrect”. Typically one would use torch.allclose or something similar rather than torch.equal to check whether the results match.</p>
</li>
<li>
<p>In your case, the JIT fuser - activated only in the second run of the module - has a different way of combining the three outputs of filter to compute cov_xy (the multiplication and subtraction), generating a bespoke fused kernel for it. This computes the slightly different result compared to the usual operators for * and -. Very likely, the exact order of addition differs. Note that PyTorch makes no guarantees here that any particular version runs, not even when you ask it to be deterministic (which this bit would be after two runs by default, when the JIT uses the fused kernels).</p>
</li>
</ul>
<p>Best regards</p>
<p>Thomas</p>
How to mention no grad in the following situation?

username: ADONAI_TZEVAOT
cooked: <p>I have a tensor x and a parameter tensor R. I want to store the matrix multiplication of x and R into z and also make sure that z does not require any gradients. How do I do that? Please help</p>
<pre><code class="lang-auto">.....
z = x @ self.R
.....
</code></pre>

username: ptrblck
cooked: <p>Wrapping the operation into a <code>no_grad()</code> context would work:</p>
<pre><code class="lang-python">R = nn.Parameter(torch.randn(1, 1))
x = torch.randn(1, 1)

with torch.no_grad():
    z = x @ R
print(z.grad_fn)
&gt; None
</code></pre>
Generalization with detectron 2 maskrcnn

username: Sylvain_Ard
cooked: <p>Hello,<br>
I have tried this code : <a href="https://github.com/facebookresearch/detectron2" class="inline-onebox" rel="noopener nofollow ugc">GitHub - facebookresearch/detectron2: Detectron2 is FAIR's next-generation platform for object detection, segmentation and other visual recognition tasks.</a> on maskrcnn and ResNet 101 and it detects well the trained images but not generalize well with new images. How to improve this please ?<br>
Thank you<br>
Best regards</p>

username: Sylvain_Ard
cooked: <p>noone have an answer please ?</p>

username: Sylvain_Ard
cooked: <p>nobody have an answer please ? for example I would be able to measure validation loss which is not displayed by default</p>

username: ptrblck
cooked: <p>Your question might be too general, as there is no silver bullet to make a model generalize. Your best bet would be to check other implementations and reuse their workflow (i.e. in particular the model architecture, data augmentation, optimization etc.).</p>

username: emcp
cooked: <blockquote>
<p>Your question might be too general, as there is no silver bullet to make a model generalize.</p>
</blockquote>
<p>My thoughts exactly… <a class="mention" href="/u/sylvain_ard">@Sylvain_Ard</a> it may help for you to share a bit how the new data looks… just how different is it?</p>
<p>For me I take a very narrow starting point with object detection… and watch very carefully to see how the data changes (some call this data drift) … without knowing that it’s hard to say what you should do in the model… or in the data other than retrain your model(s)</p>

username: Sylvain_Ard
cooked: <p>Hello, thank you for being interested in my question, here is my dataset with the JSON masks in COCO format : <a href="https://www.dropbox.com/sh/1xu93ep6xsvgm70/AACeC8PRkH2KlPCdUFXaU71ha?dl=0" class="inline-onebox" rel="noopener nofollow ugc">Dropbox - trainval - Simplify your life</a> the goal is to recognize leaflets in one-leaf images<br>
Best regards</p>

username: Sylvain_Ard
cooked: <p>for test with new data take a leaf image on Internet</p>

username: Sylvain_Ard
cooked: <p>the model is here for testing after 30000 iterations : <a href="https://www.dropbox.com/s/q7jakjx9yq2z4gq/model_final.pth?dl=0" class="inline-onebox" rel="noopener nofollow ugc">Dropbox - model_final.pth - Simplify your life</a></p>

username: Sylvain_Ard
cooked: <p>so you have an idea ?</p>
Save Pytorch Training Model in BinaryFormat insted of ZipFormat

username: SalmanLeo93
cooked: <p>I am trying to re-train the <a href="https://github.com/SpaceNetChallenge/SpaceNet_SAR_Buildings_Solutions/blob/master/README.md" rel="noopener nofollow ugc">model (solution 4)</a> but when i save the model it saves as zip file. I have pre-trained model files from original author.<br>
Is there any way to save the file to .pth but in Binary format?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/4/84e4e1ae8de805816b71aba623dcb3188a2faaa6.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/84e4e1ae8de805816b71aba623dcb3188a2faaa6" title="imgonline-com-ua-twotoone-5tr4NUNHj3BmW"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/4/84e4e1ae8de805816b71aba623dcb3188a2faaa6_2_690x194.jpeg" alt="imgonline-com-ua-twotoone-5tr4NUNHj3BmW" data-base62-sha1="iXDoXB3mOWyMLF4sxS4RMSE62Ds" width="690" height="194" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/4/84e4e1ae8de805816b71aba623dcb3188a2faaa6_2_690x194.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/8/4/84e4e1ae8de805816b71aba623dcb3188a2faaa6_2_1035x291.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/4/84e4e1ae8de805816b71aba623dcb3188a2faaa6_2_1380x388.jpeg 2x" data-dominant-color="BEBDBD"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">imgonline-com-ua-twotoone-5tr4NUNHj3BmW</span><span class="informations">3840×1080 341 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: tom
cooked: <p><code>torch.save</code> has an undocumented/not guaranteed option <code>_use_new_zipfile_serialization</code> that might help.</p>
<p>However, I would venture that it is better to use the new format. What is not working for you with it?</p>
<p>Best regards</p>
<p>Thomas</p>

username: SalmanLeo93
cooked: <p>_use_new_zipfile_serialization = False<br>
worked for me. I found it somewhere else.<br>
Thanks <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=10" title=":blush:" class="emoji" alt=":blush:"></p>
Break does not work as expected

username: hparsa
cooked: <p>When I add the break in the inner for loop, I expect to get the loss for only 1 batch. Not sure why I get 10 different lines with different loss values</p>
<pre><code class="lang-auto">loss_list=[]
for epoch in range(num_epochs):
    model.train()
    l=0
    for i, values in enumerate(train_iter):
        input_data = []
        for v in values:
            input_data.append(v.to(device)) #you should load here to GPU's if you have more than 1 GPU
            
        p_pos = [model(*t) for t in zip(*input_data[0:-1])]
        p_neg = [model(*t) for t in zip(*input_data[0:-2], input_data[-1])]
        ls = [loss(p, n) for p, n in zip(p_pos, p_neg)]
        optimizer.zero_grad()
        [l.backward(retain_graph=False) for l in ls]
        l += sum([l for l in ls])  #you should normalize by the number of devices if you have more than 1 GPU
        optimizer.step()        
        print(i, l)
        break
    loss_list.append(l)
</code></pre>
<p>Here are the results:<br>
0 tensor([[572.0556]], device=‘cuda:0’, grad_fn=)<br>
0 tensor([[560.5846]], device=‘cuda:0’, grad_fn=)<br>
0 tensor([[564.1666]], device=‘cuda:0’, grad_fn=)<br>
0 tensor([[562.2694]], device=‘cuda:0’, grad_fn=)<br>
0 tensor([[546.3769]], device=‘cuda:0’, grad_fn=)<br>
0 tensor([[564.4094]], device=‘cuda:0’, grad_fn=)<br>
0 tensor([[552.5000]], device=‘cuda:0’, grad_fn=)<br>
0 tensor([[537.7695]], device=‘cuda:0’, grad_fn=)<br>
0 tensor([[540.4388]], device=‘cuda:0’, grad_fn=)<br>
0 tensor([[543.6255]], device=‘cuda:0’, grad_fn=)</p>

username: tom
cooked: <p>It would seem that break successfully breaks from the for loop on train_iter, but you’d still get the results for every epoch (the outer for loop).</p>
Is it safe to use PyTorch 1.10 with CUDA 11.1?

username: StrausMG
cooked: <p>Hi! I have an environment where I cannot upgrade CUDA from 11.1 to 11.3. Can I still use PyTorch 1.10 despite the fact that it was built with CUDA 11.3? As far as I understand, the answer is “yes” and the biggest risk I take is that torch 1.10 uses some kernels missing in 11.3 (which will cause explicit error that I will see). Is it correct?</p>

username: ptrblck
cooked: <p>If you are installing the pip wheels or conda binaries, then note that they are shipping with their own CUDA runtime (statically linked in the pip wheels, dynamically via <code>cudatoolkit</code> in the conda binaries).<br>
Your local CUDA toolkit will be used, if you are building from source or custom CUDA extensions.</p>
<aside class="quote no-group" data-username="StrausMG" data-post="1" data-topic="135879">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/strausmg/48/8801_2.png" class="avatar"> StrausMG:</div>
<blockquote>
<p>the biggest risk I take is that torch 1.10 uses some kernels missing in 11.3 (which will cause explicit error that I will see). Is it correct?</p>
</blockquote>
</aside>
<p>No, since CUDA, cuDNN, NCCL, etc. are linked to the binaries, no kernels should be missing (there was a cuDNN issue with static linking, but that’s independent from the wheels).</p>