akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Why can the attribute "device" not be resolved within the namespace of torch?

make1234: <p>Hello,</p>
<p>I want to follow along the introductory tutorial <a href="https://pytorch.org/tutorials/beginner/basics/buildmodel_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Build the Neural Network — PyTorch Tutorials 2.6.0+cu124 documentation</a> and copied the source code. It raises the error:</p>
<pre><code class="lang-auto">AttributeError: module 'torch' has no attribute 'accelerator'
</code></pre>
<p>and I do not understand why this supposedly compiles without error for the tutorial but does not on my machine (M2 macbook air).</p>
<pre><code class="lang-auto">import os
import torch
from torch import nn
from torch.utils.data import DataLoader
from torchvision import datasets, transforms


class NeuralNetwork(nn.Module):
    def __init__(self):
        super().__init__()
        self.flatten = nn.Flatten()
        self.linear_relu_stack = nn.Sequential(
            nn.Linear(28*28, 512),
            nn.ReLU(),
            nn.Linear(512, 512),
            nn.ReLU(),
            nn.Linear(512, 10),
        )

    def forward(self, x):
        x = self.flatten(x)
        logits = self.linear_relu_stack(x)
        return logits

device = torch.accelerator.current_accelerator().type if torch.accelerator.is_available() else "cpu"
print(f"Using {device} device")

model = NeuralNetwork()
print(model)

</code></pre>

ptrblck: <p>The code works for me using <code>torch==2.6.0+cu124</code>:</p>
<pre data-code-wrap="python"><code class="lang-python">python -c "import torch; print(torch.__version__); print(torch.accelerator.current_accelerator().type)"
2.6.0+cu124
cuda
</code></pre>

make1234: <p>an update did the trick, thanks a lot!</p>

classic-21: <p>I tried upgrading torch and torchvision together and separately also, but the error was not resolved in both cases.</p>
<p><code>&gt; pip install --upgrade torch torchvision</code></p>
<p>btw I am running it on Google Colab</p>

ptrblck: <p>My code shows that the function exists and works in 2.6.0 so double check which version you are on.</p>
'Tensor.data_ptr()' not visible from TorchScript

Avi_Chapman: <p>Hi all,</p>
<p>When I try to compile a TorchScript module that includes use of the spconv library, I get an error claiming that:</p>
<blockquote>
<p>‘Tensor’ boject has no attribute or method ‘data_ptr’.</p>
</blockquote>
<p>I believe this is because ‘data_ptr’ is defined in a parent class of Tensor - ‘TensorBase’. Since TorchScript doesn’t do inheritence, it can’t find this method.</p>
<p>I’ve tried asserting that the tensor is an instance of torch.BaseTensor (or torch._C.BaseTensor). That fails, claiming that that type can’t be found.</p>
<p>Can anyone think of a way to make this work?</p>

Avi_Chapman: <p>I found a solution.</p>
<p>You can use <code>torch.library.custom_op</code> to register the operation. It does not show up by default because it is a call into C++.</p>
<p>Example:</p>
<pre><code class="lang-auto">@torch.library.custom_op("mylib::data_ptr", mutates_args=())
def data_ptr(tensor: torch.Tensor) -&gt; int:
    return tensor.data_ptr()

@data_ptr.register_fake
def _(_: torch.Tensor) -&gt; int:
    return 0
</code></pre>
<p>You then change every instance of <code>tensor.data_ptr()</code> to <code>data_ptr(tensor)</code>.</p>
Bad results at efficientnet transfer learning at CIFAR100?

mhenning: <p>First of, this is my first post here, so if I forget something, please just ask/write me. I hope it’s okay here to post comparisons for TF/PyTorch approaches.</p>
<p>I tested out some simple transfer learning in TensorFlow and PyTorch with the EfficientNet_B0, B5 and the ConvNext (for comparison) on the CIFAR100 dataset with pre-trained imagenet weights and frozen networks. The code and results are here:<br>
PyToch (lightning): <a href="https://colab.research.google.com/drive/1STie7YroLrWc77ce8QXWKgO3Rbsvr_5c?usp=sharing" class="inline-onebox" rel="noopener nofollow ugc">Google Colab</a><br>
TensorFlow: <a href="https://colab.research.google.com/drive/1CTiuR8WQnSryrscnM75KRfRc1EKMr6o9?usp=sharing" class="inline-onebox" rel="noopener nofollow ugc">Google Colab</a></p>
<p>As you can see, the TensorFlow version of EfficientNet gets 70+% accuracy out of the box, while the PyTorch version just gets ~58%. I also tested it with more epochs on another machine, and it didn’t get much better. Compared with that, the ConvNext versions perform similar to each other. I know that one can’t expect completely mirrored results, but 12-13% difference is a lot. Why is that? Did I make a mistake in the processing of the data, or the building of the network? Why does the convnext version runs fine then?<br>
To comment the workflow, I tried to make both workflows as similar as possible, so both approaches share the resize and crop transformation that torchvision returns as the IMAGENET1K transformations. Initially I also had image augmentation, but I discarded it for now for better comparison. I also know that the B5 imagenet has a higher recommened resolution, but training with that runs forever on colab, and it didn’t perform better as I tested it. And the TF version for b5 runs just fine on 224x224.<br>
I also paid attention to the right scale of the data, [0-255] for TF and [0-1] for PyTorch. The use of PyTorch Lighning should not make a difference here, right?</p>
<p>As this is my first step into transfer learning, I’d be grad to know any mistakes I made, as to avoid them in the future. Thanks in advance!</p>

Bjorn_Lindqvist: <p>It very much looks like you are not resizing the images to 224x224. When I did the same error training also got stuck at ~55%.</p>
Unexpected behavior when slicing numpy array with torch tensor

Thunderz: <p>Hi, I have an issue with slicing a numpy array with a tensor of one element. For example:</p>
<pre><code class="lang-auto">In [1]: import torch

In [2]: torch.__version__
Out[2]: '2.4.0+cu121'

In [3]: import numpy as np

In [4]: np.__version__
Out[4]: '2.0.2'

In [5]: a = np.random.normal(size=(3, 4))

In [6]: a
Out[6]: 
array([[-0.03374389, -2.06199934,  0.22114562, -0.10317616],
       [-1.40243768, -0.86554835,  0.49141156,  1.49605485],
       [ 2.05006526, -1.18818391,  0.61184828,  1.82132413]])

In [7]: torch_idx = torch.tensor([1])

In [8]: np_idx = np.array([1])

In [9]: a[:, torch_idx]
Out[9]: array([-2.06199934, -0.86554835, -1.18818391])

In [10]: a[:, np_idx]
Out[10]: 
array([[-2.06199934],
       [-0.86554835],
       [-1.18818391]])
</code></pre>
<p>I would expect that both methods would return a (3, 1) column but it seems that slicing with a torch tensor returns a (3, ) row. I was wondering if this is intentional and why is it like this?</p>