akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Introducing Nobuco: PyTorch to Tensorflow converter. Intuitive, flexible, efficient

lex.lutsenko: <p>Ever felt the need to deploy your new shiny Pytorch model on the Web / mobile devices / microcontrollers, and the native infrastructure just doesn’t cut it? Meanwhile, the Tensorflow team has been putting impressive work in their inference engines lately.</p>
<p>So, after suffering through the standard conversion process (via ONNX) for quite some time, I decided to make a dedicated library containing as little bullshit as possible.</p>
<p>Hence, <a href="https://github.com/AlexanderLutsenko/nobuco" rel="noopener nofollow ugc">Nobuco</a>. It’s designed with simplicity and hackability in mind and automatically solves mismatching channel orders offering near-optimal performance. Try it, and spread the word if you like it. If not, feel free to open an issue or feature request. All forms of contribution are highly welcome!</p>
How do I normalize the gradient of each pixel in an autoencoder

Andrew_Legend: <p>So There is this paper: <a href="https://paperswithcode.com/method/gradient-normalization" class="inline-onebox" rel="noopener nofollow ugc">Gradient Normalization Explained | Papers With Code</a></p>
<p>I want to scale the idea of this paper to the outputs of full-on autoencoder rather than a single output like a discriminator. However I am not sure or have a nice way to take the gradient norm of each pixel individually. Is there anything that I can do? Thank you in advance</p>
Problems in Training the Object Detection Model

yavatan: <p>Hello,<br>
I’m new in this field.<br>
I have some problems.<br>
Description:<br>
I have a dataset consisting of 9 classes. The folder structure is train/images and train/labels.<br>
Each image have several different object (e.g. the first image consists of 2 Cars and 3 Pedestrians).<br>
Problems:</p>
<ol>
<li>when the BATCH_SIZE is equal to 1, Dataloader works fine! But when I set the BATCH_SIZE greater than 1, and when I try to use TORCH.TENSOR(LABELS), I got this error:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/f/bf07ed863400bb8072882c0df5b31aa2e34fc326.png" alt="Screenshot from 2023-07-01 17-54-23" data-base62-sha1="rfWbibIdR3DkACd7CWFCPrrrAma" width="467" height="24"></li>
</ol>
<p>I think it is due to the length of the labels, which is different in each image!<br>
[[1.0, 1.0, 3.0, 5.0], [0.0, 2.0]]<br>
How can I solve it?</p>
<ol start="2">
<li>
<p>I want to continue with BATCH_SIZE = 1 to describe my another problems; when I to pass the targets to CROSS_ENTROPY_LOSS I get this error:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/f/3/f334239d79a8ded614af0afa0e51e18114e696b3.png" alt="Screenshot from 2023-07-01 17-59-50" data-base62-sha1="yHtJr9Vi3XLAnr7Sfw2tdXznrLd" width="515" height="49"><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/6/f/6fd5a6819a4739ae93aa10d544ec1df199ee13a6.png" alt="Screenshot from 2023-07-01 17-21-23" data-base62-sha1="fXkK4z21T8iJOexnxUxecSysWSG" width="596" height="23"></p>
</li>
<li>
<p>when I use Squeeze(0) to pass targets to my loss function this once I got this error:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/d/2/d2d4eb6a879330fe6ec77b737361a35c78659a16.png" alt="Screenshot from 2023-07-01 18-02-37" data-base62-sha1="u56rHXOTvMhLXo4ABntPjZkVffw" width="516" height="55"><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/c/5c4078e4b17050edea5786aef41640bf8492c267.png" alt="Screenshot from 2023-07-01 18-03-04" data-base62-sha1="da63CyxtXROjvAAanv3bCgFws5N" width="605" height="30"></p>
</li>
</ol>
<p>Update1;<br>
My Outputs have [1,9] shape<br>
I found that the output shape and the target shape must have same sizes. But my labels have different class indices and different sizes!<br>
How can I solve this problem?</p>
<p>Update2;<br>
I implent a code that turn labels to this; =&gt; e.g. [0., 1., 1., 0., 1., 0., 1., …], with the shape of [1,9].<br>
But this once my loss was the same amount at the end of each epoch!<br>
Is this implementation wrong?</p>
<p>Thank you for your helps and answers.</p>
Optimizer Error in non leaf tensor

Noorulain_Islam: <p>hello, I have an error in my code, can you help me to solve.<br>
I have initialized my weights<br>
weight_1 = nn.Parameter(torch.Tensor(128,128)).cuda()<br>
weight_2 = nn.Parameter(torch.Tensor(128,128)).cuda()</p>
<p>optim.Adam([{‘params’:model.parameters()}, {‘params’: [weight_1, weight_2]}], lr=1e-5) . It is giving me a value error: cannot optimize a non-leaf tensor.</p>

eqy: <p>I think you can get the desired behavior by moving the <code>.cuda()</code> call inside:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; w1 = torch.nn.Parameter(torch.Tensor(1)).cuda()
&gt;&gt;&gt; w2 = torch.nn.Parameter(torch.Tensor(1).cuda())
&gt;&gt;&gt; w1.is_leaf
False
&gt;&gt;&gt; w2.is_leaf
True
&gt;&gt;&gt;
</code></pre>
Convert output of torch.nn.functional.pdist to a full matrix

moveurbut: <p>I have a N<em>M matrix of images where each row of M values represents a vector. I would like to compare these vectors with each other for which I am using <code>pdist</code> method as shown below. This method returns a 1-D tensor and I am not able find the index of any 2 vectors that matched completely (distance = 0). Is there a way I can get a N</em>N matrix from this 1-D array?</p>
<pre><code class="lang-auto">a = torch.nn.functional.pdist(torch.rand(10, 10), p=2)
(a == 0).nonzero(as_tuple=True)[0]
</code></pre>

ptrblck: <p>As described in the <a href="https://pytorch.org/docs/stable/generated/torch.nn.functional.pdist.html?highlight=pdist#torch.nn.functional.pdist">docs</a>, <code>pdist</code> is identical to the upper triangular portion, excluding the diagonal, of <code>torch.norm(input[:, None] - input, dim=2, p=p)</code>so you should be able to remap these values to these indices if needed.</p>

umarzein: <p>I use scipy’s squareform function (<a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.squareform.html" class="inline-onebox" rel="noopener nofollow ugc">scipy.spatial.distance.squareform — SciPy v1.11.1 Manual</a>):</p>
<pre><code class="lang-python">from scipy.spatial.distance import squareform
from torch import Tensor
import torch
tmp=torch.nn.functional.pdist(Tensor([[1,2],[3,4],[2,1]]))
print(Tensor(squareform(tmp)))
#stdout
tensor([[0.0000, 2.8284, 1.4142],
        [2.8284, 0.0000, 3.1623],
        [1.4142, 3.1623, 0.0000]])
</code></pre>
How to use NVIDIA's `sparsity` feature when building models

vgoklani: <p>Hey there,</p>
<p>NVIDIA has a hardware based <code>sparsity</code> feature, but how exactly do I use this when building models in pytorch? Could you please point me to some examples.</p>
<p>thanks!</p>

ptrblck: <p>You could check the experimental sparsity research code from <a href="https://github.com/NVIDIA/apex/tree/master/apex/contrib/sparsity">here</a>.</p>
Efficient Memory Management: How to manage and store a tensor of shape [1000, 1 billion] efficiently without crashing workstation

MR1319: <p>I’m Actually dealing with the combinations, so i easily shoot up my columns upto 1 billion and i’m not sure how to handle this tensor efficiently without crashing my workstation.</p>
<p>CODE:<br>
combinations1 = list(combinations(self.columns,2))<br>
combinations2 = torch.combinations(torch.arange(self.df_feature_values.shape[1]),2)<br>
comb_tensor = self.df_feature_values.T[combinations2,:]</p>
<p>NOTE: I don’t want to do the chunking operation which will add overhead time to execution and i don’t want to decrease the dtype from  float 32 to float16.</p>
<p>I know it’s lot of constraints but if there’s any way to handle this problem then help is much appreciated.</p>
<p>Best,<br>
Madhav</p>
Pytorch getting error with valid cuDNN

anshs: <p>While running a pytorch script on a cluster, I’m getting the following <strong>error</strong>:</p>
<blockquote>
<p>Traceback (most recent call last):<br>
File “/global/u2/a/anshuman/StructRepGen_Dev/diff_gpu.py”, line 681, in <br>
z , z_mu, z_var = unet(batch_noisy, t)<br>
File “/global/homes/a/anshuman/.conda/envs/srg/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1130, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “/global/u2/a/anshuman/StructRepGen_Dev/diff_gpu.py”, line 560, in forward<br>
x = self.init_conv(x)<br>
File “/global/homes/a/anshuman/.conda/envs/srg/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1130, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “/global/homes/a/anshuman/.conda/envs/srg/lib/python3.9/site-packages/torch/nn/modules/conv.py”, line 307, in forward<br>
return self._conv_forward(input, self.weight, self.bias)<br>
File “/global/homes/a/anshuman/.conda/envs/srg/lib/python3.9/site-packages/torch/nn/modules/conv.py”, line 303, in _conv_forward<br>
return F.conv1d(input, weight, bias, self.stride,<br>
RuntimeError: Unable to find a valid cuDNN algorithm to run convolution</p>
</blockquote>
<p>I saw a post related to this error, where it was mentioned about the mismatch between cuda version and pytorch version. I tried to re-install the pytorch version but still the problem persists. Do I need to explicitly give the cuda path and the pytorch path? If so, then how may I do so? Thanks.</p>
<p>I have the following versions:</p>
<blockquote>
<p>torch.<strong>version</strong> = 1.12.1<br>
torch.version.cuda = 11.3<br>
torch.backends.cudnn.version() = 8302</p>
</blockquote>
<p>path of cuda ( <code>which nvcc</code> )=</p>
<blockquote>
<p>/opt/nvidia/hpc_sdk/Linux_x86_64/22.7/cuda/11.7/bin/nvcc</p>
</blockquote>
<p>cuda version on terminal = <code>nvcc --version</code></p>
<blockquote>
<p>nvcc: NVIDIA (R) Cuda compiler driver<br>
Copyright (c) 2005-2022 NVIDIA Corporation<br>
Built on Tue_May__3_18:49:52_PDT_2022<br>
Cuda compilation tools, release 11.7, V11.7.64<br>
Build cuda_11.7.r11.7/compiler.31294372_0</p>
</blockquote>

ptrblck: <p>Your locally installed CUDA toolkit and cuDNN won’t be used if you’ve installed the PyTorch binaries as they ship with their own CUDA dependencies. Could you update PyTorch to the latest version and check if you would still hit the error?</p>

anshs: <p>Surprisingly it worked when I re-installed it with cuda 11. Thanks.</p>
Get RuntimeError: mat1 and mat2 shapes cannot be multiplied when trying to load a test image in the trained model

akurtiev7: <p>Shapes of images are the same as the training and i use Grayscale both on training and testing. Here’s the code of model training # Import dependencies<br>
import torch<br>
from torch import nn, save, load<br>
from torch.optim import Adam<br>
from torch.utils.data import DataLoader<br>
from RuHandwrittenLetters import RuHandwrittenDataset</p>
<h1>
<a name="get-data-1" class="anchor" href="#get-data-1"></a>Get data</h1>
<p>train = RuHandwrittenDataset(‘labels.csv’,‘dataset’)<br>
dataset = DataLoader(train, 64)</p>
<h1>
<a name="image-classifier-neural-network-2" class="anchor" href="#image-classifier-neural-network-2"></a>Image Classifier Neural Network</h1>
<p>class ImageClassifier(nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
self.model = nn.Sequential(<br>
nn.Conv2d(1, 32, (3, 3)),<br>
nn.ReLU(),<br>
nn.Conv2d(32, 64, (3, 3)),<br>
nn.ReLU(),<br>
nn.Conv2d(64, 64, (3, 3)),<br>
nn.ReLU(),<br>
nn.Flatten(),<br>
nn.Linear(64 * (32 - 6) * (32 - 6), 33)<br>
)</p>
<pre><code>def forward(self, x):
    return self.model(x)
</code></pre>
<p>device = torch.device(“cpu”)</p>
<h1>
<a name="instance-of-the-neural-network-loss-optimizer-3" class="anchor" href="#instance-of-the-neural-network-loss-optimizer-3"></a>Instance of the neural network, loss, optimizer/</h1>
<p>clf = ImageClassifier().to(device)<br>
opt = Adam(clf.parameters(), lr=0.09)<br>
loss_fn = nn.CrossEntropyLoss()</p>
<h1>
<a name="training-flow-4" class="anchor" href="#training-flow-4"></a>Training flow</h1>
<p>if <strong>name</strong> == “<strong>main</strong>”:<br>
for epoch in range(15):  # train for 10 epochs<br>
for batch in dataset:<br>
X, y = batch<br>
X, y = X.to(device), y.to(device)<br>
yhat = clf(X)<br>
loss = loss_fn(yhat, y)</p>
<pre><code>        # Apply backprop
        opt.zero_grad()
        loss.backward()
        opt.step()

    print(f"Epoch:{epoch} loss is {loss.item()}")

with open('model_state2.pt', 'wb') as f:
    save(clf.state_dict(), f)
</code></pre>
<p>and model opener<br>
import <a href="http://torchvision.io" rel="noopener nofollow ugc">torchvision.io</a><br>
from torch import load<br>
import test2<br>
import torch<br>
from torchvision.transforms.functional import convert_image_dtype</p>
<p>device=‘cpu’</p>
<p>with open(‘model_state.pt’, ‘rb’) as f:<br>
test2.clf.load_state_dict(load(f))</p>
<p>img_tensor=torchvision.io.read_image(‘00_00_00_0000.png’,torchvision.io.ImageReadMode.GRAY)<br>
img_tensor=convert_image_dtype(img_tensor, dtype=torch.float32)<br>
img_tensor.to(device)</p>
<p>print(torch.argmax(test2.clf(img_tensor)))</p>

ptrblck: <p>You haven’t posted the stacktrace with the error message but I guess the shape mismatch is raised in the first linear layer. If so you should see the activation shapes as well as the weight shape of the linear layer in the error message. Change the <code>in_features</code> of the linear layer to the activation feature size and it should work.</p>
Cnn regression handling

Ludicio_casanova: <p>Hello !</p>
<p>I want to adapt a model from segmentation_model_pytorch to predict a continuoiuys value, regression.<br>
So i have image and a label (in range 0 to 100)</p>
<p>what should i do ? i am a beginner and i did a dummy model:</p>
<pre><code class="lang-auto">class CustomModel(nn.Module):
    def __init__(self, cfg, weight=None):
        super().__init__()
        self.cfg = cfg

        self.encoder = smp.Unet(
            encoder_name=cfg.backbone, 
            encoder_weights=weight,
            classes=cfg.target_size,
            activation='identity',
        )
        self.global_pool = nn.AdaptiveAvgPool2d(1)
    def forward(self, image):
        output = self.encoder(image)
        output = self.global_pool(output)
        output = output.view(output.size(0), -1)  # Flatten
        return output

def build_model(cfg, weight="imagenet"):
    print('model_name', cfg.model_name)
    print('backbone', cfg.backbone)

    model = CustomModel(cfg, weight)
    return model
</code></pre>
<p>and i try a random tensor :</p>
<pre><code class="lang-auto">import torch
import torch.nn.functional as F
input_tensor = torch.randn(1, 3, 256, 256)
weight = "imagenet"  
model = build_model(CFG, weight)
output = model(input_tensor)
regression_output = output.squeeze().cpu().detach().numpy()
print(regression_output)
</code></pre>
<p>Is it all i should do ?? i mean just set the activation to none (plus avg pool/flatten?) shouldn’t i remove the segmentation head and the decoder part?<br>
I am a bit lost, if someone can just guide me ? It seems logical that i just need the encoder part right?<br>
Have a good day !</p>

Ludicio_casanova: <p>Sorry i think because its more in the ecosystem but not pytorch i dont get any response, or maybe my question does not make any sense!</p>
<p>But the question is more:<br>
Using a segmentation library and modifying only the lasty activation function to be linaer is ok for a regression task (i am pretty sure yes because it seems working on my side) ?<br>
Or i should remove the segmentation head and the decoder part to get better result?</p>
<p>Have a good day!</p>

Ludicio_casanova: <p>wow i am worst than i already thought, i was already doing only on the encoder …<br>
well i am pretty sure is ok:</p>
<pre><code class="lang-auto">class CustomModel(nn.Module):
    def __init__(self, cfg, weight=None):
        super().__init__()
        self.cfg = cfg

        self.encoder = smp.Unet(
            encoder_name=cfg.backbone, 
            encoder_weights=weight,
            classes=cfg.target_size,
            activation='identity',
        )
        self.fc = nn.LazyLinear( out_features=1) 

    def forward(self, image):
        output = self.encoder(image)
        output = output.view(output.size(0), -1) 
        output = self.fc(output)  
        return output

</code></pre>
Unable to install pytorch on mac with pip and conda

SL12138: <p>The command:</p>
<pre><code class="lang-auto">pip3 install torch torchvision torchaudio
</code></pre>
<p>The error:</p>
<pre><code class="lang-auto">WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1002)'))': /simple/torch/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1002)'))': /simple/torch/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1002)'))': /simple/torch/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1002)'))': /simple/torch/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1002)'))': /simple/torch/
Could not fetch URL https://pypi.org/simple/torch/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/torch/ (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1002)'))) - skipping
ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
ERROR: No matching distribution found for torch
</code></pre>
<p>The other command:</p>
<pre><code class="lang-auto">conda install pytorch::pytorch torchvision torchaudio -c pytorch
</code></pre>
<p>The error:</p>
<pre><code class="lang-auto">Collecting package metadata (current_repodata.json): failed

CondaSSLError: Encountered an SSL error. Most likely a certificate verification issue.

Exception: HTTPSConnectionPool(host='conda.anaconda.org', port=443): Max retries exceeded with url: /pytorch/osx-64/current_repodata.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1007)')))
</code></pre>
<p>I am on MacOS 13.2.1 and Python 3.11.4.<br>
Anyone know how to fix it? Appreciate it in advance.</p>
Compute weight for CrossEntropyLoss function

aditya_dhaulakhandi: <p>Hi, I want to use the weight parameter while calculating the loss. I want to know how to supply the weights, for example, I have a batch of 32 and there are 6 classes, now I have a tensor of size 6 that contains the number of occurrences of each class in the batch. I want to know, how to compute the weight value for each class.</p>
<p>thanks.</p>

ptrblck: <p>A common approach is to calculate the weights based on the class frequencies from the entire training dataset instead of recomputing it for each batch.</p>

aditya_dhaulakhandi: <p>Okay, so I calculated the frequency distribution for the training data, now how do I calculate the weights?<br>
Thanks.</p>
Tensor beneath data access is not in accordance with print result

Alex_Chen: <p>When I am writing a class inherited from torch.autograd.Function, I found that in backward function, the data printed out in python([[[1., 1., 1., 1., 1., 1., 1., 1.]]]) and fetched in beneath the cuda code([[[1., 0., 0., 0., 0., 0., 0., 0.]]]) are not the same unless I clone the output_grad. I wonder what might be problem there. Thanks.</p>
<pre><code class="lang-auto">class FusedRotaryEmbeddingFunction(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x, cos, sin, position_ids, tensor_index, k_size, rotary_size, base):
        ctx.save_for_backward(cos, sin, position_ids)
        ctx.tensor_index = tensor_index
        ctx.k_size = k_size
        ctx.rotary_size = rotary_size
        ctx.base = base
        return fused_apply_rotary_emb_cuda.forward(x, cos, sin, position_ids, tensor_index, k_size, rotary_size, base)

    @staticmethod
    def backward(ctx, output_grad):
        cos, sin, position_ids = ctx.saved_tensors
        tensor_index = ctx.tensor_index
        k_size = ctx.k_size
        rotary_size = ctx.rotary_size
        base = ctx.base

        # incorrect result, unless I input output_grad.clone()
        x_grad = fused_apply_rotary_emb_cuda.backward(output_grad, cos, sin, position_ids, tensor_index, k_size, rotary_size, base)

        return (x_grad, None, None, None, None, None, None, None)
</code></pre>
Vmap and autograd for grad of output wrt input

pytorch_user13: <p>Hi all,</p>
<p>I need to compute the second derivative of the network output wrt input and I need access to the output per sample.</p>
<p>Following Pytorch’s tutorial (<a href="https://pytorch.org/tutorials/intermediate/per_sample_grads.html" class="inline-onebox" rel="noopener nofollow ugc">Per-sample-gradients — PyTorch Tutorials 2.0.1+cu117 documentation</a>), I managed to construct some simple code:</p>
<pre><code class="lang-auto">
        u_pred = self.forward(inputs)

        def compute_grad(u_in, x_in): 
            grad_x     = torch.autograd.grad(u_in.unsqueeze(0), x_in, grad_outputs=torch.ones_like(u_in.unsqueeze(0)), retain_graph=True, create_graph=True, only_inputs=True)[0]
            grad_xx    = torch.autograd.grad(grad_x.unsqueeze(0), x_in, grad_outputs=torch.ones_like(u_in.unsqueeze(0)), retain_graph=True, create_graph=True, only_inputs=True)[0]
            return grad_xx

        sample_grads = [compute_grad(u_pred[i], inputs) for i in range(bs)]
        sample_grads = zip(*sample_grads)
        sample_grads = [torch.stack(shards) for shards in sample_grads]
        sample_grads = torch.stack(sample_grads)
</code></pre>
<p>It works, but it is super slow.<br>
I also attempted to use vmap, but it does not seem work with adagrad.<br>
Does anybody have a recommendation on how to speedup these computations?</p>
Module 'qtorch' has no attribute 'quant'

AshwinB: <p>Hi, I want to use <a href="https://qpytorch.readthedocs.io/en/latest/quant.html#qtorch.quant.fixed_point_quantize" rel="noopener nofollow ugc">qtorch.quant.fixed_point_quantize</a> however when I try</p>
<pre><code class="lang-python">import qtorch
qtorch.quant.fixed_point_quantize
</code></pre>
<p>I get the following error</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
AttributeError: module 'qtorch' has no attribute 'quant'
</code></pre>
<p><a href="https://qpytorch.readthedocs.io/en/latest/qtorch.html#qtorch.FixedPoint" rel="noopener nofollow ugc">qtorch.FixedPoint</a> seems to be there though.<br>
When I try</p>
<pre><code class="lang-python">qtorch.FixedPoint
</code></pre>
<p>I get</p>
<pre><code class="lang-auto">&lt;class 'qtorch.number.FixedPoint'&gt;
</code></pre>
<p>Can someone help me diagnose why some modules are there and others aren’t?</p>
How do I checkpoint the last layer using `torch.utils.checkpoint`?

raining_day513: <p>I will use a very simple example to explain my problem. Say I have a 6-layer feedforward model represented as an array [1, 2, 3, 4, 5, 6], and I want to divide it into 3 segments, i.e. [1,2], [3,4], [5,6], for gradient-checkpointing. Given the comment at line516 of <code>torch.utils.checkpoint_sequential</code> as below, PyTorch will only call the <code>torch.utils.checkpoint</code> for the first two segments [1,2] [3,4] and the remaining two layers 5, 6 normally with their output data being saved for backward. With this design, I don’t know how to save the output data of the last layer 6 for backward phase.</p>
<p>Say I call <code>torch.utils.checkpoint</code> for the segment [5, 6] too, is it possible to write some extra code to make sure that the result of layer 6 will be saved for backward? Or maybe I don’t need to worry about this, because PyTorch will always save the output data of the last layer to compute the first gradient (the partial derivative of the last layer output w.r.t. the output of the last layer)?</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/pytorch/blob/b3e60ee05250a0b3289f4611d916e8ac4b0a0a78/torch/utils/checkpoint.py#L515_L525">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/blob/b3e60ee05250a0b3289f4611d916e8ac4b0a0a78/torch/utils/checkpoint.py#L515_L525" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/b3e60ee05250a0b3289f4611d916e8ac4b0a0a78/torch/utils/checkpoint.py#L515_L525" target="_blank" rel="noopener nofollow ugc">pytorch/pytorch/blob/b3e60ee05250a0b3289f4611d916e8ac4b0a0a78/torch/utils/checkpoint.py#L515_L525</a></h4>



    <pre class="onebox"><code class="lang-py">
      <ol class="start lines" start="505" style="counter-reset: li-counter 504 ;">
          <li>    def forward(input):</li>
          <li>        for j in range(start, end + 1):</li>
          <li>            input = functions[j](input)</li>
          <li>        return input</li>
          <li>
          </li>
<li>    return forward</li>
          <li>
          </li>
<li>if isinstance(functions, torch.nn.Sequential):</li>
          <li>    functions = list(functions.children())</li>
          <li>
          </li>
<li class="selected">segment_size = len(functions) // segments</li>
          <li># the last chunk has to be non-volatile</li>
          <li>end = -1</li>
          <li>for start in range(0, segment_size * (segments - 1), segment_size):</li>
          <li>    end = start + segment_size - 1</li>
          <li>    input = checkpoint(</li>
          <li>        run_function(start, end, functions),</li>
          <li>        input,</li>
          <li>        use_reentrant=use_reentrant,</li>
          <li>        preserve_rng_state=preserve,</li>
          <li>    )</li>
      </ol>
    </code></pre>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
AttributeError: 'TextDataset' object has no attribute 'to'

Chukwujike_Ezemandu: <p>I attempted to pass my input and model to the device as instructed during training, However, I was confronted with the  following error</p>
<pre><code class="lang-auto">Cell In[27], line 27, in train_chatbot(directory, model_output_path)
     11 training_args = TrainingArguments(
     12     output_dir=model_output_path,
     13     overwrite_output_dir=True,
   (...)
     19     logging_dir='./logs',
     20 )
     22 # Train the model
     23 trainer = Trainer(
     24     model=model.to(device),
     25     args=training_args,
     26     data_collator=data_collator,
---&gt; 27     train_dataset=train_dataset.to(device),
...
     29 )
     31 trainer.train()
     32 trainer.save_model(model_output_path)

AttributeError: 'TextDataset' object has no attribute 'to'
</code></pre>
<p>Here is the code used for implementation</p>
<pre><code class="lang-auto">def train_chatbot(directory, model_output_path):
    device = torch.device("cuda" if torch.cuda.is_available() else "mps" if torch.backends.mps.is_available() else "cpu")
    tokenizer = GPT2Tokenizer.from_pretrained("./pretrained")
    model = GPT2LMHeadModel.from_pretrained("./pretrained")

    train_dataset = TextDataset(tokenizer=tokenizer, file_path= (directory + "train.txt"), block_size=128)
    val_dataset = TextDataset(tokenizer=tokenizer, file_path=(directory + "test.txt"), block_size=128)
    data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)

    # Set up the training arguments
    training_args = TrainingArguments(
        output_dir=model_output_path,
        overwrite_output_dir=True,
        per_device_train_batch_size=4,
        per_device_eval_batch_size=4,
        num_train_epochs=100,
        save_steps=10_000,
        save_total_limit=2,
        logging_dir='./logs',
    )

    # Train the model
    trainer = Trainer(
        model=model.to(device),
        args=training_args,
        data_collator=data_collator,
        train_dataset=train_dataset.to(device),
        eval_dataset=val_dataset.to(device),
    )

    trainer.train()
    trainer.save_model(model_output_path)
    
    # Save the tokenizer
    tokenizer.save_pretrained(model_output_path)
</code></pre>

ptrblck: <p>The <code>.to()</code> method is defined for tensors and modules, but not datasets. Are you using a custom <code>Dataset</code> where this method should be available but isn’t?</p>

Chukwujike_Ezemandu: <p>I fixed the issue. Simply set use_mps_device in the training arguments to True.</p>

Chukwujike_Ezemandu: <p>I do have another question but it is related to Macs.<br>
is there a way to use CPU only rather than mps while training</p>