akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
OpenXExperienceReplay fails

yash_as: <p>Hi,<br>
I’m trying to use OpenX API for loading a dataset:</p>
<pre><code class="lang-auto">dataset = OpenXExperienceReplay(
        "berkeley_gnm_cory_hall",
        download='force',
        streaming=False,
        root=ds_root
    )
</code></pre>
<p>But it fails on any dataset from this collection, similarly, with this error:</p>
<pre><code class="lang-auto">  File "~/.cache/huggingface/modules/datasets_modules/datasets/jxu124--OpenX-Embodiment/317e9044a9bb97bb1db9ea5aebf1c15f5cc3e1e071e5da025e97892e96dae22b/OpenX-Embodiment.py", line 29, in decode_image
    data = data.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  ...
  File "...", line 130, in main
    dataset = OpenXExperienceReplay(
  File ".../python3.10/site-packages/torchrl/data/datasets/openx.py", line 358, in __init__
    storage = self._download_and_preproc()
  File ".../python3.10/site-packages/torchrl/data/datasets/openx.py", line 484, in _download_and_preproc
    dataset = datasets.load_dataset(
  File ".../python3.10/site-packages/datasets/load.py", line 2096, in load_dataset
    builder_instance.download_and_prepare(
  File ".../python3.10/site-packages/datasets/builder.py", line 924, in download_and_prepare
    self._download_and_prepare(
  File ".../python3.10/site-packages/datasets/builder.py", line 1647, in _download_and_prepare
    super()._download_and_prepare(
  File ".../python3.10/site-packages/datasets/builder.py", line 999, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File ".../python3.10/site-packages/datasets/builder.py", line 1485, in _prepare_split
    for job_id, done, content in self._prepare_split_single(
  File ".../python3.10/site-packages/datasets/builder.py", line 1642, in _prepare_split_single
    raise DatasetGenerationError("An error occurred while generating the dataset") from e
datasets.exceptions.DatasetGenerationError: An error occurred while generating the dataset
</code></pre>
<p>Has anyone had a successful experience with this API? Seems like it still needs some work.</p>

vmoens: <p>I can reprod.<br>
We’re calling <code>datasets.load_datasets</code> here and the <code>datasets</code> lib from HF fails to do that. The error stack suggests it’s a datasets error, not TorchRL.<br>
You can reproduce this via</p>
<pre data-code-wrap="python"><code class="lang-python">
import datasets

datasets.load_dataset(
                "jxu124/OpenX-Embodiment",
                "berkeley_gnm_cory_hall",
                streaming=False,
                split="train",
                cache_dir="./dump",
                trust_remote_code=True,
            )
</code></pre>
<p>I started a discussion <a href="https://huggingface.co/datasets/jxu124/OpenX-Embodiment/discussions/2" rel="noopener nofollow ugc">here</a></p>
Can't set weights_only=True if the device is GPU

3di: <p>I first get a future warning telling me to set weights_only to True for security puporses, but when I do, it says I can’t use weights_only = True.  I get the below error.</p>
<p><code>state_dict = torch.load(path_to_state_dict, map_location="gpu",weights_only=True)</code></p>
<p>torch\serialization.py", line 1096, in load<br>
raise pickle.UnpicklingError(_get_wo_message(str(e))) from None<br>
_pickle.UnpicklingError: Weights only load failed. Re-running <code>torch.load</code> with <code>weights_only</code> set to <code>False</code> will likely succeed, but it can result in arbitrary code execution. Do<br>
it only if you got the file from a trusted source.<br>
Please file an issue with the following so that we can make <code>weights_only=True</code> compatible with your use case: WeightsUnpickler error: don’t know how to restore data location of t<br>
orch.storage.UntypedStorage (tagged with gpu)<br>
Check the documentation of torch.load to learn more about types accepted by default with weights_only <a href="https://pytorch.org/docs/stable/generated/torch.load.html" class="inline-onebox" rel="noopener nofollow ugc">torch.load — PyTorch 2.4 documentation</a>.</p>
<p>No problem if I use weights_only=True if the map_location is ‘cpu’.</p>

albanD: <p>Hi,</p>
<p>The error message here mentions: “don’t know how to restore data location of t<br>
orch.storage.UntypedStorage (tagged with gpu)”<br>
Indeed, “gpu” is not a valid device in PyTorch, I guess you meant “cuda” ?</p>

3di: <p>Thanks, you were exactly right.  Legend <img src="https://discuss.pytorch.org/images/emoji/apple/+1.png?v=12" title=":+1:" class="emoji" alt=":+1:" loading="lazy" width="20" height="20"></p>
Why models training is extremely slow?

ciruzzo: <p>I’m encountering strange behavior when training any model: models that previously required little time to complete a training epoch now require much more. For example, a relatively simple model like DGCNN with a dataset fixed and trained on cuda used to take approximately 100 seconds for each epoch, including forward and bacward pass, loss computation and other small computation related to the training script, but now the same identical script takes 30 minutes!<br>
The same behavior occurs for training other models which has now become very slow.</p>
<p>And I also noticed that CPU training has slowed down (could it be related?)</p>
<p>I originally encountered this problem with an installation of pytorch 2.2 and cuda 12.1, so thinking it was due to these versions, I re-ran a clean install of both, but the same problem also occurs with versions 2.4 and 12.4</p>
<p>Can anyone tell me what the cause could be and/or what to check?</p>

ptrblck: <p>Double post from <a href="https://discuss.pytorch.org/t/why-models-training-is-extremely-slow/211599">here</a>.</p>

ciruzzo: <p><a class="mention" href="/u/ptrblck">@ptrblck</a><br>
Yes, i’m sorry. I will delete it</p>
Why models training is extremely slow

peppino: <p>I’m experiencing strange behavior in training all my models: the training time of any model (even a relatively simple one) has increased significantly for no apparent reason. For example, to train a model like <a href="https://github.com/antao97/dgcnn.pytorch/blob/07d534c2702905010ec9991619f552d8cacae45b/model.py#L214" rel="noopener nofollow ugc">this</a> on a fixed dataset using cuda, it previously took about 100 seconds for each epoch including backward pass, loss computation and other small computation related to the training script, but now it takes up to 30 minutes!<br>
This slowdown also occurs when training other models that were previously very fast and I also noticed that CPU training has slowed down (could it be related?).</p>
<p>I originally encountered this problem with version 2.2 of pytorch and cuda 11.8, thinking it was due to the versions of these, I tried to perform a clean installation of both framworks again, but the same behavior also occurs with the versions 2.4 and 12.4.</p>
<p>Could anyone kindly suggest to me what it could be due to or what to check?</p>
Modifying Convolution Weights

Izan_C_G: <p>Hello, I would like to implement an error correction mechanism for the weights of a convolution during the inference of a pre-trained model in PyTorch (no training process needed).</p>
<p>I am considering two options: the first is to perform a check on the weights before the convolution (could this be done via a forward pre-hook?), and the second is to modify the operator/convolution code directly also to perform the checking before the convolution.</p>
<p>I would like to know which option would be easier to implement and which would be more optimal regarding temporal overheads.</p>
<p>In the first case, I have experience with hooks. Still, in the case of modifying an operator’s code in CUDA, like convolution, I am unsure how to proceed, so any tutorial or guidance would be greatly appreciated.</p>
<p>Thank you.</p>
Torch isend irecv hang?

zili_zhang: <p>I write a simple program where two process send a tensor to each other using torch.isend and torch.irecv. I use different stream for sending and receiving but it seems to be dead-lock. I use <code> torchrun --nproc_per_node=2 test.py</code> to launch this program. But the program (the both two ranks) hangs on <code>req2.wait()</code>.</p>
<pre><code class="lang-auto">import torch
import torch.distributed as dist

def run():
    rank = dist.get_rank()
    size = dist.get_world_size()
    tensor = torch.zeros(1, device=f'cuda:{rank}')

    send_stream = torch.cuda.Stream(device=f'cuda:{rank}')
    recv_stream = torch.cuda.Stream(device=f'cuda:{rank}')
    
    if rank == 0:
        tensor += 1 
        with torch.cuda.stream(send_stream):
            req1 = dist.isend(tensor=tensor, dst=1)
        with torch.cuda.stream(recv_stream):
            req2 = dist.irecv(tensor=tensor, src=1)
        req1.wait() 
        print("req1 finished")
        req2.wait() # will hang here
        print('Rank 0 received:', tensor.item())
    elif rank == 1:
        tensor += 2 
        with torch.cuda.stream(send_stream):
            req1 = dist.isend(tensor=tensor, dst=0)
        with torch.cuda.stream(recv_stream):
            req2 = dist.irecv(tensor=tensor, src=0)
        req1.wait()  
        print("req1 finished")
        req2.wait() # will hang here
        print('Rank 1 received:', tensor.item())

if __name__ == "__main__":
    dist.init_process_group("nccl", init_method='env://', world_size=2)  # init nccl
    run()

</code></pre>

flother_xi: <p>hi, have you solved this problem?<br>
i also encountered this problem</p>

zili_zhang: <p>I have to reorder the communication topology (e.g., process one: send then receive, process two: receive then send) to resolve the issue. My scenario isn’t complex, so this solution works for me.</p>

flother_xi: <p>sounds good, thank you! i will try it.</p>
Lightning: Loading a checkpoint when using json/yaml configs

JLenz: <p>Hello! I’m running into an issue with the code I have for loading checkpoints. Within my wrapped lightning module I have a single nn model which is instantiated with a model config and tokenizer config. The model config is a .json file specifying various model hyperparameters and the tokenizer config is a python file that similarly defines the tokenizer characteristics.</p>
<p>Within pytorch lightning (pl) I use the common way of saving checkpoints:</p>
<pre><code class="lang-auto">class ModelTrainer(pl.LightningModule):
    def __init__(
        self,
        train_config: dict or str,
        tokenizer_config: str = None,
        model_config: str = None,
    ):
        super().__init__()
        self.save_hyperparameters()
        self.model: MyModel = MyModel(tokenizer_config, model_config)
        (rest of the code to load the model)

</code></pre>
<p>The problem arises when I try to instantiate it:</p>
<p><code>model: ModelTrainer = ModelTrainer.load_from_checkpoint(checkpoint)</code><br>
It gives me an error from the init method of the model class itself (MyModel) at the part where it tries to actually load the json:</p>
<pre><code class="lang-auto">with open(model_config, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '.../model_configs/model_medium.json'
</code></pre>
<p>Prior to using PL I was using lucidrains pytorch custom utils for checkpoint saving/loading, which somehow avoided this issue altogether and is able to load the weights without attempting to read any of the json files:</p><aside class="onebox githubrepo" data-onebox-src="https://github.com/lucidrains/pytorch-custom-utils">
  <header class="source">

      <a href="https://github.com/lucidrains/pytorch-custom-utils" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <div class="github-row" data-github-private-repo="false">
  <img width="690" height="344" src="https://opengraph.githubassets.com/f1caefc9c59334cda3fddc49e7e93ea8/lucidrains/pytorch-custom-utils" class="thumbnail">

  <h3><a href="https://github.com/lucidrains/pytorch-custom-utils" target="_blank" rel="noopener nofollow ugc">GitHub - lucidrains/pytorch-custom-utils: Just some miscellaneous utility functions /...</a></h3>

    <p><span class="github-repo-description">Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new AI research</span></p>
</div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
(specifically the <code>init_and_load</code> functions)<br>
But with the pl lightning module I have to use the pytorch model-saving framework. I guess my question here is, how can I instantiate the model from trained weights, while bypassing the need for the configs altogether?</p>
<p>Many thanks in advance!</p>
How to change default return type of `torch.argmax`

saul: <p>I need to convert a torch model to TFLite and accelerate it on an Android device using the NNAPI. I can perform the conversion itself, but my model has an <code>argmax</code> node in it that is causing trouble. The reason it’s causing trouble, I’m quite sure, is that the default and <em>only</em> return type of <code>torch.argmax</code> is a <code>LongTensor</code>, i.e. an <code>int64</code>. <a href="https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0af78048ff71a1e79bbc6034d30cad7cbc" rel="noopener nofollow ugc">Android NN docs show</a> that only <code>int</code>s up to <code>32</code>-bit are supported.</p>
<p>Typically I would just cast the output to whatever data type I needed. But because the whole graph is trying to be loaded by the NNAPI, it ends up breaking the model into partitions and sending everything after the argmax to the CPU. This causes a big slowdown. Is there any way I can avoid generating an <code>int64</code> in the forward pass <em>at all</em>?</p>

demq: <p>Hello,</p>
<p>Did you by any chance find a solution to this problem?</p>

saul: <p>Unfortunately not really. The work around was to first convert to ONNX, where I could edit that graph to explicitly use an <code>int32</code> in the argmax, and then make a jump to TFLite. Frustrating process.</p>

Valerio_Nogueira: <p>Hi, I’m facing the same problem with the INT64 not being supported by the TFLite Micro. Can you give some adive on how to change the argmax datatype to int32?</p>

Valerio_Nogueira: <p>Although I couldn’t find a way to change the ArgMax node data type I found this tool that lets you visualize and edit the ONNX model: <a href="https://github.com/ZhangGe6/onnx-modifier" class="inline-onebox" rel="noopener nofollow ugc">GitHub - ZhangGe6/onnx-modifier: A tool to modify ONNX models in a visualization fashion, based on Netron and Flask.</a>. Using this tool I was able to remove the ArgMax node from the graph and now it works nicely with the TFLite Micro</p>
Stack different number of crops

External_happy: <p>Hi, I have a batch of 32 images, and I used an algorithm to generate crops from each image. However, the algorithm produced a different number of crops for each image. How can I stack these crops so I can feed them into a model, given that <code>torch.stack</code> requires all batches to have the same number of crops?</p>

AbdulsalamBande: <p>I would recommend you either pad images with the highest number of crops (maybe using zero-filled crops) OR If you are using a Dataloader, you can use  a custom collate function that dynamically handles batches with different crops.</p>
Install pytorch 2.5 using intel GPU (XPU)

triilman25: <p>I got error bellow errors when I was run command:<br>
the commands:</p>
<pre><code class="lang-auto">export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
python setup.py develop
</code></pre>
<p>the error:</p>
<pre><code class="lang-auto">CMake Error at third_party/kineto/libkineto/src/plugin/xpupti/CMakeLists.txt:23 (find_package):
  By not providing "FindPti.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Pti", but
  CMake did not find one.

  Could not find a package configuration file provided by "Pti" with any of
  the following names:

    PtiConfig.cmake
    pti-config.cmake

  Add the installation prefix of "Pti" to CMAKE_PREFIX_PATH or set "Pti_DIR"
  to a directory containing one of the above files.  If "Pti" provides a
  separate development package or SDK, be sure it has been installed.
</code></pre>
<p>I have installed “Pti” in <strong>WSL2</strong>. the path is on “/opt/intel/oneapi/pti/…”</p>
<p>link for the installation tutorial:</p><aside class="onebox githubrepo" data-onebox-src="https://github.com/pytorch/pytorch?tab=readme-ov-file#install-pytorch">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch?tab=readme-ov-file#install-pytorch" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <div class="github-row" data-github-private-repo="false">
  <img width="690" height="344" src="https://opengraph.githubassets.com/ced5c5f4d7bf35e24e65a1e6934b1353/pytorch/pytorch?tab=readme-ov-file#install-pytorch" class="thumbnail">

  <h3><a href="https://github.com/pytorch/pytorch?tab=readme-ov-file#install-pytorch" target="_blank" rel="noopener nofollow ugc">GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with...</a></h3>

    <p><span class="github-repo-description">Tensors and Dynamic neural networks in Python with strong GPU acceleration</span></p>
</div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


Kanya: <p>Hi,</p>
<p>Could you try</p>
<blockquote>
<p>source /opt/intel/oneapi/pti/latest/env/vars.sh</p>
</blockquote>
<p>before python setup.py develop?</p>
<p>Here is a detailed <a href="https://www.intel.com/content/www/us/en/developer/articles/tool/pytorch-prerequisites-for-intel-gpu/2-5.html" rel="noopener nofollow ugc">instruction</a> for building pytorch with intel xpu.</p>
LibTorch distribution for GH200

Matthias_Moller: <p>Dear all,</p>
<p>is there a prebuilt version of LibTorch for the NVIDIA GH200 superchip? If not, do you have some recommendations how to build it from the sources?</p>
<p>Best regards,<br>
Matthias</p>

ptrblck: <p>You can install the latest stable or nightly pip wheels with CUDA 12.4 and extract the <code>libtorch*.so</code> libraries for your use case.</p>
How to define forward of nn.Embedding

lazysloth: <p>Hi,<br>
I would like to apply the code below to nn.Embedding.<br>
However, the forward function still calls the original nn.Embedding forward, instead of calling the one I created.<br>
I would appreciate your help. Could you tell me why the model behaves like this?</p>
<pre><code class="lang-auto">class Linear(nn.Linear): 
    def __init__(self, in_features, out_features):
        super(Linear, self).__init__(in_features, out_features)
        self.weight.fast = None #Lazy hack to add fast weight link
        self.bias.fast = None

    def forward(self, x):
        if self.weight.fast is not None and self.bias.fast is not None:
            out = F.linear(x, self.weight.fast, self.bias.fast)
        else:
            out = super(Linear, self).forward(x)
        return out
</code></pre>
<pre><code class="lang-auto">class Embedding(nn.Embedding): 
    def __init__(self, num_embeddings, embedding_dim):
        super(Embedding, self).__init__(num_embeddings, embedding_dim)
        self.weight.fast = None
        
    def forward(self, user_id):
        if self.weight.fast is not None:
            user_emb = F.embedding(user_id, self.weight.fast)
        else:
            user_emb = super(Embedding, self).forward(user_id)
        return user_emb
</code></pre>

ptrblck: <p>The code works for me and calls the custom <code>forward</code>:</p>
<pre data-code-wrap="python"><code class="lang-python">class Embedding(nn.Embedding): 
    def __init__(self, num_embeddings, embedding_dim):
        super(Embedding, self).__init__(num_embeddings, embedding_dim)
        self.weight.fast = None
        
    def forward(self, user_id):
        print("calling custom forward")
        if self.weight.fast is not None:
            print("calling fast custom")
            user_emb = F.embedding(user_id, self.weight.fast)
        else:
            print("calling default custom")
            user_emb = super(Embedding, self).forward(user_id)
        return user_emb

emb = Embedding(10, 10)
x = torch.randint(0, 10, (10,))

out = emb(x)
# calling custom forward
# calling default custom

emb.weight.fast = emb.weight.clone()
out = emb(x)
# calling custom forward
# calling fast custom
</code></pre>
Moving tensor to cuda looses precision

sina-y: <p>I am trying to move a tensor to cuda, but unfortunately, I am loosing precision. The first entry is 5.02e-5 which is converted to 0. Is there a work around? I have tried a lot of things but could unfortunately not find the answer.</p>
<pre><code> vals = vals.to(torch.float64).to(device_of_t)
</code></pre>

ptrblck: <p>I cannot reproduce any issue using the mentioned value:</p>
<pre data-code-wrap="python"><code class="lang-python">vals = torch.tensor(5.02e-5)
print(vals)
# tensor(5.0200e-05)

vals = vals.to(torch.float64).to("cuda")
print(vals)
# tensor(5.0200e-05, device='cuda:0', dtype=torch.float64)
</code></pre>

sina-y: <p>Thank you very much for your quick answer! I was also quite confused and I also can’t reproduce the error on another laptop. Do you think it could be a problem with my installation?</p>
Undo pruning - How to 'unmask' pruned weights

IdoGalil: <p>Hey!,</p>
<p>I’m using PyTorch pruning. After pruning a model (a specific module in it, or globally), how can I undo the pruning? or at least ‘unmasking’ all the weights?</p>
<p>I was trying to modify the mask itself to contain only ones using custom_from_mask, but it seems to only work for pruning more weights, not ‘unmasking’ already pruned ones.</p>
<p>Thanks!</p>

StaelTchinda: <p>Any update on this question.</p>
<p>I’ve tried to delete the forward pre hook created during pruning, but it is insufficient. The weight mask still exists.</p>

Cola_Lightyear: <p>The only way working for me is to save the model before pruning and reloading it afterwards</p>

Cola_Lightyear: <p>I appear to have success doing the following (I have two sequential submodels encoder and decoder in my model; adjust it to your needs):</p>
<pre><code class="lang-auto">def RemovePruningMask(model):
    encoder = model.encoder
    decoder = model.decoder

    for lrs in encoder:
        if hasattr(lrs, 'weight_mask'):
            lrs.weight_mask = torch.ones_like(lrs.weight_mask)

    for lrs in decoder:
        if hasattr(lrs, 'weight_mask'):
            lrs.weight_mask = torch.ones_like(lrs.weight_mask)
</code></pre>
<p>This appears to replace the weight mask successfully, that is, the mask is all ones afterwards and previous zero values of the mask are turned into ones.</p>
What is the best way to sample batches from different datasets in sequence in DDP

LY_yang: <p>I have many datasets that I would like to train jointly but with different shapes. More specifically, for each batch, I would like to have samples only from one dataset. Samples in each batch are sampled randomly from their belonging datasets. Different datasets are sampled in sequence to form batches. In short, the training looks like this (taking 3 datasets for illustration): [batch 1 from dataset 1]-[batch 2 from dataset 2]-[batch 3 from dataset 3]-[batch 4 from dataset 1]- etc…<br>
On the Internet, I found two commonly used approaches to achieve this: either constructing dataloaders per dataset or writing a custom batch sampler. I implemented both and performed some simple experiments, and I found that the two approaches do not seem super optimized for DDP training with multiple nodes and GPUs. As when I tested the performance, I found the wall time for each batch during DDP training is significantly larger than training on one GPU (eg., 0.8s vs 0.3s). I suspect the issue may come from CPU and data IO, since data are loaded on the fly and to optimize the IO time, I set the num_workers for each dataloader to be 4; and in the batch sampler approach, I set the num_replicas for each distributed_sampler also to be 4. However, when I have many datasets (&gt;10), the number of processes I create will be larger than the number of available CPU cores, and it causes a huge overhead.</p>
<p>I wonder if my reasoning makes sense here. If so, is there a recommended way to fix it, or maybe just a better approach to achieve my desired multi-dataset sampling behavior?</p>
<p>Many thanks.</p>
May i call collate_fn within module.forward instead of dataloader

crazyboy9103: <p>Hello, I’m wondering whether it is appropriate to collate inputs within the model, instead of the standard procedure, supplying collate_fn to dataloader.<br>
The reason is that for a detection or segmentation model, it is better to keep inputs as List[Tensor] instead of Tensor for multi-scale images because we need the original shapes for each individual image for post-processing (e.g. clipping). The standard procedure for such case that i know of is to pad the batch images by collate_fn, and wrap the images with a structure that holds the tensor and image shapes together, like <a href="https://github.com/pytorch/vision/blob/main/torchvision/models/detection/image_list.py" rel="noopener nofollow ugc">ImageList</a>. But such custom structure as an input to module.forward is not permitted for onnx export or tracing, which prevents deployment.<br>
Or i can input images:Tensor, image_sizes: List[Tuple[int, int]] to module.forward, but it seems redundant in deployment point of view.<br>
A simple solution that i could think of is to keep inputs as List[Tensor], and collate within the model, not from dataloader, which allows onnx export.</p>
<p>What are the drawbacks of such workaround and is there any other solution for this problem?<br>
Any help would be very much appreciated!</p>
_has_torch_function already has a docstring

3di: <p>To avoid issues with conflicting versions of torch, I’m doing the following:</p>
<p>remove conflicting versions:</p>
<pre><code class="lang-auto">def backupmodules_and_remove():
    # Store original modules in a global variable
    old_modules = {}
    for m in ['torch','PIL']:
        for module_name in list(sys.modules):
            if module_name == m or module_name.startswith(m + "."):
                # Store the current module in a dictionary
                old_modules[module_name] = sys.modules[module_name]
                # Remove the current module
                del sys.modules[module_name]
</code></pre>
<p>import correct versions after adding my module folder to the top of sys.path and invalidating caches with importlib:</p>
<pre><code class="lang-auto">def import_addon_modules():
    try:
        for m in ['torch','PIL']:
            importlib.import_module(m)
    except Exception as e:
        print('\nEXCEPTION IN IMPORT MODULES\n')
        print(str(e))
</code></pre>
<p>import_addon_modules throws this error:  <code>function '_has_torch_function' already has a docstring</code></p>
<p>I can’t use a virtual environment because I’m developing a Blender addon, hence the need to remove incorrect versions prior to my code running, and then putting it all back so other addons can continue to use the versions they require.</p>
<p>This is with the current torch version (cpu), and python 3.11.7 inside of Blender.</p>
How to see distribution of labels in dataset

flummery: <p>I would like to see how many of each label is in this random subset. When I type this it gives me an error:</p>
<blockquote>
<p>print(dict(Counter(train_dataset_subset.targets)))</p>
</blockquote>
<p>Below is my code:</p>
<pre><code class="lang-auto"># import the required modules
import torch
import torchvision
from torchvision.datasets import CIFAR10
from collections import Counter

trainset = CIFAR10(root='./data', train=True, download=True, transform=torchvision.transforms.ToTensor())

subset_size = 3000
train_dataset_subset = torch.utils.data.random_split(trainset, [subset_size, len(trainset)-subset_size])[0]
</code></pre>
<p>How do I go about solving this? Is there a way to use access trainset.targets when it is wrapped in torch.utils.data.random_split?</p>

ptrblck: <p>Yes, you could access the internal <code>.dataset</code> attribute:</p>
<pre><code class="lang-python">Counter(train_dataset_subset.dataset.targets)
</code></pre>

flummery: <p>When I try that it gives me the following:</p>
<blockquote>
<p>Counter({0: 5000, 1:5000, 2 : 5000, … , 9 : 5000})</p>
</blockquote>
<p>However I need the train_dataset_subset to return something like this:</p>
<blockquote>
<p>Counter ({ 0: 300, 1: 300, 2: 300, … , 9: 300})</p>
</blockquote>
<p>The reason it is 300, is because it is 3000 split over the 10 labels. How would I go about doing this? I specifically want the subset, not the original dataset.</p>

ptrblck: <p>You won’t be able to directly access it and would need to iterate the <code>Subset</code> since the samples will be created from the passed <code>indices</code> while the samples are drawn.</p>

qomhmd: <p>For anyone who visits this page…</p>
<p>The <code>dataset</code> attribute gives me the original dataset before splitting, no matter I get it from validation or train subsets. I used the <code>indices</code>, as <em>ptrblck</em> has briefly mentioned, to come up with this code:</p>
<pre><code class="lang-auto">original_ds.targets[train_ds.indices].value_counts()
original_ds.targets[val_ds.indices].value_counts()
</code></pre>
<p>Consider that you should replace dataset names, as well as <code>targets</code> based on what you named them in your custom dataset class.</p>
Script runs on v5p-8 but gets stucked on xmp.spawn on v5p-32

Jakub_Skrajny: <p>Hi, I struggle with running scripts on any accelerator type that has more than 8 TPU cores. I wonder if it is related to the flags like --internal-ips or --tunnel-through-iap</p>
<p>gcloud compute tpus tpu-vm create ${TPU_NAME} <br>
–project ${PROJECT_ID} <br>
–zone ${ZONE} <br>
–accelerator-type ${ACCELERATOR_TYPE} <br>
–version ${RUNTIME_VERSION} <br>
–subnetwork projects/${PROJECT_ID}/regions/${REGION}/subnetworks/${SUBNET_NAME}<br>
–internal-ips</p>
<p>gcloud alpha compute tpus tpu-vm ssh ${TPU_NAME} <br>
–project ${PROJECT_ID} --zone ${ZONE} --worker=all --tunnel-through-iap <br>
–command='…</p>
Inefficient broadcasting in pytorch

Joel_Pasapera: <p>I made a broadcasting but this consumes extremely high resources on my computer that even when running it in the cloud the server restarts for lack of resources, and my computer froze several times when trying to run the code. The broadcasting I did in pytorch in python, this is to compare whether it is true or false a row tensor with a column tensor. for example entry_time_boolean = entry_time &lt;= date . where entry_time has 1 dimension and date has 2 dimensions</p>

ptrblck: <p>What are the sizes of the used tensor and their <code>dtype</code>?</p>