Dataset object has no attribute map

cvclpl: <p>Hello,<br>
I create a dataset object with the purpose to use map. However, when I try to call the function map, I get the error that the object has no attribute map.<br>
Any ideas how to fix this?</p>
<pre><code class="lang-auto">class MyDataset(Dataset):
    def __init__(self, text, tags):
        self.sentence = text
        self.tags_per_token = tags

    def __getitem__(self, idx):
        sentence = self.sentence[idx]
        tags_per_token = self.tags_per_token[idx]
        #return one_text, one_label
        return {"text_words": one_text,"tags":tags_per_token}

    def __len__(self):
        return len(self.tags_per_token)
      
dataset_train = MyDataset(train_data, train_tags)
dataset_train.map(
    my_function,
    batched=True)

AttributeError: 'MyDataset' object has no attribute 'map'
</code></pre>

ptrblck: <p>This seems to be the same issue as described <a href="https://discuss.pytorch.org/t/map-method-doesnt-work-with-custom-dataset/150066/2">here</a> (unfortunately without a follow up).<br>
Could you describe where you’ve seen the <code>.map</code> method applied on <code>torch.utils.data.Dataset</code> as it’s not a built-in method?</p>

cvclpl: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
Thanks for your reply  - I was following the huggingface dataset preparation in</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://huggingface.co/course/chapter7/2?fw=pt">
  <header class="source">

      <a href="https://huggingface.co/course/chapter7/2?fw=pt" target="_blank" rel="noopener nofollow ugc">huggingface.co</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/372;"><img src="https://huggingface.co/front/thumbnails/course.png" class="thumbnail" width="690" height="372"></div>

<h3><a href="https://huggingface.co/course/chapter7/2?fw=pt" target="_blank" rel="noopener nofollow ugc">Token classification - Hugging Face Course</a></h3>

  <p>We’re on a journey to advance and democratize artificial intelligence through open source and open science.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Perhaps a huggingface dataset is different to a pytorch dataset.</p>

ptrblck: <p>Yes, this seems to be the case so you might want to ask in the HF discussion board.</p>

cvclpl: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a> ! that cleared out the confusion</p>

specteross: <p><a class="mention" href="/u/cvclpl">@cvclpl</a> <a class="mention" href="/u/ptrblck">@ptrblck</a> Seeing the exact same issue. How did you solve it?</p>

nivek: <p>PyTorch’s built-in <code>Dataset</code> doesn’t supports <code>.map()</code> as an operation. If you would like that feature, please use <a href="https://pytorch.org/data/beta/torchdata.datapipes.iter.html#torchdata.datapipes.iter.IterDataPipe" rel="noopener nofollow ugc">DataPipe</a> from <a href="https://github.com/pytorch/data" rel="noopener nofollow ugc"><code>TorchData</code></a>.</p>
<p>Please note that the built-in PyTorch <code>Dataset</code> is not the same as the one provided by Hugging Face.</p>