Dataloader for Name Generator Tutorial

Daniel_Dsouza: <p>Hi Everyone! I’m trying to use a Data Loader in the Pytorch Name Generator Tutorial. I’ve written a simple version of the Dataset and Dataloader, but I get a slightly different output with the Dataloader.</p>
<pre><code class="lang-auto">from torch.utils.data import Dataset, DataLoader

class SampleDataset(Dataset):
    def __init__(self):
        temp = {'English':['Adam','Bill','Chet']}
        categories = list(temp.keys())
        self.data = [(list(name), list(name[1:])+['&lt;EOS&gt;'], category) for category in categories for name in temp[category]]
        
    def __getitem__(self,index):
        return self.data[index]
    
    def __len__(self):
        return len(self.data)

x = SampleDataset()
x_DataLoader = DataLoader(x, batch_size=1)

print(x[0])

for piece in x_DataLoader:
    print(piece)
    break
</code></pre>
<p>([‘A’, ‘d’, ‘a’, ‘m’], [‘d’, ‘a’, ‘m’, ‘’], ‘English’)</p>
<p>[[(‘A’,), (‘d’,), (‘a’,), (‘m’,)], [(‘d’,), (‘a’,), (‘m’,), (’’,)], (‘English’,)]</p>
<p>I get this weird tuple around each of my characters with the DataLoader. I don’t understand why <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>
<p>Am I missing something obvious ? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> Thanks so much!</p>

ptrblck: <p>Your <code>.data</code> is stored internally as:</p>
<pre><code class="lang-python">[(['A', 'd', 'a', 'm'], ['d', 'a', 'm', '&lt;EOS&gt;'], 'English'),
 (['B', 'i', 'l', 'l'], ['i', 'l', 'l', '&lt;EOS&gt;'], 'English'),
 (['C', 'h', 'e', 't'], ['h', 'e', 't', '&lt;EOS&gt;'], 'English')]
</code></pre>
<p>since you are creating a list of tuples of lists here:</p>
<pre><code class="lang-python">self.data = [(list(name), list(name[1:])+['&lt;EOS&gt;'], category) for category in categories for name in temp[category]]
</code></pre>
<p>How would you like to store the data instead?<br>
(I couldn’t find a <code>Dataset</code> code by skimming through the tutorial)</p>

Daniel_Dsouza: <p>Hey <a class="mention" href="/u/ptrblck">@ptrblck</a>, Thanks for responding! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> Yes, I create the tuples because the rest of my code uses it like that(it could also use lists so that can be changed). There is no Dataset code in the Tutorial ( I was indicating that I’m trying to take the tutorial further by using Datasets/Dataloaders)</p>
<p>My doubt is not around the fact that the tuple has become a list, it is more around why:</p>
<p>[‘A’,‘d’,‘a’,‘m’] become [(‘A’,), (‘d’,), (‘a’,), (‘m’,)]</p>
<p>Is this because of my tuple decision above ?</p>

ptrblck: <p>Thanks for the update!</p>
<p>The output “form” of your batches will be defined by the default collate function form the <code>DataLoader</code> as shown <a href="https://github.com/pytorch/pytorch/blob/bf860a4ebafbfcb75e61a8603bded72f6d0b0970/torch/utils/data/_utils/collate.py#L42-L81">here</a>.</p>
<p>You could thus pass in a custom collate function or change the data wrapping in your dataset.</p>
<p>Could you post how the desired batch should look like?</p>

Daniel_Dsouza: <p>Ah okay! This is exciting!<br>
So say I just want it in the same way :</p>
<pre><code class="lang-auto">[(['A', 'd', 'a', 'm'], ['d', 'a', 'm', '&lt;EOS&gt;'], 'English'),
 (['B', 'i', 'l', 'l'], ['i', 'l', 'l', '&lt;EOS&gt;'], 'English'),
 (['C', 'h', 'e', 't'], ['h', 'e', 't', '&lt;EOS&gt;'], 'English')]
</code></pre>
<p>Shouldn’t this be the default ? Actually maybe I get what you’re sayin. Is it like "Collate is where I’d get it ready for the model directly ? " Are they any resources you’d recommend ?</p>

ptrblck: <aside class="quote no-group" data-username="Daniel_Dsouza" data-post="5" data-topic="78324">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/daniel_dsouza/48/16622_2.png" class="avatar"> Daniel_Dsouza:</div>
<blockquote>
<p>Shouldn’t this be the default ? Actually maybe I get what you’re sayin. Is it like "Collate is where I’d get it ready for the model directly ?</p>
</blockquote>
</aside>
<p>I’m not sure I understand the question correctly.<br>
The collate function gets the samples from multiple calls into <code>Dataset.__getitem__</code> and creates a batch, which you will get in the <code>DataLoader</code> loop.<br>
If you are fine with the current shape, then there is no need to write a custom collate function.<br>
However, if you would like to change something,  have a look at <a href="https://pytorch.org/docs/stable/data.html#dataloader-collate-fn">this doc</a> about the <code>collate_fn</code>.</p>

Daniel_Dsouza: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a>, that actually answered my question <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> I know where to look now to change the format of my Dataloader batch.</p>