How can i create a single dataloader for my two csv files . I have tried this

username: sam101
cooked: <p>data_file1 = “/Data_train_A.csv”<br>
rnaseq = pd.read_csv(data_file1, index_col=0, header=0)<br>
rnaseq_tensor1 = torch.FloatTensor(rnaseq.values)<br>
<span class="hashtag">#print</span>(rnaseq.shape)</p>
<p>data_file2 = “/Data_train_B.csv”<br>
rnaseq = pd.read_csv(data_file2, index_col=0, header=0)<br>
rnaseq_tensor2 = torch.FloatTensor(rnaseq.values)<br>
<span class="hashtag">#print</span>(rnaseq.shape)</p>
<p>dataset = TensorDataset(rnaseq_tensor1,rnaseq_tensor2)<br>
dataloader = DataLoader(dataset,batchsize=2)</p>
<p>for batch_idx, (a,b) in enumerate(dataloader):<br>
print(a.shape, b.shape)</p>

username: ptrblck
cooked: <p>What does the <code>print</code> statement output and what is your expectation? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: dhpollack
cooked: <p>Do you want this as one big list or do you want them parallel?</p>

username: sam101
cooked: <p>i want them as an input to my cycleGan, where we have data from domain A and domain B</p>

username: sam101
cooked: <p>i want them as an input to my cycleGan, where we have data from domain A and domain B. It is used for mapping from domain A to B . So my inputs are both ata time to cycleGan</p>

username: ptrblck
cooked: <p>And do you get any error using this approach?<br>
We would need the error message or some more information on what’s not working to help debugging. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: sam101
cooked: <p>yeah it shows me this</p>
<hr>
<p>AssertionError                            Traceback (most recent call last)</p>
<p> in ()<br>
10 <span class="hashtag">#print</span>(rnaseq.shape)<br>
11<br>
—&gt; 12 dataset = TensorDataset(rnaseq_tensor1,rnaseq_tensor2)<br>
13 dataloader = DataLoader(dataset,batchsize=2)<br>
14</p>
<p>/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataset.py in <strong>init</strong>(self, *tensors)<br>
156<br>
157     def <strong>init</strong>(self, *tensors):<br>
–&gt; 158         assert all(tensors[0].size(0) == tensor.size(0) for tensor in tensors)<br>
159         self.tensors = tensors<br>
160</p>
<p>AssertionError:</p>

username: ptrblck
cooked: <p>This error points to different lengths of both input tensors.<br>
<code>TensorDataset</code> would return samples from both inputs using the same index internally.<br>
If one tensor contains more samples (dim0 is used for indexing), this error is thrown.</p>
<p>You could e.g. crop the larger tensor to the same length as the smaller one or duplicate the smaller one.<br>
If you want to apply some more complicated sampling strategy, I would recommend to write a custom <code>Dataset</code> and create the pairs in <code>__getitem__</code>.</p>

username: sam101
cooked: <p>means i have to create separate dataloaders then</p>

username: ptrblck
cooked: <p>No. You should pass the same amount of samples or create some correspondence between both tensors in your custom <code>Dataset</code> (or <code>collate_fn</code> etc.).</p>

username: sam101
cooked: <p>could you please provide me an example</p>

username: ptrblck
cooked: <p>Here is a small example to reproduce this error and how to slice the larger tensor:</p>
<pre><code class="lang-python"># Works, since a and b have the same length
a = torch.arange(10).view(-1, 1)
b = torch.arange(10).view(-1, 1)

dataset = TensorDataset(a, b)
loader = DataLoader(
    dataset,
    batch_size=5
)

for idx, (data1, data2) in enumerate(loader):
    print('Idx ', idx)
    print(data1)
    print(data2)

# Use different lengths
a = torch.arange(10).view(-1, 1)
b = torch.arange(20).view(-1, 1)

dataset = TensorDataset(a, b) # fails
# Slice b to have the same length
dataset = TensorDataset(a, b[:a.size(0)])
loader = DataLoader(
    dataset,
    batch_size=5
)

for idx, (data1, data2) in enumerate(loader):
    print('Idx ', idx)
    print(data1)
    print(data2)
</code></pre>
<p>Note that this approach might not be the best for your use case, so you should apply your method to yield corresponding pairs of both input tensors.</p>

username: sam101
cooked: <p>Thanks it really helped. Thanks again</p>