Train two Dataset together in Capsule Network

Md_Mahfujur_Rahman_0: <p>I am working with this code: <a href="https://github.com/AlexHex7/CapsNet_pytorch" rel="nofollow noopener">https://github.com/AlexHex7/CapsNet_pytorch</a></p>
<p>here the training images come from train_loader. I want train the images from valid_loader at the same time. How can I do that?</p>
<p>train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=cfg.batch_size, shuffle=True)<br>
valid_loader = torch.utils.data.DataLoader(valid_dataset, batch_size=cfg.batch_size, shuffle=True)</p>
<p>for epoch_index in range(cfg.epoch):<br>
for train_batch_index, (img_batch, label_batch) in enumerate(train_loader):<br>
img_batch = variable(img_batch)<br>
label_batch = variable(label_batch).unsqueeze(dim=1)<br>
predict, reconstruct_img = net(img_batch, label_batch, train=True)</p>
<pre><code>    acc = net.calc_acc(predict, label_batch)
    margin_loss = net.margin_loss(predict, label_batch)
    reconstruct_loss = net.reconstruction_loss(img_batch, reconstruct_img)      
    loss = margin_loss + reconstruct_loss 
    net.zero_grad()
    loss.backward()
    opt.step()</code></pre>

Md_Mahfujur_Rahman_0: <p>Actually my intension is like that</p>
<p>for train_batch_index, (img_batch_v, label_batch_v) in enumerate(valid_loader):</p>
<p>so that i can get the valuse of img_batch_v and label_batch_v as well as  img_batch and label_batch</p>

ptrblck: <p>It depends if you need to know if a sample is from the train or validation data set.<br>
If it doesn’t matter, the most elegant solution might be to use a <code>ConcatDataset</code>.<br>
Using this approach you can just concatenate both data sets into a new one. The <code>DataLoader</code> might take care of shuffling if needed.</p>
<p>Here is a small example:</p>
<pre><code>class MyData(Dataset):
    def __init__(self):
        self.data = torch.randn(100, 1)
        self.target = torch.Tensor(100).uniform_(0, 10).long()
    
    def __getitem__(self, index):
        x = self.data[index]
        y = self.data[index]
        return x, y

    def __len__(self):
        return len(self.data)

train_dataset = MyData()
val_dataset = MyData()
train_val_dataset = ConcatDataset([train_dataset, val_dataset])

for batch_idx, (data, target) in enumerate(train_val_dataset):
    print(batch_idx)
</code></pre>
<p>However, if you need to call both data sets separately, you could zip both sets in the for loop.<br>
One drawback of this approach (at least in this simple form) is that both datasets must have the same length, since the loop is iterating both sets in one loop.<br>
Here is a small modification to the aforementioned code:</p>
<pre><code>train_loader = DataLoader(train_dataset)
val_loader = DataLoader(val_dataset)

for batch_idx, (train_batch, val_batch) in enumerate(zip(train_dataset, val_dataset)):
    print(batch_idx)
    data, target = train_batch[0], train_batch[1]
    val, target_val = val_batch[0], val_batch[1]
</code></pre>
<p>I would prefer the first approach, if it fits your needs <img src="//discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Md_Mahfujur_Rahman_0: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> thank you so much for your kind reply.</p>
<p>Then after training how can I get the values.</p>
<p>After training, I need to get the values for source_data1 and source_data2. For an example,</p>
<p>img_batch ?<br>
label_batch?</p>
<p>for both source .</p>

Md_Mahfujur_Rahman_0: <p>In caffe we are using slice to make separating. Is there any function that we can use?</p>
<p>layer {<br>
name: "fc7_slice"<br>
type: "Slice"<br>
bottom: "fc7_grl"<br>
top: "source_fc7_1"<br>
top: "target_fc7_1"<br>
slice_param {<br>
axis: 0<br>
}<br>
include: { phase: TRAIN }<br>
}</p>

Md_Mahfujur_Rahman_0: <p>for epoch_index in range(cfg.epoch):</p>
<p>for train_batch_index, (img_batch, label_batch) in enumerate(train_loader):<br>
img_batch = variable(img_batch)<br>
label_batch = variable(label_batch).unsqueeze(dim=1)</p>
<pre><code> img_batcg_s = torch.slice(img_batch)
 img_batcg_t = torch.slice(img_batch)

May I use this **torch.slice** to do this?</code></pre>

ptrblck: <p>I am not sure, why you were using <code>slice</code> in caffe.</p>
<p>After the training you can get the data and target from both datasets either by calling</p>
<pre><code>data, target = train_dataset[0]
data_val, target_val = val_dataset[0]
</code></pre>
<p>or by wrapping it again in a <code>DataLoader</code> and iterate it.</p>
<p>I’m not sure, but it seems the caffe <code>slice</code> layer separates a blob along an axis.<br>
You don’t need it in this case.<br>
Also <code>torch.slice</code> does not exist. You can use <code>narrow</code> however.</p>
<p>Could you explain your use case a little bit more? I’m not sure I understand it correctly.</p>

Md_Mahfujur_Rahman_0: <p>The original code is:<br>
train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=cfg.batch_size, shuffle=True)</p>
<p>for epoch_index in range(cfg.epoch):<br>
for train_batch_index, (img_batch, label_batch) in enumerate(train_loader):<br>
img_batch = variable(img_batch)<br>
label_batch = variable(label_batch).unsqueeze(dim=1)<br>
predict, reconstruct_img = net(img_batch, label_batch, train=True)</p>
<p>acc = net.calc_acc(predict, label_batch)<br>
margin_loss = net.margin_loss(predict, label_batch)<br>
reconstruct_loss = net.reconstruction_loss(img_batch, reconstruct_img)<br>
loss = margin_loss + reconstruct_loss<br>
net.zero_grad()<br>
loss.backward()<br>
opt.step()</p>
<p>The original code take data from one source. My intension is to take data from two sources and during training I will add a custom loss in between two sources.<br>
For this reason, I need train data from two sources. For an example on is from train folder another one is from valid folder.</p>

ptrblck: <p>Ok, since you need them simultaneously, I would suggest my 2nd approach (zipping both <code>DataLoaders</code>).</p>
<pre><code>for batch_idx, (train_batch, val_batch) in enumerate(zip(train_loader, val_loader)):
    data, target = train_batch[0], train_batch[1]
    val, target_val = val_batch[0], val_batch[1]

    # Forward call
    ....
    # Loss calculation
    acc = ...
    your_custom_loss = ...
</code></pre>
<p>Will this meet your needs?</p>

Md_Mahfujur_Rahman_0: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank you so much. I think it will work.</p>
<p>My original code is:</p>
<p>for epoch_index in range(cfg.epoch):<br>
for train_batch_index, (img_batch, label_batch) in enumerate(train_loader):<br>
img_batch = variable(img_batch)<br>
label_batch = variable(label_batch).unsqueeze(dim=1)<br>
predict, reconstruct_img = net(img_batch, label_batch, train=True)<br>
acc = net.calc_acc(predict, label_batch)<br>
margin_loss = net.margin_loss(predict, label_batch)<br>
reconstruct_loss = net.reconstruction_loss(img_batch, reconstruct_img)<br>
loss = margin_loss + reconstruct_loss<br>
net.zero_grad()<br>
loss.backward()<br>
opt.step()</p>
<p>My new code is:</p>
<p>for epoch_index in range(cfg.epoch):</p>
<p>for train_batch_index, (img_batch, label_batch) in enumerate (zip (train_loader, valid_loader )):<br>
img_batch, label_batch = img_batch[0], img_batch[1]<br>
img_batch_v, label_batch_v = label_batch[0], label_batch[1]</p>
<pre><code> img_batch = variable(img_batch)
 label_batch = variable(label_batch).unsqueeze(dim=1)

 img_batch_v = variable(img_batch_v)
 label_batch_v = variable(label_batch_v).unsqueeze(dim=1)

 predict, reconstruct_img = net(img_batch, label_batch, train=True)

 acc = net.calc_acc(predict, label_batch)
 margin_loss = net.margin_loss(predict, label_batch)
 reconstruct_loss = net.reconstruction_loss(img_batch, reconstruct_img) 
 Custom_loss = custom_loss(img_batch, img_batch_v)  # img_batch from 1 source and img_batch_v for another

 loss = margin_loss + reconstruct_loss + Custom_loss
 net.zero_grad()
 loss.backward()
 opt.step()
</code></pre>
<p>But got an error.</p>

Md_Mahfujur_Rahman_0: <p>for epoch_index in range(cfg.epoch):</p>
<p>for train_batch_index, (img_batch, label_batch) in enumerate (zip (train_loader, valid_loader )):</p>
<pre><code> img_batch, label_batch = img_batch[0], img_batch[1]
 img_batch_v, label_batch_v = label_batch[0], label_batch[1]

 img_batch = variable(img_batch)
 label_batch = variable(label_batch).unsqueeze(dim=1)

 img_batch_v = variable(img_batch_v)
 label_batch_v = variable(label_batch_v).unsqueeze(dim=1)

 predict, reconstruct_img = net(img_batch, label_batch, train=True)
 acc = net.calc_acc(predict, label_batch)
 margin_loss = net.margin_loss(predict, label_batch)
 reconstruct_loss = net.reconstruction_loss(img_batch, reconstruct_img) 
 Custom_loss = custom_lossl(img_batch, img_batch_v)

 loss = margin_loss + reconstruct_loss +Custom_loss
 net.zero_grad()
 loss.backward()
 opt.step()
</code></pre>
<p>But got the error:</p>
<p>File “main_mahfuj.py”, line 95, in <br>
img_batch_v = variable(img_batch_v)<br>
File “/media/mahfuj/DATA/New_CODE/Working/CapsNet_pytorch/lib/cuda_utils.py”, line 7, in variable<br>
return Variable(tensor, volatile=volatile).cuda()<br>
RuntimeError: Variable data has to be a tensor, but got int</p>
<p>I got the value of img_batch but I did not get the value of img_batch_v.</p>
<p>Thanks in advanced.</p>

ptrblck: <p>You mixed up the variable names a bit.<br>
I would change <code>img_batch</code> and <code>label_batch</code> to <code>train_batch</code> and <code>val_batch</code>, respectively.</p>
<p>Could you check, what both loaders are returning, i.e. the shape of both tensor inside the tuple?</p>

Md_Mahfujur_Rahman_0: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank you so much for your kind help.</p>
<p>for epoch_index in range(cfg.epoch):</p>
<p>for train_batch_index, (img_batch, label_batch) in enumerate (zip (train_loader, valid_loader )):</p>
<p>img_batch, label_batch = img_batch[0], img_batch[1]<br>
img_batch_v, label_batch_v = label_batch[0], label_batch[1]</p>
<p>img_batch = variable(img_batch)<br>
label_batch = variable(label_batch).unsqueeze(dim=1)</p>
<p>img_batch_v = variable(img_batch_v)<br>
label_batch_v = variable(label_batch_v).unsqueeze(dim=1)</p>
<p>predict, reconstruct_img = net(img_batch, label_batch, train=True)<br>
acc = net.calc_acc(predict, label_batch)<br>
margin_loss = net.margin_loss(predict, label_batch)<br>
reconstruct_loss = net.reconstruction_loss(img_batch, reconstruct_img)<br>
Custom_loss = custom_lossl(img_batch, img_batch_v)</p>
<p>loss = margin_loss + reconstruct_loss +Custom_loss<br>
net.zero_grad()<br>
loss.backward()<br>
opt.step()</p>
<p>Results:<br>
img_batch size: (28, 28)</p>
<p>But did not get the value of img_batch_v.</p>
<p>torch.Size([28, 28])<br>
Traceback (most recent call last):<br>
File “main_mahfuj.py”, line 91, in <br>
print(img_batch_v[0][0].shape)<br>
TypeError: ‘int’ object is not subscriptable</p>

ptrblck: <p>You still have some errors with the sizes.<br>
Please try <code>print(img_batch_v.shape)</code> (without the indexing).<br>
Also it seems your naming is still a bit confusing. <img src="//discuss.pytorch.org/images/emoji/apple/wink.png?v=5" title=":wink:" class="emoji" alt=":wink:"></p>

Md_Mahfujur_Rahman_0: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank you so much.</p>
<p>Now I got it. I have changed the naming and its working. Thank you so much for your kind help. I really appreciate it.</p>

ptrblck: <p>I’m glad it’s working now! <img src="//discuss.pytorch.org/images/emoji/apple/wink.png?v=5" title=":wink:" class="emoji" alt=":wink:"></p>

Md_Mahfujur_Rahman_0: <p><a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>for epoch_index in range(cfg.epoch):<br>
for batch_idx, (train_batch, val_batch) in enumerate(zip(train_loader, valid_loader)):<br>
data, target = train_batch[0], train_batch[1]<br>
val, target_val = val_batch[0], val_batch[1]<br>
img_batch = variable(data)<br>
label_batch = variable(target).unsqueeze(dim=1)<br>
img_batch_v = variable(val)<br>
label_batch_v = variable(target_val).unsqueeze(dim=1)</p>
<p>For my understanding,  data and target take the data and label form <strong>train_loader</strong> whereas val and target_val take the value of data and label from <strong>valid_loader.</strong> Is that correct?</p>
<p>I am facing the problem that train_loader contains 60000 images and valid_loader contains 10000 images.<br>
I think need all the images to take from train_loader and valid_loader.</p>

ptrblck: <p>You are right in your understanding.</p>
<p>Yeah, that’s the issue I mentioned in my first post.</p>
<aside class="quote no-group" data-username="ptrblck" data-post="3" data-topic="12298">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>One drawback of this approach (at least in this simple form) is that both datasets must have the same length, since the loop is iterating both sets in one loop.</p>
</blockquote>
</aside>
<p>If I understand you correctly, you need a pair of training and validation data.<br>
So your training data will be repeated (in your example 6 times) for each sample of the validation data.<br>
Is that correct?</p>
<p>You could artificially make the validation <code>Dataset</code> larger with this somewhat ugly hack:</p>
<pre><code class="lang-auto">class MyDataTrain(Dataset):
    def __init__(self, length):
        self.data = torch.randn(length, 1)
        self.target = torch.Tensor(length).uniform_(0, 10).long()
    
    def __getitem__(self, index):
        x = self.data[index]
        y = self.data[index]
        return x, y

    def __len__(self):
        return len(self.data)

class MyDataVal(Dataset):
    def __init__(self, length, fake_length):
        self.data = torch.randn(length, 1)
        self.target = torch.Tensor(length).uniform_(0, 10).long()
        self.fake_length = fake_length
        self.real_length = len(self.data)
    
    def __getitem__(self, index):
        index = index % self.real_length
        x = self.data[index]
        y = self.data[index]
        return x, y

    def __len__(self):
        return self.fake_length

train_dataset = MyDataTrain(length=60000)
val_dataset = MyDataVal(length=10000, fake_length=60000)

train_loader = DataLoader(train_dataset)
val_loader = DataLoader(val_dataset)

for batch_idx, (train_batch, val_batch) in enumerate(zip(train_loader, val_loader)):
    # your code here
</code></pre>
<p>Note the modulo operation in <code>__getitem__</code> of the validation <code>Dataset</code>.<br>
It will give <code>fake_length</code> samples, iterating over the same smaller dataset.</p>
<p>Can you work with this?</p>

Md_Mahfujur_Rahman_0: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank you so much for your kind help and extraordinary explanation. Yes, I need to make the validation Dataset larger artificially. Then it will serve my purpose.</p>
<pre><code class="lang-auto">transform = transforms.Compose([
                    transforms.Resize((28,28)),
                    transforms.Grayscale(num_output_channels=1),
                    transforms.ToTensor(),      
                    #transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
                    transforms.Normalize((0.1307,), (0.3081,))])
train_dataset = datasets.SVHN('SVHN', download=True, transform=transform, split='train')
valid_dataset = datasets.MNIST('MNIST', download=True, transform=transform, train=True)
class MyDataTrain(Dataset):
    def __init__(self, length):
        self.data = torch.randn(length, 1)
        self.target = torch.Tensor(length).uniform_(0, 10).long()
    def __getitem__(self, index):
        x = self.data[index]
        y = self.data[index]
        return x, y

    def __len__(self):
        return len(self.data)

class MyDataVal(Dataset):
    def __init__(self, length, fake_length):
        self.data = torch.randn(length, 1)
        self.target = torch.Tensor(length).uniform_(0, 10).long()
        self.fake_length = fake_length
        self.real_length = len(self.data)
    
    def __getitem__(self, index):
        index = index % self.real_length
        x = self.data[index]
        y = self.data[index]
        return x, y

    def __len__(self):
        return self.fake_length
train_dataset = MyDataTrain(length=60000)
val_dataset = MyDataVal(length=10000, fake_length=60000)
train_loader = DataLoader(train_dataset)
val_loader = DataLoader(val_dataset)
</code></pre>
<p>Gives me error.</p>
<pre><code class="lang-auto">NameError: name 'Dataset' is not defined
</code></pre>

ptrblck: <p>You have to import the <code>Dataset</code> class:</p>
<pre><code>from torch.utils.data import Dataset</code></pre>