multiple dataset to train model. Sharing backbone, Separable classifier.
What is your question?
I want to use two different dataloader to train model.
According to this two dataset, backbone network is shared and classifier is classified.
How should i?
In training_step, i expect "batch" is list containing two dataloader but there is just one dataloader.
Code
def forward(self, x1, x2):
    x1 = self.backbone(x1)
    x2 = self.backbone(x2)

    x1 = self.avgpool1(x1)
    x1 = x1.reshape(x1.size(0), -1)
    x1 = self.fc1(x1)
    x1 = self.classifier1(x1)

    x2 = self.avgpool2(x2)
    x2 = x2.reshape(x2.size(0), -1)
    x2 = self.fc2(x2)
    x2 = self.classifier2(x2)

    return x1, x2

def training_step(self, batch, batch_idx):
    x1, y1 = batch[0]
    x2, y2 = batch[1]
    pred1, pred2 = self(x1), self(x2)
    loss = self.loss_function(pred1, y1) + self.loss_function(pred2, y2)

    return {'loss': loss}

def train_dataloader(self):
    img_path = '/home/choieq/pet_recog/data/Useful/Image_dataset/Training_aug2_400/'
    video_path = '/home/choieq/pet_recog/data/Useful/video/total_video/train'
    img_train_dataset = KistdogDataset(root_path=img_path, phase='train', resize=224)
    video_train_dataset = KistdogDataset(root_path=video_path, phase='train', resize=224)

    img_train_loader = DataLoader(img_train_dataset, batch_size=256, shuffle=True, num_workers=10, pin_memory=True)
    video_train_loader = DataLoader(video_train_dataset, batch_size=256, shuffle=True, num_workers=10, pin_memory=True)
    #loaders = {'image': img_train_loader, 'video': video_train_loader}
    return [img_train_loader, video_train_loader]

What have you tried?
What's your environment?

OS: [ubuntu 18.04]
Packaging [conda]
Version [torch:1.7, pytorch-lightning:0.9.1rc2]