Train and Test a GAN model

Najeh_Nafti: <p>for a GAN model, I have an image dataset folder splitted between train, test and validation sub-folders. In each sub-folder 4 classes (1 Normal and 3 abnormal patients). How can I split this dataset to train my model only on normal patients and test the model on unseen normal and abnormal patients?</p>

ptrblck: <p>Depending how you are currently loading the data, you could use the targets (assuming you are already creating them beforehand), use the target indices to filter out the desired classes, and pass them to different <code>Subset</code>s.</p>

Najeh_Nafti: <p>Below you can find my code. How can I create the targets for this dataset ?</p>
<pre><code class="lang-auto">data_root = "./data/OCT2017/"

train_transform = transforms.Compose([
                                  transforms.Resize(img_size),
                                  transforms.CenterCrop(img_size),
                                  transforms.Grayscale(),
                                  transforms.RandomHorizontalFlip(),
                                  transforms.ToTensor(),
                                  transforms.Normalize([0.5], [0.5]) 
                              ])



test_transform = transforms.Compose([
                                  transforms.Resize(img_size),
                                  transforms.CenterCrop(img_size),
                                  torchvision.transforms.Grayscale(),
                                  transforms.ToTensor(),
                                  transforms.Normalize([0.5], [0.5])
                              ])


train_data = dataset.ImageFolder(root = data_root+ 'train', transform= train_transform)
test_data = dataset.ImageFolder(root = data_root + 'test', transform= test_transform)

trainloader = torch.utils.data.DataLoader(train_data, batch_size=batch_size, shuffle=True)                                
testloader = torch.utils.data.DataLoader(test_data, batch_size=1,shuffle=True)  
                           
</code></pre>

ptrblck: <p>You can access the <code>.targets</code> attribute in <code>train_data.targets</code> and <code>test_data.targets</code>.</p>

Najeh_Nafti: <p>is the below code correct? and then how can I load the data ?</p>
<pre><code class="lang-auto">x_train = train_data[train_data.targets == 1]
y_train = train_data.targets[train_data.targets==1]

x_test = test_data[test_data.targets != 1]
y_test = test_data.targets[test_data.targets!=1]
</code></pre>

ptrblck: <p>Not quite. Here is an example how to use the internal <code>targets</code> attribute to filter out samples and create a <code>Subset</code> with only one class:</p>
<pre><code class="lang-python">class MyDataset(Dataset):
    def __init__(self):
        self.data = torch.randn(100, 1)
        self.targets = torch.randint(0, 2, (100,))
        
    def __getitem__(self, index):
        x = self.data[index]
        y = self.targets[index]
        return x, y
    
    def __len__(self):
        return len(self.data)

dataset = MyDataset()
print(dataset.targets.unique(return_counts=True))
&gt; (tensor([0, 1]), tensor([55, 45]))

# Filter out class1 and get indices for all samples corresponding to class1
idx = (dataset.targets == 1).nonzero()
print('{} class1 samples found'.format(len(idx)))
&gt; 45 class1 samples found

dataset_class1 = Subset(dataset, indices=idx)
loader = DataLoader(dataset_class1)

for data, target in loader:
    print(target)

&gt; tensor([[1]])
  tensor([[1]])
  tensor([[1]])
  ..
</code></pre>

Najeh_Nafti: <p>And then  target data how can be used during training?</p>

ptrblck: <p>You could use the data as in any other training script. I’m not familiar with your use case and thus don’t know what you are exactly trying to achieve, as “training” with a single class is usually not working.<br>
Your model would in this case only predict the single class and I don’t think it can learn anything else (but I think you’ve explained your approach in your other topics in more details).</p>