How can i use sklearn.Kfold with ImageFolder?

username: skyunyoo
cooked: <p>My code is…</p>
<pre><code class="lang-auto">batch_size = 16

transform = transforms.Compose([transforms.Resize((299,299))
                                       ,transforms.ToTensor()
                                       ,transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])

dataset = ImageFolder('.data/',transform=transform)

kf = KFold(n_splits=5, shuffle=True)

for i, (train_index, test_index) in enumerate(kf.split(dataset)):

    trainloader = torch.utils.data.DataLoader(train_index, batch_size=batch_size, shuffle=True, num_workers=0, pin_memory=False)
    testloader = torch.utils.data.DataLoader(test_index, batch_size=batch_size, shuffle=True, num_workers=0, pin_memory=False)
    
    print('Fold : {}, train : {}, test : {}'.format(i+1, len(trainloader.dataset), len(testloader.dataset)))
    
    for batch_idx, (data, target) in enumerate(trainloader):
        print('Train Batch idx : {}, data shape : {}, target shape : {}'.format(batch_idx, data.shape, target.shape))
</code></pre>
<p>error occurred.</p>
<pre><code class="lang-auto">
Fold : 1, train : 579, test : 145

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
&lt;ipython-input-6-0fe2bfb82b09&gt; in &lt;module&gt;
     16     print('Fold : {}, len train : {}, len test : {}'.format(i+1, len(trainloader.dataset), len(testloader.dataset)))
     17 
---&gt; 18     for batch_idx, (data, target) in enumerate(trainloader):
     19         print('Train Batch idx : {}, data shape : {}, target shape : {}'.format(batch_idx, data.shape, target.shape))

ValueError: too many values to unpack (expected 2)
</code></pre>
<p>I don’t know how to handle train_index, test_index.<br>
Could anyone give me a help ?</p>

username: ptrblck
cooked: <p><code>kf.split</code> will return the train and test indices as far as I know.<br>
Currently you are passing these indices to a <code>DataLoader</code>, which will just return a batch of indices.</p>
<p>I think you should pass the train and test indices to a <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.Subset" rel="nofollow noopener">Subset</a> to create new <code>Datasets</code> and pass these to the <code>DataLoaders</code>.<br>
Let me know, if that works for you.</p>

username: skyunyoo
cooked: <p>The code I modified is as follows:</p>
<pre><code class="lang-auto">for i, (train_index, test_index) in enumerate(kf.split(dataset)):
    
    train = torch.utils.data.Subset(dataset, train_index)
    test = torch.utils.data.Subset(dataset, test_index)

    trainloader = torch.utils.data.DataLoader(train, batch_size=batch_size, shuffle=True, num_workers=0, pin_memory=False)
    testloader = torch.utils.data.DataLoader(test, batch_size=batch_size, shuffle=True, num_workers=0, pin_memory=False)
</code></pre>
<p>Thank you for reply. It’s works well!<img src="https://discuss.pytorch.org/images/emoji/apple/grin.png?v=6" title=":grin:" class="emoji" alt=":grin:"></p>

username: skyunyoo
cooked: <p>Label has mismatched with pic.<br>
I have ‘0’, ‘1’ in label, but output is always ‘0’.<br>
I’d did something miss in the code…<br>
Could you help me one more time?</p>

username: ptrblck
cooked: <p>Is your model only predicting <code>class0</code>?<br>
If so, could you create a new thread and tag me in it?<br>
Also, could you provide some information regarding your model, training, class distribution etc. in the new thread?</p>

username: skyunyoo
cooked: <p>I’m not yet ran all of code.<br>
I ran just ‘imshow’</p>
<pre><code class="lang-auto">def imshow(inp, img_num):
    inp = inp.numpy().transpose((1, 2, 0))
    mean = np.array([0.485, 0.456, 0.406])
    std = np.array([0.229, 0.224, 0.225])
    inp = std * inp + mean
    inp = np.clip(inp, 0, 1)
    print('label : {}'.format(train[img_num][1]))
    plt.imshow(inp)

imshow(train[40][0], 40)
</code></pre>
<p>But output shows mismatched label with pic</p>

username: ptrblck
cooked: <p>The <code>index</code> you are passing to your <code>Dataset</code> doesn’t correspond to the <code>target</code> for the sample.<br>
I’m not sure, if I understand your code correctly, but you can get the <code>target</code> using <code>target = train[40][1]</code>.</p>

username: skyunyoo
cooked: <p>I get the target by<br>
<code>print('label : {}'.format(train[img_num][1]))</code><br>
but as you mentioned, the target doesn’t correspond to sample.<br>
how could I modify the code that the target correspond to the sample?</p>

username: ptrblck
cooked: <p><code>ImageFolder</code> will create the labels based on the passed folders, so the labels should also mismatch in the original <code>Dataset</code> before the splitting. Could you check that and see if some images might be stored in the wrong folder?</p>

username: skyunyoo
cooked: <pre><code class="lang-auto">data/
                melanoma/
                    AM(1).jpg
                    AM(2).jpg
                    AM(3).jpg
                    ...
                benign/
                    BN(1).jpg
                    BN(2).jpg
                    BN(3).jpg
                    ....
</code></pre>
<p>all images are stored in the right folder…<br>
And</p>
<pre><code class="lang-auto">print(dataset.class_to_idx)
</code></pre>
<pre><code class="lang-auto">{'benign': 0, 'melanoma': 1}
</code></pre>
<p>I need to think about what the problem is.<img src="https://discuss.pytorch.org/images/emoji/apple/sob.png?v=6" title=":sob:" class="emoji" alt=":sob:"></p>

username: ptrblck
cooked: <p>If your original <code>Dataset</code> is fine, the <code>Subsets</code> shouldn’t be changed, since only the passed <code>indices</code> are called in <a href="https://github.com/pytorch/pytorch/blob/0d366e1bde42265d764a6a9aad27a38753156fb0/torch/utils/data/dataset.py#L103" rel="nofollow noopener">this line of code</a>.<br>
Could you check which index in your subset is giving the wrong class and then try to get the original index based on this information? Maybe this helps debugging which file seems to be wrong.</p>
<p>Alternatively, you could write your own custom <code>Dataset</code> and return the image names along the data and target, so that debugging might be a bit easier.</p>

username: skyunyoo
cooked: <p>Oh… your first reply was worked well…<br>
I had print all classes index and that showed class ‘1’  were gatherd end of index in Dataset.<br>
I’m sorry for waste your time.<br>
Thanks a lot of your detailed reply!<img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=6" title=":smiley:" class="emoji" alt=":smiley:"></p>

username: vishnu_vardhan1
cooked: <p>Hi <a class="mention" href="/u/skyunyoo">@skyunyoo</a>, <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
if we want to use StratifiedKFold for ImageFolder, then we have to pass the labels to split as well.<br>
so it will be something like this :  kf.split(dataset,y)<br>
but for y, how can we assign labels from imagefolder?</p>

username: ptrblck
cooked: <p><code>ImageFolder</code> contains the labels in its <a href="https://github.com/pytorch/vision/blob/be8192e20d2529fa552bcfc099974da45365ffd6/torchvision/datasets/folder.py#L122"><code>.target</code> attribute</a>.</p>