Splitting Pytorch Dataset Into Separate Tensor of Labels and Images

James_H: <p>Hi,<br>
I’m currently trying to train a basic CNN on the CIFAR10 dataset, which I loaded using <code>train_dataset = torchvision.datasets.CIFAR10(DATA_PATH, train=True, transform=transform, download=True)</code>, and was able to achieve decent accuracy. However, I noticed that I was tuning the hyperparameters to the test set and seeing the response, possibly overfitting it. So, I looked into using the GridSearchCV class of Sklearn to find the best combination of hyperparameters through K-Fold cross validation. Code below:</p>
<pre><code class="lang-auto">net = NeuralNetClassifier(module=cNet, criterion=nn.CrossEntropyLoss,
    optimizer=torch.optim.Adam, lr=learning_rate, max_epochs=100,
    batch_size=batch_size,)
params = {
    'lr': [0.001, 0.005, 0.01, 0.05, 0.1],
    'max_epochs': np.linspace(5, 30, 6, dtype=np.int64),
    'batch_size': np.exp2(np.arange(5,8))
}
gs = GridSearchCV(estimator=net, param_grid=params, refit=False, scoring='accuracy', cv=10)
gs.fit(train_dataset)
optimizednet = gs.best_estimator_
lr = optimizednet.learning_rate
batch_size = optimizednet.batch_size
num_epochs = optimizednet.max_epochs
</code></pre>
<p>In the process, I was unable to find a way to split the train_dataset into X (images) and Y(labels) for gs.fit() which threw this error <code>TypeError: fit() missing 1 required positional argument: 'y'.</code> Does anyone know a way to split the dataset back into images and labels?</p>

boto: <p>Hi James!</p>
<p>Take a look at <a href="https://pytorch.org/docs/stable/torchvision/datasets.html#torchvision.datasets.CIFAR10" rel="nofollow noopener">torchvision.datasets.CIFAR10</a>. As you can see the <code>__getitem__</code> returns a tuple with <code>(image, target)</code>, where target corresponds to the class/label of the image. You can iterate through the dataset in the following manner:</p>
<pre><code class="lang-auto">...
dataset = torchvision.datasets.CIFAR10(DATA_PATH, train=True, transform=transform, download=True)
for i, data in enumerate(dataset): # i == Index
   image, label = data
...
</code></pre>
<p>Now you have your image as well as the corresponding label and you can work with it.</p>

James_H: <p>Hi boto,<br>
Sorry for the late reply. I tried split the dataset into x and y by</p>
<pre><code class="lang-auto">def split_XY(dataset):

    l = []
    a = torch.Tensor(50000, 3, 32, 32)
    for i, (image, label) in enumerate(train_dataset):
        a[i, :, :, :] = image
        l.append(label)
    return a, torch.Tensor(l)

</code></pre>
<p>and tried</p>
<pre><code class="lang-auto">x, y = split_XY(train_dataset)
gs.fit(x, y)
</code></pre>
<p>but when I ran it, it threw <code>ValueError: Cannot perform a CV split if dataset and y have different lengths.</code> which didn’t make sense to me because x is 50000 x 3 x 32 x 32 and y is 50000 x 1. Maybe I’m missing something? Thanks!</p>

boto: <p>Which line of code throws that error?</p>

James_H: <pre><code class="lang-auto"> File "cifar10.py", line 75, in &lt;module&gt;
    gs.fit(x, y)
</code></pre>

boto: <p>Unfortunately, I don’t have any experience with <code>GridSearchCV</code>.<br>
But the error states that <code>x</code> and <code>y</code> do not have the length/shape, which is true as <code>x.shape=(500000,3,32,32)</code> and <code>y.shape=(50000, 1)</code>. That’s all I can tell you. Maybe try looking at <a href="https://skorch.readthedocs.io/en/latest/user/FAQ.html" rel="nofollow noopener">Skorch FAQ</a>.</p>

James_H: <p>Thanks for the help!</p>

yhl3051: <aside class="quote no-group" data-username="James_H" data-post="1" data-topic="75970" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/james_h/48/12526_2.png" class="avatar"> James_H:</div>
<blockquote>
<p>Hi,<br>
I’m currently trying to train a basic CNN on the CIFAR10 dataset, which I loaded using <code>train_dataset = torchvision.datasets.CIFAR10(DATA_PATH, train=True, transform=transform, download=True)</code>, and was able to achieve decent accuracy. However, I noticed that I was tuning the hyperparameters to the test set and seeing the response, possibly overfitting it. So, I looked into using the GridSearchCV class of Sklearn to find the best combination of hyperparameters through K-Fold cross validation. Code below:</p>
<pre><code class="lang-auto">net = NeuralNetClassifier(module=cNet, criterion=nn.CrossEntropyLoss,
    optimizer=torch.optim.Adam, lr=learning_rate, max_epochs=100,
    batch_size=batch_size,)
params = {
    'lr': [0.001, 0.005, 0.01, 0.05, 0.1],
    'max_epochs': np.linspace(5, 30, 6, dtype=np.int64),
    'batch_size': np.exp2(np.arange(5,8))
}
gs = GridSearchCV(estimator=net, param_grid=params, refit=False, scoring='accuracy', cv=10)
gs.fit(train_dataset)
optimizednet = gs.best_estimator_
lr = optimizednet.learning_rate
batch_size = optimizednet.batch_size
num_epochs = optimizednet.max_epochs
</code></pre>
<p>In the process, I was unable to find a way to split the train_dataset into X (images) and Y(labels) for gs.fit() which threw this error <code>TypeError: fit() missing 1 required positional argument: 'y'.</code> Does anyone know a way to split the dataset back into images and labels?</p>
</blockquote>
</aside>
<p>I am surprised that this has not been suggested yet, but the <code>map</code> function is much more efficient than using a <code>for</code> loop, instantly creating the arrays:</p>
<pre><code class="lang-auto">inds = list(range(1, len(dataset)))
images = list(map(lambda x: np.copy(dataset[x][0]),inds))
labels = list(map(lambda x: np.copy(dataset[x][1]),inds))
</code></pre>
<p>Slicing doesn’t seem to work, from what I know.  I added <code>np.copy</code> to prevent too many sockets from being loaded at once, but I think that is optional.</p>