Obtain train and test data from Image and a section in the image name

Embabazi: <p>Hello,<br>
I have images, that i need to classify. the target data is included in the image name, but not the entire image name, so i extract this by splitting the image name.<br>
However, I am challenged on how to move this to the dataloader, and later split into train and test sets. The code is attached below.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/0/c04e3eda4dcf8ea14bf4506173f743c48ea8cbb1.png" data-download-href="https://discuss.pytorch.org/uploads/default/c04e3eda4dcf8ea14bf4506173f743c48ea8cbb1" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/0/c04e3eda4dcf8ea14bf4506173f743c48ea8cbb1_2_606x500.png" alt="image" data-base62-sha1="rrdjvep7bkBcEP6T9JJSbVImd3j" width="606" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/0/c04e3eda4dcf8ea14bf4506173f743c48ea8cbb1_2_606x500.png, https://discuss.pytorch.org/uploads/default/original/3X/c/0/c04e3eda4dcf8ea14bf4506173f743c48ea8cbb1.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/c/0/c04e3eda4dcf8ea14bf4506173f743c48ea8cbb1.png 2x" data-dominant-color="EBEDED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">780×643 57 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
After this split, i would like to use the dataloader, as below:<br>
<a class="hashtag-cooked" href="/c/data/37" data-type="category" data-slug="data" data-id="37"><span class="hashtag-icon-placeholder"><svg class="fa d-icon d-icon-square-full svg-icon svg-node"><use href="#square-full"></use></svg></span><span>data</span></a> loader<br>
mean = 0.5<br>
std = 0.5<br>
batch_size=16<br>
transformTrain = transforms.Compose(<br>
[<br>
transforms.RandomVerticalFlip(0.5), # Vertical flip<br>
transforms.RandomHorizontalFlip(0.5), # Horizontally flip the given PIL Image randomly with a given probability.<br>
transforms.RandomRotation(degrees=45), # Rotate the image by angle.<br>
transforms.ToTensor(),<br>
transforms.Normalize((mean, mean, mean), (std, std, std))<br>
])<br>
X_train, X_test, y_train, y_test = train_test_split(images, nsex, test_size=0.2)</p>
<p><span class="hashtag-raw">#X_train</span> = ImageFolder(X_train,transform=transformTrain)<br>
<span class="hashtag-raw">#train_data</span> = custom_dataset(X_train, transform=transformTrain, train=True) # Resize transform<br>
<span class="hashtag-raw">#test_data</span> = custom_dataset(X_test, transform=transform) # Resize transform<br>
X_train = torch.Tensor(X_train)<br>
X_test = torch.Tensor(X_test)<br>
y_train = torch.Tensor(y_train)<br>
y_test = torch.Tensor(y_test)<br>
X_train = X_train.permute(0,3, 1, 2)<br>
X_test = X_test.permute(0,3, 1, 2)</p>
<p>train_dataset = TensorDataset(X_train, y_train)<br>
test_dataset = TensorDataset(X_test, y_test)<br>
train_loader = DataLoader(train_dataset, batch_size=16, shuffle=True)<br>
test_loader = DataLoader(test_dataset, batch_size=16, shuffle=False)</p>

bloos: <p>you should turn it into a tensor so that it is easier for pytorch to handle the data in a multiprocessing environment like dataloaders.</p>
<p>The following snippet is taken from <a href="https://www.geeksforgeeks.org/converting-an-image-to-a-torch-tensor-in-python/" class="inline-onebox" rel="noopener nofollow ugc">Converting an image to a Torch Tensor in Python - GeeksforGeeks</a></p>
<pre><code class="lang-auto"># Import necessary libraries
import torch
from PIL import Image
import torchvision.transforms as transforms
  
# Read a PIL image
image = Image.open('iceland.jpg')
  
# Define a transform to convert PIL 
# image to a Torch tensor
transform = transforms.Compose([
    transforms.PILToTensor()
])
  
# transform = transforms.PILToTensor()
# Convert the PIL image to Torch tensor
img_tensor = transform(image)
  
# print the converted Torch tensor
print(img_tensor)
</code></pre>
<p>in short: You need to apply the transforms in your dataset. At the moment you only store them as a member variable without ever touching them again</p>
<p>Update: Btw: what happens if your path does not exists? Looks like a possible exception for me</p>