Problem in building my own MNIST custom dataset

username: Feng
cooked: <p>Hi everyone! I am having problems in reading MNIST training csv file (size: 60000* 785, the first column is the label, where I downloaded through the link: <a href="https://pjreddie.com/projects/mnist-in-csv/" rel="nofollow noopener">https://pjreddie.com/projects/mnist-in-csv/</a> ) from my local computer.</p>
<p>The following is the detail information, could somebody help me to identify what is the cause, thanks in advance!</p>
<ol>
<li>I am running Pytorch in Win10 with pytorch-0.4.1, python 3.6.6;</li>
<li>My source codes:</li>
</ol>
<pre><code class="lang-python">''' -*- coding: utf-8 -*-'''
import torch 
import torch.nn as nn
from skimage import transform
import torchvision.transforms as transforms
from torch.autograd import Variable
import numpy as np
from torch.utils.data import Dataset

''' Ignore warnings'''
import warnings
warnings.filterwarnings("ignore")

''' Hyper Parameters''' 
input_size = 784
hidden_size1 = 500
hidden_size2 = 300
num_classes = 10
num_epochs = 5
batch_size = 100
learning_rate = 0.001


class NosiyMNISTDataset(Dataset):
    
    def __init__(self):
        xy = np.loadtxt('./data_MNIST/mnist_train.csv',
                        delimiter=',', dtype=np.float32)
        self.len = xy.shape[0]
        self.x_data = torch.from_numpy(xy[:, 1:])
        self.y_data = torch.from_numpy(xy[:, [0]])

    def __getitem__(self, index):
        return (self.x_data[index], self.y_data[index])

    def __len__(self):
        return self.len
    

''' MNIST Dataset '''
train_dataset = NosiyMNISTDataset()
train_loader = torch.utils.data.DataLoader(dataset=train_dataset, 
                                           batch_size=batch_size, 
                                           shuffle=True,
                                           num_workers = 2)

''' Neural Network Model (2 hidden layer)'''
class Net(nn.Module):
    def __init__(self, input_size, hidden_size1, hidden_size2, num_classes):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size1) 
        self.fc2 = nn.Linear(hidden_size1, hidden_size2) 
        self.fc3 = nn.Linear(hidden_size2, num_classes)
        self.relu = nn.ReLU() 
        # self.dropout = nn.Dropout() 
    
    def forward(self, x):
        out = self.fc1(x)
        out = self.relu(out)
        # out = self.dropout(out) 
        out = self.fc2(out)
        out = self.relu(out)
        # out = self.dropout(out)
        out = self.fc3(out)
        return out


if __name__ == '__main__':   
  net = Net(input_size, hidden_size1, hidden_size2, num_classes)
  net.cuda() 
''' Loss and Optimizer'''
  criterion = nn.CrossEntropyLoss()
  optimizer = torch.optim.Adam(net.parameters(), lr = learning_rate)  

'''Train the Model'''
  for epoch in range(num_epochs):
    for i, (images, labels) in enumerate(train_loader,0):  
        # Convert torch tensor to Variable
        images = Variable(images.view(-1, 28*28).cuda())
        labels = Variable(labels.cuda())
       
        
        # Forward + Backward + Optimize
        optimizer.zero_grad()  # zero the gradient buffer
        outputs = net(images)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
     
        if (i+1) % 100 == 0:
            print ('Epoch [%d/%d], Step [%d/%d], Loss: %.4f' 
                   %(epoch+1, num_epochs, i+1, len(train_dataset)//batch_size, loss.data[0]))
</code></pre>
<ol start="3">
<li>The error I got is:</li>
</ol>
<pre><code class="lang-python">"Traceback (most recent call last):
  File "Local_MNIST.py", line 92, in &lt;module&gt;
    loss = criterion(outputs, labels)
 ................
RuntimeError: Expected object of type torch.cuda.LongTensor but found type torch.cuda.FloatTensor for argument #2 'target'"
</code></pre>
<p>I have tried<br>
&lt;1&gt; . change the “dtype” in my NosiyMNISTDataset class to ‘np.float’, ‘np.long’, and ‘np.int64’<br>
&lt;2&gt;.  add long() to  torch.from_numpy(xy[:, [0]]) as torch.from_numpy(xy[:, [0]]).long()<br>
&lt;3&gt; remove cuda() from my codes</p>
<p>All the three ways do not solve my problem.</p>
<p>P.S. I know there is a standard way to laod MNIST dataset, however, I would like to modify dataset that’s why I am using csv file.</p>
<p>Looking forward to you guys comments.</p>
<p>Thanks!</p>

username: ptrblck
cooked: <p>The second approach should have worked.<br>
Could you add the <code>.long()</code> cast again and print the type of <code>labels</code> in your training loop?</p>
<p>PS: I’ve formatted your code as it was a bit difficult to read. You can add code using three backticks `.</p>

username: Feng
cooked: <p><span class="mention">@ptrblck_de</span> Thank you for formatting my code in the post.</p>
<p>I have tried to add the .long() to my NosiyMNISTDataset class as</p>
<pre><code>                       self.y_data = torch.from_numpy(xy[:, [0]]).long()
</code></pre>
<p>and add a print function under the training loop<br>
labels = Variable(labels.cuda())<br>
print(labels.shape)</p>
<p>it gave “torch.Size([100,1])”  (matches with my batch size).</p>
<p>However, it still has an error as follows:<br>
“RuntimeError: multi-target not supported at c:\programdata\miniconda3\conda-bld\pytorch_1533090623466\work\aten\src\thcunn\generic/ClassNLLCriterion.cu:15”</p>

username: ptrblck
cooked: <p>The current error message seems to be unrelated to the type, but rather to the shape.<br>
Try to remove <code>dim1</code> with <code>labels = labels.squeeze()</code> and run it again.</p>

username: Feng
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="24080">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>labels = labels.squeeze()</code></p>
</blockquote>
</aside>
<p>Yes, I tried to print the standard way of loading MNIST dataset, yes, indeed the label size is<br>
torch.Size([100]).</p>
<p>After I added the code “labels = labels.squeeze()” , it works!!!</p>
<p>I really appreciate your help!!</p>

username: A_9
cooked: <p>Hi everyone <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
I have the same problem, and I can’t solve it. I want to use my own MNIST data set.<br>
I have two .csv files for train and test data and labels (I mean that in each .csv file, the first column is the labels and other columns from 2 to 785 are pixels of images).</p>
<h1>
<a name="read-the-data-1" class="anchor" href="#read-the-data-1"></a>read the data</h1>
<p>df_train = pd.read_csv(‘Train_Data_FS_with_Label.csv’,header=None)<br>
df_test = pd.read_csv(‘Test_Data_FS_with_Label.csv’,header=None)</p>
<h1>
<a name="get-the-image-pixel-values-and-labels-2" class="anchor" href="#get-the-image-pixel-values-and-labels-2"></a>get the image pixel values and labels</h1>
<p>train_labels = df_train.iloc[:, 0]<br>
train_images = df_train.iloc[:, 1:]<br>
test_labels = df_test.iloc[:, 0]<br>
test_images = df_test.iloc[:, 1:]</p>
<h1>
<a name="define-transforms-3" class="anchor" href="#define-transforms-3"></a>define transforms</h1>
<p>transform = transforms.Compose(<br>
[transforms.ToPILImage(),<br>
transforms.RandomCrop(24),<br>
transforms.ToTensor()<br>
])</p>
<h1>
<a name="custom-dataset-4" class="anchor" href="#custom-dataset-4"></a>custom dataset</h1>
<p>class MNISTDataset(Dataset):<br>
def <strong>init</strong>(self, images, labels, transforms):<br>
self.X = images<br>
self.y = labels<br>
self.transforms = transforms<br>
def <strong>len</strong>(self):<br>
return (len(self.X))<br>
def <strong>getitem</strong>(self, i):<br>
data = self.X.iloc[i, :]<br>
data = np.asarray(data).astype(np.uint8).reshape(24,24, 1)<br>
if self.transforms:<br>
data = self.transforms(data)<br>
if self.y is not None:<br>
return (data, self.y[i])<br>
else:<br>
return data<br>
train_data = MNISTDataset(train_images, train_labels, transform)<br>
test_data = MNISTDataset(test_images, test_labels, transform)</p>
<h1>
<a name="dataloaders-5" class="anchor" href="#dataloaders-5"></a>dataloaders</h1>
<p>trainloader = DataLoader(train_data, batch_size=1, shuffle=True)<br>
testloader = DataLoader(test_data, batch_size=1, shuffle=True)</p>
<p>and I need to split the test data set into 2 parts:<br>
test_ds, valid_ds_before = torch.utils.data.random_split(testloader , (9500, 500))<br>
small_shared_dataset = create_shared_dataset(valid_ds_before, 200)</p>
<p>In the “create_shared_dataset” function:<br>
def create_shared_dataset(valid_ds, size):<br>
data_loader = DataLoader(valid_ds, batch_size=1)<br>
for idx, (data, target) in enumerate(data_loader): (error occur here!!)<br>
…</p>
<p>my code has error in this part: “‘DataLoader’ object is not subscriptable”</p>
<p>how can I solve this error??<br>
I would be very grateful for any help you can give me.</p>

username: ptrblck
cooked: <p>Use <code>random_split</code> on <code>Dataset</code> instances, not on the <code>DataLoader</code> object.<br>
Once you’ve created the datasets, wrap it once into a <code>DataLoader</code> and it should work.</p>

username: A_9
cooked: <p>It works <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
I really appreciate your help <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>