Raised keyerror when training deep learning model

LY292: <h2><a name="weird-thing-is-it-works-in-the-start-theres-been-some-normal-results-then-after-a-few-batches-it-wont-work-again-outputs-are-like-epoch-1-1" class="anchor" href="#weird-thing-is-it-works-in-the-start-theres-been-some-normal-results-then-after-a-few-batches-it-wont-work-again-outputs-are-like-epoch-1-1"></a>weird thing is it works in the start, there’s been some normal results, then after a few batches it won’t work again<br>
Outputs are like:<br>
Epoch 1</h2>
<p>loss: 2.291789  [  128/40000]<br>
loss: 2.298886  [12928/40000]<br>
loss: 2.304425  [25728/40000]<br>
loss: 2.292490  [38528/40000]<br>
Traceback (most recent call last):<br>
File “D:\Projects\project1_model0\project1\main.py”, line 194, in <br>
main()<br>
File “D:\Projects\project1_model0\project1\main.py”, line 190, in main<br>
trainer.Train_session(name)<br>
File “D:\Projects\project1_model0\project1\main.py”, line 157, in Train_session<br>
self.test(“train”)<br>
File “D:\Projects\project1_model0\project1\main.py”, line 104, in test<br>
for batch, (img, label) in enumerate(self.test_loader, 0):<br>
File “C:\Users\yxf\AppData\Roaming\Python\Python310\site-packages\torch\utils\data\dataloader.py”, line 633, in <strong>next</strong><br>
data = self._next_data()<br>
File “C:\Users\yxf\AppData\Roaming\Python\Python310\site-packages\torch\utils\data\dataloader.py”, line 677, in _next_data<br>
data = self._dataset_fetcher.fetch(index)  # may raise StopIteration<br>
File “C:\Users\yxf\AppData\Roaming\Python\Python310\site-packages\torch\utils\data_utils\fetch.py”, line 51, in fetch<br>
data = [self.dataset[idx] for idx in possibly_batched_index]<br>
File “C:\Users\yxf\AppData\Roaming\Python\Python310\site-packages\torch\utils\data_utils\fetch.py”, line 51, in <br>
data = [self.dataset[idx] for idx in possibly_batched_index]<br>
File “D:\Projects\project1_model0\project1\dataset.py”, line 26, in <strong>getitem</strong><br>
img = self.data[index]<br>
KeyError: 1</p>
<p>I used a customized dataset:<br>
import torch<br>
from torch.utils.data import Dataset, DataLoader</p>
<p>class my_dataset(Dataset):<br>
def <strong>init</strong>(self, name):<br>
if name == “train”:<br>
self.path = “…/data_set/train.pt”<br>
self.data = torch.load(self.path, map_location=“cpu”)[“data”]<br>
self.labels = torch.load(self.path, map_location=“cpu”)[“labels”]</p>
<pre><code>    elif name == "valid":
        self.path = "../data_set/valid.pt"
        self.data = torch.load(self.path, map_location="cpu")
        self.labels = torch.load(self.path, map_location="cpu")["labels"]

    elif name == "test":
        self.path = "../data_set/test.pt"
        self.data = torch.load(self.path, map_location="cpu")
        self.labels = torch.load(self.path, map_location="cpu")["labels"]

    else:
        raise NotImplementedError

def __getitem__(self, index):
    img = self.data[index]
    label = self.labels[index]  # (3, 28, 28), (1)
    return img, label

def __len__(self):
    return len(self.data)
</code></pre>

ptrblck: <p>Did you check which <code>index</code> failed and if <code>self.data</code> contains an entry at this <code>index</code>?</p>

LY292: <p>yes, that’s the point. I solved it earlier today, stupid mistakes lol.</p>