Index 2700 is out of bounds for axis 0 with size 2700

alpha2210: <p>I am getting above error while executing following function. My module runs very well for Training and Validation, but while in testing phase it gives me above error. Any suggestion/solution are welcome <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<pre><code class="lang-auto">def evaluate_test(self):
        # restore model args

        args = self.args

        # evaluation mode
        self.model.load_state_dict(torch.load(osp.join(self.args.save_path, 'max_acc.pth'))['params'])
        self.model.eval()
        record = np.zeros((2700, 2)) # loss and acc
        label = torch.arange(args.eval_way, dtype=torch.int16).repeat(args.eval_query)
        label = label.type(torch.LongTensor)
        if torch.cuda.is_available():
            label = label.cuda()
        print('best epoch {}, best val acc={:.4f} + {:.4f}'.format(
                self.trlog['max_acc_epoch'],
                self.trlog['max_acc'],
                self.trlog['max_acc_interval']))
        confusion_matrix=torch.zeros(args.eval_way,args.eval_way) 
        with torch.no_grad():
            for i, batch in tqdm(enumerate(self.test_loader, 1)):
                if torch.cuda.is_available():
                    data, real_label= [_.cuda() for _ in batch]
                else:
                    data = batch[0]
                real_label = real_label.type(torch.LongTensor)[args.eval_way:]
                if torch.cuda.is_available():
                    real_label = real_label.cuda()
                logits = self.model(data)
                loss = F.cross_entropy(logits, label)
                pred = torch.argmax(logits, dim=1)
                actual_pred=torch.where(pred.eq(label),real_label,real_label[label[pred]])
                #print('actual_label is:', real_label)
                #print('actual_pred is:',actual_pred)
                acc = count_acc(logits, label)
                record[i-1, 0] = loss.item()
                #print('record is:', record)
                record[i-1, 1] = acc
                real_label = batch[1]
                #print(real_label)
                for t,p in zip(real_label.view(-1), actual_pred.view(-1)):
                    confusion_matrix[t.long(),p.long()]+=1
        cm=confusion_matrix.numpy()
        import pandas as pd
        cm_col=cm / cm.sum(axis=1)  
        cm_row=cm / cm.sum(axis=0)
</code></pre>
<p>It is causing me error at this line in the above error<br>
<code> record[i-1, 0] = loss.item()</code></p>

eqy: <p>Indexing is 0-based, so the maximum index of a length 2700 index is 2699. What happens if you do <code>record = np.zeros(len(self.test_loader), 2))</code>?</p>

alpha2210: <p>Thanks but if I give 2699 also it gives me same error stating <code>[Index 2699 is out of bounds for axis 0 with size 2699]</code>. If I do ``record = np.zeros(len(self.test_loader), 2))<code> ?</code> then it gives <code>TypeError: len() takes exactly one argument (2 given)</code> <a class="mention" href="/u/eqy">@eqy</a> <a class="mention" href="/u/ptrblck">@ptrblck</a></p>

eqy: <p>Can you check the type of <code>self.test_loader</code>? e.g., <code>type(self.test_loader)</code>?</p>

alpha2210: <p>Hello <a class="mention" href="/u/eqy">@eqy</a>, I checked the type it is <code>class 'torch.utils.data.dataloader.DataLoader'&gt;</code>.</p>

eqy: <p>Can you double check then that <code>len(self.test_loader)</code> is causing the problem?</p>

alpha2210: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/6/360ed3e27a64f3c16e2a171418509877b82530cc.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/360ed3e27a64f3c16e2a171418509877b82530cc" title="1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/6/360ed3e27a64f3c16e2a171418509877b82530cc_2_690x89.jpeg" alt="1" data-base62-sha1="7Idyc9UjYLNCH1R7Zk1dXJoOosk" width="690" height="89" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/6/360ed3e27a64f3c16e2a171418509877b82530cc_2_690x89.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/3/6/360ed3e27a64f3c16e2a171418509877b82530cc.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/6/360ed3e27a64f3c16e2a171418509877b82530cc.jpeg 2x" data-dominant-color="2C2C2C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">1</span><span class="informations">798×104 16.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Here is snapshot of error I am getting when I give<code> record = np.zeros(len(self.test_loader, 2)) # loss and acc</code> <a class="mention" href="/u/eqy">@eqy</a></p>

eqy: <p>I think this should be <code>record = np.zeros(len(self.test_loader), 2)</code> rather than <code>record = np.zeros(len(self.test_loader, 2))</code>.</p>

alpha2210: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/8/38a35dff4bfd53d7bff72419a8a81970d3c6ddbd.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/38a35dff4bfd53d7bff72419a8a81970d3c6ddbd" title="1"><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/8/38a35dff4bfd53d7bff72419a8a81970d3c6ddbd.jpeg" alt="1" data-base62-sha1="852Kyucaczr0znvnHFfJmOvWlHv" width="690" height="117" data-dominant-color="2D2D2D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">1</span><span class="informations">701×119 18.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Now I get this error with <code>record=np.zeros(len(self.test_loader), 2)</code></p>

eqy: <p>Sorry, it should have been a tuple: <code>record=np.zeros((len(self.test_loader), 2))</code></p>

alpha2210: <p>It worked. Thanks. But when I was entering <code>record=np.zeros((10000, 2))</code> it was working fine initially. But later on it is giving me above error. I wonder why? What caused this error? In the above line 10000 are no evaluation episodes.</p>