Error during inference: Non-empty Empty 3D or 4D input tensor expected

username: Mdhvince
cooked: <p>Hi everyone,</p>
<p>I trained <code>keypointrcnn_resnet50_fpn</code> from torchvision to predict 12 keypoints. The training phase was good but I got an error when doing inference.</p>
<p>Here is my training loop:</p>
<pre><code class="lang-auto">valid_loss_min = np.inf
	for epoch in range(1, num_epochs+1):
		epoch += epoch_resume

		train_loss_per_batch = []
		val_loss_per_batch = []

		for images, targets in train_loader:

			images = list(image.type(dtype).to(device) for image in images)
			targets = [{k: v.type(dtype).to(device) for k, v in t.items()} for t in targets]

			loss_dict = model(images, targets)
			losses = loss_dict['loss_keypoint']

			optimizer.zero_grad()
			losses.backward()
			optimizer.step()
			train_loss_per_batch.append(losses.item())
			

		# Validation
		with torch.no_grad():
			for images, targets in valid_loader:
				images = list(image.type(dtype).to(device) for image in images)
				targets = [{k: v.type(dtype).to(device) for k, v in t.items()} for t in targets]

				loss_dict = model(images, targets)
				losses = loss_dict['loss_keypoint']
				val_loss_per_batch.append(losses.item())

		avg_val_loss = np.mean(np.array(val_loss_per_batch))
		avg_train_loss = np.mean(np.array(train_loss_per_batch))
		print(f"Epoch [{epoch}]\t- Train Loss: {avg_train_loss}\t- Validation Loss: {avg_val_loss} ")

		if avg_val_loss &lt;= valid_loss_min:
			print(f"\nValidation loss decreased ({valid_loss_min} --&gt; {avg_val_loss}).  Saving model ...\n")
			torch.save({
					'epoch': epoch,
					'model_state_dict': model.state_dict(),
					'optimizer_state_dict': optimizer.state_dict(), 
					'best_loss': avg_val_loss
				}, saved_model_path)

			valid_loss_min = avg_val_loss
</code></pre>
<p>And here is the code for inference:</p>
<pre><code class="lang-auto">model, device, dtype = load_model(model_path='models/model_resume_fot.pt', nbkpts=12)

	img_name = 'img_test/1.png'
	img = cv2.imread(img_name)
	new_h, new_w = img.shape[:2]

	img_resized = cv2.resize(img, (448, 448))
	img_pil = T.ToPILImage()(img_resized).convert('RGB')
	im_tensor = T.ToTensor()(img_pil)

	with torch.no_grad():
	    prediction = model([im_tensor.to(device)])
</code></pre>
<p>And got this error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "test_inf_image.py", line 75, in &lt;module&gt;
    prediction = model([im_tensor.to(device)])
  File "/Users/medhyvinceslas/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/medhyvinceslas/anaconda3/lib/python3.7/site-packages/torchvision-0.3.0a0+427633a-py3.7-macosx-10.7-x86_64.egg/torchvision/models/detection/generalized_rcnn.py", line 52, in forward
  File "/Users/medhyvinceslas/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/medhyvinceslas/anaconda3/lib/python3.7/site-packages/torchvision-0.3.0a0+427633a-py3.7-macosx-10.7-x86_64.egg/torchvision/models/detection/roi_heads.py", line 598, in forward
  File "/Users/medhyvinceslas/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/medhyvinceslas/anaconda3/lib/python3.7/site-packages/torchvision-0.3.0a0+427633a-py3.7-macosx-10.7-x86_64.egg/torchvision/models/detection/keypoint_rcnn.py", line 253, in forward
  File "/Users/medhyvinceslas/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/medhyvinceslas/anaconda3/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 796, in forward
    output_padding, self.groups, self.dilation)
RuntimeError: invalid argument 2: non-empty 3D or 4D input tensor expected but got: [0 x 512 x 14 x 14] at /Users/distiller/project/conda/conda-bld/pytorch_1556653464916/work/aten/src/THNN/generic/SpatialFullDilatedConvolution.c:44
</code></pre>
<p>I’m sure that my image exists, so I don’t know what happens here. Can you help me?</p>
<p>Thank you.<br>
Medhy</p>

username: Mdhvince
cooked: <p>Putting the validation phase before the training phase in my for loop solve my problem. I don’t know if it was the right thing to do (just an intuition).<br>
If yes, can you explain why ?</p>

username: DD0922
cooked: <p>I also met this error , and I  found that my input was [N , C , D , W, H ] , but  I used the avgpool2d.<br>
In fact , I should use avgpool3d. <img src="https://discuss.pytorch.org/images/emoji/apple/cry.png?v=9" title=":cry:" class="emoji" alt=":cry:"></p>