Detic, ValueError from swintransformer.py

a_friend: <p>I am following the <a href="https://colab.research.google.com/drive/1QtTW9-ukX2HKZGvt0QvVGqjuqEykoZKI#scrollTo=B9ErX0ZvYYHI" rel="noopener nofollow ugc">Detectron/ Detic tutorial </a> in order to auto crop a large training dataset from the <a href="https://www.kaggle.com/c/happy-whale-and-dolphin/data" rel="noopener nofollow ugc">Kaggle Whale and dolphin competition</a>.</p>
<p>The config <code>cfg</code>, vocabulary etc. are the same as in the tutorial. I have asked a <a href="https://discuss.pytorch.org/t/detectron2-test-loader-error/145190">different question</a> about my code. The solution to that specific problem, seems to be pasting an exact copy of the built-in <code>collate_fn</code> function into the notebook(!)</p>
<p>However, I have not been able to run a complete evalutation/ forward run of the dataset yet. The <code>model.eval()</code> procedure runs for at least 30 minutes on a Tesla T4, then stops with the traceback presented below. Any ideas why?</p>
<p>Trying</p>
<pre><code class="lang-auto">image =Image.open('/home/jupyter/happy-whale-and-dolphin/train_images/00021adfb725ed.jpg')
image.mode

</code></pre>
<p>returns ‘RGB’.</p>
<p>Here is a snippet of my code:</p>
<pre><code class="lang-auto">def my_dataset_generator():
    data_list = []
    for index, observation in df.iterrows():
        data_dict = {}
        data_dict["file_name"] = observation['file_path']
        data_dict['image_id'] = observation['image']
        data_list.append(data_dict)
    data_list = np.array(data_list)
    print(data_list[0])
    return data_list

DatasetCatalog.register("my_whales5", my_dataset_generator)
data = DatasetCatalog.get('my_whales5')
#print(type(data), data[0])
model = build_model(cfg)  # returns a torch.nn.Module
test_loader = build_detection_test_loader(cfg, 
                                          'my_whales5', 
                                          batch_size=32,
                                         collate_fn = custom_collate,
                                         mapper=DatasetMapper(cfg, augmentations=[T.Resize((90, 90))]))

model.eval()
with torch.no_grad():
    outputs = model(test_loader)
</code></pre>
<p>which yields</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_13624/1901225259.py in &lt;module&gt;
      2 model.eval()
      3 with torch.no_grad():
----&gt; 4     outputs = model(test_loader)
      5 
      6 print('Stopped running at: {}'.format(datetime.now().strftime('%H:%M')))

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

~/Detic/detic/modeling/meta_arch/custom_rcnn.py in forward(self, batched_inputs)
    113         """
    114         if not self.training:
--&gt; 115             return self.inference(batched_inputs)
    116 
    117         images = self.preprocess_image(batched_inputs)

~/Detic/detic/modeling/meta_arch/custom_rcnn.py in inference(self, batched_inputs, detected_instances, do_postprocess)
     95 
     96         images = self.preprocess_image(batched_inputs)
---&gt; 97         features = self.backbone(images.tensor)
     98         proposals, _ = self.proposal_generator(images, features, None)
     99         results, _ = self.roi_heads(images, features, proposals)

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

/opt/conda/lib/python3.7/site-packages/detectron2/modeling/backbone/fpn.py in forward(self, x)
    124                 ["p2", "p3", ..., "p6"].
    125         """
--&gt; 126         bottom_up_features = self.bottom_up(x)
    127         results = []
    128         prev_features = self.lateral_convs[0](bottom_up_features[self.in_features[-1]])

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

~/Detic/detic/modeling/backbone/swintransformer.py in forward(self, x)
    602     def forward(self, x):
    603         """Forward function."""
--&gt; 604         x = self.patch_embed(x)
    605 
    606         Wh, Ww = x.size(2), x.size(3)

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

~/Detic/detic/modeling/backbone/swintransformer.py in forward(self, x)
    427         """Forward function."""
    428         # padding
--&gt; 429         _, _, H, W = x.size()
    430         if W % self.patch_size[1] != 0:
    431             x = F.pad(x, (0, self.patch_size[1] - W % self.patch_size[1]))

ValueError: too many values to unpack (expected 4)
</code></pre>

ptrblck: <p>It seems this internal method:</p>
<pre><code class="lang-python">_, _, H, W = x.size()
</code></pre>
<p>expects <code>x</code> to have 4 dimensions, which isn’t the case in your script.<br>
Could you check which input shape is expected and what you are currently passing to the model?</p>

a_friend: <p>Thanks, I found that the dimension of the tensor given to the Detic machinery is 5, namely <code>[&lt;Number of batches&gt;, &lt;batch size&gt;, &lt;number of channels&gt;, &lt;height&gt;, &lt;width&gt;]</code></p>
<p>I solved the specific error by changing</p>
<pre><code class="lang-auto">_, _, H, W = x.size()
</code></pre>
<p>to</p>
<pre><code class="lang-auto">try:
  H = x.size()[-2]
  W = x.size()[-1]
except ValueError:
  logging.info('Crash: {}'.format(x.size()))
</code></pre>
<p>That fixed the issue, but gave this traceback later in the evaluation:</p>
<p><code>RuntimeError: Expected 4-dimensional input for 4-dimensional weight [128, 3, 4, 4], but got 5-dimensional input of size [3189, 16, 3, 96, 96] instead</code></p>
<p>As mentioned, I have copypasted the <code>default_collate</code> function into my notebook, and renamed it <code>custom_collate</code>. Since I am in a cloud Jupyter environment, my debugging options are limited. Without being able to step through, I find the <code>default_collate</code> function a bit dense for reverse engineering.</p>
<p>Any clues to where I should modify my code, in order to strip the fifth dimension of my tensors being passed to Detic?</p>