Repository: open-mmlab/mmdetection

Issue:
VFNET device error when run inference_detector
Hi everyone,

When a run the inference_detector to VFNET algorithm, I got this error message (only for VFNET, for ATSS ... no problems):

/usr/local/lib/python3.6/dist-packages/mmcv/parallel/_functions.py in forward(target_gpus, input)
     71             # Perform CPU to GPU copies in a background stream
     72             streams = [_get_stream(device) for device in target_gpus]
---> 73 
     74         outputs = scatter(input, target_gpus, streams)
     75         # Synchronize with the copy stream

/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/_functions.py in _get_stream(device)
    117     if _streams is None:
    118         _streams = [None] * torch.cuda.device_count()
--> 119     if _streams[device] is None:
    120         _streams[device] = torch.cuda.Stream(device)
    121     return _streams[device]

TypeError: list indices must be integers or slices, not torch.device

The device paramenter of init_detector is default ('cuda:0') and distributed=False. Can you help me?



Discussion:
I had the same problem with VFNet. I think it's a fundamental bug in the code of mmcv and nothing to do with the method. More specifically here:
https://github.com/open-mmlab/mmcv/blob/91a7fee03a3973a56cb5f687a6859ef0aaacf15e/mmcv/parallel/_functions.py#L72

However torch _get_stream formulates a list, therefore device should be integer and not torch.device  https://github.com/pytorch/pytorch/blob/18ae12a841bdc99c6cce65ac5c77cc1149dc8564/torch/nn/parallel/_functions.py#L111-L120

The fix is to pass device.index rather than device while calling _get_stream.

I don't know however why it normally works, e.g. with simple faster-rcnn detectors. I think a first exception is muted in https://github.com/open-mmlab/mmcv/blob/91a7fee03a3973a56cb5f687a6859ef0aaacf15e/mmcv/parallel/scatter_gather.py#L44

EDIT: Just did a quick and dirty fix in the local mmcv code as I suggested and inference using VFNet worked.
EDIT2: This bug (wrong parameter type) happens only through inference, so the workaround is WRONG for training. Looking into the real cause of the problem
EDIT3: The correct fix is to pass device.index rather than device in https://github.com/open-mmlab/mmdetection/blob/1989b69c918ac63488918207e5ca02ffb81a1b53/mmdet/apis/inference.py#L112

Resolving patch:
diff --git a/mmdet/apis/inference.py b/mmdet/apis/inference.py
--- a/mmdet/apis/inference.py
+++ b/mmdet/apis/inference.py
@@ -9,6 +9,7 @@
 from mmcv.runner import load_checkpoint
 
 from mmdet.core import get_classes
+from mmdet.datasets import replace_ImageToTensor
 from mmdet.datasets.pipelines import Compose
 from mmdet.models import build_detector
 
@@ -104,9 +105,13 @@ def inference_detector(model, img):
         # add information into dict
         data = dict(img_info=dict(filename=img), img_prefix=None)
     # build the data pipeline
+    cfg.data.test.pipeline = replace_ImageToTensor(cfg.data.test.pipeline)
     test_pipeline = Compose(cfg.data.test.pipeline)
     data = test_pipeline(data)
     data = collate([data], samples_per_gpu=1)
+    # just get the actual data from DataContainer
+    data['img_metas'] = [img_metas.data[0] for img_metas in data['img_metas']]
+    data['img'] = [img.data[0] for img in data['img']]
     if next(model.parameters()).is_cuda:
         # scatter to specified GPU
         data = scatter(data, [device])[0]
@@ -115,8 +120,6 @@ def inference_detector(model, img):
             assert not isinstance(
                 m, RoIPool
             ), 'CPU inference with RoIPool is not supported currently.'
-        # just get the actual data from DataContainer
-        data['img_metas'] = data['img_metas'][0].data
 
     # forward the model
     with torch.no_grad():


Test patch: