Repository: open-mmlab/mmpose

Issue:
Support different result formats of mmdet in demo
```
bbox_xywh[:, 2] = bbox_xywh[:, 2] - bbox_xywh[:, 0] + 1
IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed
```

The following script is used to call mmpose on a video
`
python $PATH_MMPOSE/demo/top_down_video_demo_with_mmdet.py $PATH_MMDET/configs/cascade_rcnn/cascade_mask_rcnn_x101_32x4d_fpn_20e_coco.py $PATH_WEIGHTS/MMDet/cascade_mask_rcnn_x101_32x4d_fpn_20e_coco_20200528_083917-ed1f4751.pth $PATH_MMPOSE/configs/top_down/darkpose/coco-wholebody/hrnet_w48_coco_wholebody_384x288_dark_plus.py $PATH_WEIGHTS/MMPose/hrnet_w48_coco_wholebody_384x288_dark-f5726563_20200918.pth --video-path test_video.mp4 --out-video-root ./results --show`




Discussion:
A print of the variable `bbox_xywh` revealed that the bounding boxes were almost completely empty


```
[array([[221.10242   , 285.07632   , 462.33337   , 713.4651    ,
          0.9974692 ],
       [559.7312    , 416.71567   , 728.47736   , 714.90625   ,
          0.99740857]], dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([[3.9461374e+02, 4.1644919e+02, 4.9405991e+02, 6.1316681e+02,
        2.5452349e-01],
       [0.0000000e+00, 3.2507635e+02, 1.0003251e+02, 6.5151324e+02,
        6.8182789e-02],
       [3.8382455e+02, 4.1591391e+02, 4.0037955e+02, 4.9973083e+02,
        5.0756339e-02]], dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)
 array([], shape=(0, 5), dtype=float32)]
```

are these empty bboxes the output of det, or their bbox values were lost in mmpose?
After `mmdet`. They were in the process of being given as input to `mmpose`. I'll share the video which I've used to play with.
Note that: the result format of faster_rcnn & cascade_mask_rcnn are different.
For faster_rcnn, after calling `inference_detector`, you will get `det`. `det` is a list containing bboxes for 80 categories, and `det[0]` is a ndarray shaped (P, 5). P is the number of people in the image.
For cascade_mask_rcnn, after calling `inference_detector`, you will get a tensor (`det`, `mask`). So to get the person bounding boxes, you should modify https://github.com/open-mmlab/mmpose/blob/1a2b1d85023a4697dbec37a804c358818f1a489b/demo/top_down_video_demo_with_mmdet.py#L81
`person_bboxes = det_results[0].copy()` to `person_bboxes = det_results[0][0].copy()`.
can we support both result format in demo?

Resolving patch:
diff --git a/demo/top_down_img_demo_with_mmdet.py b/demo/top_down_img_demo_with_mmdet.py
--- a/demo/top_down_img_demo_with_mmdet.py
+++ b/demo/top_down_img_demo_with_mmdet.py
@@ -7,6 +7,20 @@
                          vis_pose_result)
 
 
+def process_mmdet_results(mmdet_results, cat_id=0):
+    """Process mmdet results, and return a list of bboxes.
+
+    :param mmdet_results:
+    :param cat_id: category id (default: 0 for human)
+    :return: a list of detected bounding boxes
+    """
+    if isinstance(mmdet_results, tuple):
+        det_results = mmdet_results[0]
+    else:
+        det_results = mmdet_results
+    return det_results[cat_id]
+
+
 def main():
     """Visualize the demo images.
 
@@ -58,10 +72,10 @@ def main():
     image_name = os.path.join(args.img_root, args.img)
 
     # test a single image, the resulting box is (x1, y1, x2, y2)
-    det_results = inference_detector(det_model, image_name)
+    mmdet_results = inference_detector(det_model, image_name)
 
-    # keep the person class bounding boxes. (FasterRCNN)
-    person_bboxes = det_results[0].copy()
+    # keep the person class bounding boxes.
+    person_bboxes = process_mmdet_results(mmdet_results)
 
     # test a single image, with a list of bboxes.
     pose_results, heatmaps = inference_top_down_pose_model(
diff --git a/demo/top_down_video_demo_with_mmdet.py b/demo/top_down_video_demo_with_mmdet.py
--- a/demo/top_down_video_demo_with_mmdet.py
+++ b/demo/top_down_video_demo_with_mmdet.py
@@ -8,6 +8,20 @@
                          vis_pose_result)
 
 
+def process_mmdet_results(mmdet_results, cat_id=0):
+    """Process mmdet results, and return a list of bboxes.
+
+    :param mmdet_results:
+    :param cat_id: category id (default: 0 for human)
+    :return: a list of detected bounding boxes
+    """
+    if isinstance(mmdet_results, tuple):
+        det_results = mmdet_results[0]
+    else:
+        det_results = mmdet_results
+    return det_results[cat_id]
+
+
 def main():
     """Visualize the demo images.
 
@@ -76,9 +90,10 @@ def main():
         if not flag:
             break
         # test a single image, the resulting box is (x1, y1, x2, y2)
-        det_results = inference_detector(det_model, img)
+        mmdet_results = inference_detector(det_model, img)
+
         # keep the person class bounding boxes.
-        person_bboxes = det_results[0].copy()
+        person_bboxes = process_mmdet_results(mmdet_results)
 
         # test a single image, with a list of bboxes.
         pose_results, heatmaps = inference_top_down_pose_model(


Test patch: