Repository: open-mmlab/mmdetection

Issue:
how to get the map score based on the cfg/pth/pkl file？
As  mentioned above，and I also can not find coco_eval.py 
Thank you for your time and consideration.


Discussion:
Yes, `coco_eval.py` is removed in the current version of mmdet. I will upload a new one soon. But a temporal solution is to comment the inference codes in [test.py](https://github.com/open-mmlab/mmdetection/blob/master/tools/test.py):
https://github.com/open-mmlab/mmdetection/blob/9596b9a4c916ae601f9a8a641c3a0ea47265abec/tools/test.py#L156-L166
and replace `mmcv.dump` with `mmcv.load` to load your pkl results.
Then run the normal test command so that it directly evaluates your pkl results.

Resolving patch:
diff --git a/tools/eval_metric.py b/tools/eval_metric.py
new file mode 100644
--- /dev/null
+++ b/tools/eval_metric.py
@@ -0,0 +1,72 @@
+import argparse
+
+import mmcv
+from mmcv import Config, DictAction
+
+from mmdet.datasets import build_dataset
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description='Evaluate metric of the '
+                                     'results saved in pkl format')
+    parser.add_argument('config', help='Config of the model')
+    parser.add_argument('pkl_results', help='Results in pickle format')
+    parser.add_argument(
+        '--format-only',
+        action='store_true',
+        help='Format the output results without perform evaluation. It is'
+        'useful when you want to format the result to a specific format and '
+        'submit it to the test server')
+    parser.add_argument(
+        '--eval',
+        type=str,
+        nargs='+',
+        help='Evaluation metrics, which depends on the dataset, e.g., "bbox",'
+        ' "segm", "proposal" for COCO, and "mAP", "recall" for PASCAL VOC')
+    parser.add_argument(
+        '--cfg-options',
+        nargs='+',
+        action=DictAction,
+        help='override some settings in the used config, the key-value pair '
+        'in xxx=yyy format will be merged into config file.')
+    parser.add_argument(
+        '--eval-options',
+        nargs='+',
+        action=DictAction,
+        help='custom options for evaluation, the key-value pair in xxx=yyy '
+        'format will be kwargs for dataset.evaluate() function')
+    args = parser.parse_args()
+    return args
+
+
+def main():
+    args = parse_args()
+
+    cfg = Config.fromfile(args.config)
+    assert args.eval or args.format_only, (
+        'Please specify at least one operation (eval/format the results) with '
+        'the argument "--eval", "--format-only"')
+    if args.eval and args.format_only:
+        raise ValueError('--eval and --format_only cannot be both specified')
+
+    if args.cfg_options is not None:
+        cfg.merge_from_dict(args.cfg_options)
+    cfg.data.test.test_mode = True
+
+    dataset = build_dataset(cfg.data.test)
+    outputs = mmcv.load(args.pkl_results)
+
+    kwargs = {} if args.eval_options is None else args.eval_options
+    if args.format_only:
+        dataset.format_results(outputs, **kwargs)
+    if args.eval:
+        eval_kwargs = cfg.get('evaluation', {}).copy()
+        # hard-code way to remove EvalHook args
+        for key in ['interval', 'tmpdir', 'start', 'gpu_collect']:
+            eval_kwargs.pop(key, None)
+        eval_kwargs.update(dict(metric=args.eval, **kwargs))
+        print(dataset.evaluate(outputs, **eval_kwargs))
+
+
+if __name__ == '__main__':
+    main()


Test patch: