Repository: mlcommons/inference
Issue: ONNX SSD-Mobilenet - runtime errors, model issues
Root cause: External Dependency Breakage

Hi - I'm having some trouble running ONNX ssd-mobilenet. https://github.com/mlperf/inference/tree/master/v0.5/classification_and_detection notes that this is "tested on onnxruntime with some runtime warnings". Is this expected to work on glow? Is there an effort to root cause and understand the warnings?

I see these errors in onnxruntime, running with `./run_local.sh onnxruntime ssd-mobilenet cpu`.
```
2019-07-01 14:51:31.881344187 [W:onnxruntime:InferenceSession, session_state_initializer.cc:417 SaveInputOutputNamesToNodeMapping] Graph input with name i__22 is not associated with a node.

2019-07-01 14:51:32.325647195 [W:onnxruntime:Default, execution_frame.cc:281 AllocateMLValueTensorSelfOwnBufferHelper] For ort_value with index: 8592, block in memory pattern size is: 128 but the actually size is: 0, fall back to default allocation behavior

2019-07-01 14:52:32.211781349 [E:onnxruntime:, sequential_executor.cc:123 Execute] Non-zero status code returned while running Node: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Gather_90/Gather_3 Status Message: indices element out of data bounds, idx=0 data_dim=0

2019-07-01 14:52:32.212096419 [E:onnxruntime:, sequential_executor.cc:123 Execute] Non-zero status code returned while running Node: generic_loop_Loop__69 Status Message: Non-zero status code returned while running Node: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Gather_90/Gather_3 Status Message: indices element out of data bounds, idx=0 data_dim=0

ERROR:main:thread: failed on contentid=['/home/jose/dev/cocoapi/val2017/000000458790.jpg'], Method run failed due to: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running Node: generic_loop_Loop__69 Status Message: Non-zero status code returned while running Node: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Gather_90/Gather_3 Status Message: indices element out of data bounds, idx=0 data_dim=0

TestScenario.SingleStream qps=16.94, mean=0.0590, time=60.45, queries=1024, tiles=50.0:0.0578,80.0:0.0618,90.0:0.0636,95.0:0.0672,99.0:0.0862,99.9:0.1354
```

The ssd-mobilenet model is derived from Tensorflow model "ssd_mobilenet_v1_coco_2018_01_28" and converted to ONNX using tf2onnx. I'm concerned about the correctness of this conversion. The tf2onnx maintainers note that there are "issues with control flow" in the conversion of this graph. See https://github.com/onnx/tensorflow-onnx/blob/master/tests/run_pretrained_models.yaml.

It looks like the model needs to be cleaned up for batch size 1 inference. The mlperf provided onnx graph for ssd-mobilenet has a bunch of preprocessing and postprocessing steps (loops, etc.) that don't look like they belong. See the Loop node in the netron screenshot below.

![2019-07-01-122632_2560x1600_scrot](https://user-images.githubusercontent.com/1208477/60466537-86ba9100-9c19-11e9-91c2-4e83d389e0ec.png)

Is ssd-mobilenet known to be working and accurate on the reference CPU model? If so how might I reproduce a result that achieves mAP 0.20 on CPU? If not what is the timeline for this to be working? I'm concerned that the ONNX graph for ssd-mobilenet needs some tidying before anyone can run serious benchmarks against it.

Thanks and regards.

The comment in tf2onnx/run_pretrained_models.yaml is really old ... we'll change it enable that test in tf2onnx.
The mAP for the onnx model is identical to tensorflow and I inspected some images manual. I have not seen any issue with the onnx model.

The input tensor for the onnx model is:
```
type: uint8[unk__5148,unk__5149,unk__5150,3]
aka
[-1.-1,-1,3]
```
which is the same as tensorflow ... it takes any batchsize, height and width as long there are 3 channels.
For mlperf the images are resized to 300x300 with [this tool](https://github.com/mlperf/inference/tree/master/v0.5/tools/upscale_coco).

As far I know glow is not on the latest onnx version and will have issues with the control flow in the graph as well has nms that came in with opset10.

onnxruntime-1.5.1 had some issues with a few images and also issued some warnings - those issues are  fixed in onnxruntime-master.




closing this one - don't think there is an issue.