Repository: mlcommons/inference
Issue: ssd-resnet34: Default MaxPoolingOp only supports NHWC on device type CPU
Root cause: Architectural Constraint

Model: ssd_resnet-34_from_onnx.pb, which has shape '(1, 3, 1200, 1200)'
python: 3.7

Hi all, I am running object detection as `./run_and_time.sh tf ssd-resnet34 cpu` and getting the below error:

`InvalidArgumentError (see above for traceback): Default MaxPoolingOp only supports NHWC on device type CPU`


I'm seeing the same issue when running the test in a container as @vilmara did (using ssd_resnet-34_from_onnx.pb from https://zenodo.org/record/3060467/files/ssd_resnet-34_from_onnx.zip as referenced in https://github.com/mlperf/inference/blob/master/others/cloud/single_stage_detector/tensorflow/download_model.sh).

However, I don't see it when I manually compile the benchmark following the directions in https://github.com/mlperf/inference/tree/master/v0.5/classification_and_detection and use run_local. Even when I modify Dockerfile.cpu to closely reproduce those directions I still see the issue when run in a container. The biggest remaining difference is that I am running Ubuntu 18.04 locally and 16.04 in the container.

Still see issue with Docker container built on Ubuntu 18.04

can you try the latest TF model updated in the readme recently and see if you still have the same issue? It works for me on both CPU and GPU with TF. @vilmara @DaveJaffe 

hi @christ1ne, thanks. Did you run it local or as docker container?

@guschmue Can you take a look at this?
@christ1ne, running it as run_local now works with the new TF model but running it as a container with run_and_time fails in a different way from before:
root@ubuntu3:/home/ubuntu/mlperf_inference/v0.5/classification_and_detection# ./run_and_time.sh tf ssd-resnet34 cpu
...
Traceback (most recent call last):
  File "python/main.py", line 526, in <module>
    main()
  File "python/main.py", line 481, in main
    sut = lg.ConstructSUT(issue_queries, process_latencies)
TypeError: ConstructSUT(): incompatible function arguments. The following argument types are supported:
    1. (arg0: Callable[[List[mlperf_loadgen.QuerySample]], None], arg1: Callable[[], None], arg2: Callable[[List[int]], None]) -> int

Invoked with: <function main.<locals>.issue_queries at 0x7ff6fbf3ca60>, <function main.<locals>.process_latencies at 0x7ff6fbf3cb70>

See https://gist.github.com/DaveJaffe/561201c087a3e5fc694f712a827a3600 for full output

@guschmue please take a look and update the container code if needed.

I think there are multiple issues here:

./run_and_time.sh tf ssd-resnet34 cpu reports 'Default MaxPoolingOp only supports NHWC'
Yes, that is true  - the latest resnet34_tf.22.1.pb comes as NCHW and tensorflow does not support this for cpu.

ssd_resnet-34_from_onnx.pb only supports bs=1 and it is not trivial to change.  
I made [this change](https://github.com/mlperf/inference/blob/master/v0.5/classification_and_detection/python/main.py#L157) some time ago to limit the batch size to 1 for this specific model. 
Not great but because for ssd-resnet34 computation is high and sending larger batches might not change perf.
I want to see if we can convert the onnx model from the tensorflow model but the latest model has some complication that we are still looking at.

@guschmue 
Guenther, I am also seeing the NCHW not supported error with run_and_time.sh but the same command run locally *works* despite the fact that the format is still NCHW:

root@ubuntu1:~/mlperf_inference/v0.5/classification_and_detection# ./run_local.sh tf ssd-resnet34 cpu
INFO:main:Namespace(accuracy=False, backend='tensorflow', cache=0, count=None, data_format='NCHW', dataset='coco-1200-tf', dataset_list=None, dataset_path='/data/1/coco/dataset-coco-2017-val', inputs=['image:0'], max_batchsize=32, max_latency=[0.1], model='/root/mlperf_inference/v0.5/classification_and_detection/resnet34_tf.22.1.pb', output='/root/mlperf_inference/v0.5/classification_and_detection/output/ssd-resnet34-tf-cpu/results.json', outputs=['detection_bboxes:0', 'detection_classes:0', 'detection_scores:0'], profile='ssd-resnet34-tf', qps=10, queries_multi=24576, queries_offline=24576, queries_single=1024, scenario=[TestScenario.SingleStream], threads=96, time=None)
...
I0806 18:22:39.178235 140692671305536 main.py:527] starting TestScenario.SingleStream
TestScenario.SingleStream qps...

any update on this issue? it's now a blocker issue for us. we are using the pb file from readme link as https://zenodo.org/record/3345892/files/tf_ssd_resnet34_22.1.zip?download=1

I'm not really the model owner for this model (last version was published by @pkanwar23 I think) but my 5 cents:

There is nothing really wrong with the model, aka there is no real issue here.
Ok, I understand if you depend on NHWC its a problem for you but mlperf in large will be ok since this model is so costly that you need a gpu anyway. And the model will work on tensorflow/gpu .

If one needs this model as NHWC there are 2 options:
- train it with NHWC 
- write a tool to transpose the model from NCHW to NHWC (possible, we have revere issue with onnx  and during conversion from tensorflow to onnx we transpose the model)


> I'm not really the model owner for this model (last version was published by @pkanwar23 I think) but my 5 cents:
> 
> There is nothing really wrong with the model, aka there is no real issue here.
> Ok, I understand if you depend on NHWC its a problem for you but mlperf in large will be ok since this model is so costly that you need a gpu anyway. And the model will work on tensorflow/gpu .
> 
> If one needs this model as NHWC there are 2 options:
> 
> * train it with NHWC
> * write a tool to transpose the model from NCHW to NHWC (possible, we have revere issue with onnx  and during conversion from tensorflow to onnx we transpose the model)

ok, I am going to do the second option, which is what in mind when i found this issue at the first place, if no one else is doing it. 

This model is advertised as NHWC on the README page though, i think you need to correct that to avoid confusion.

True - we need to fix the readme (I think some earlier version was NHWC).

@aaronzhongii - Any success on writing a tool for transposing the model?

We tried and failed. We are looking at other options right now.

Aaron





come from dingtalk mailbox------------------------------------------------------------------
Sender:Shama Yazdani<notifications@github.com>
Date:2019/11/01 03:01:31
Recipient:mlperf/inference<inference@noreply.github.com>
Cc:Rong, Zhong<aaron.zhong@alibaba-inc.com>; Mention<mention@noreply.github.com>
Theme:Re: [mlperf/inference] ssd-resnet34: Default MaxPoolingOp only supports NHWC on device type CPU (#219)

@aaronzhongii - Any success on writing a tool for transposing the model?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe. 


can you check if you install cudnn?
```
ii  libcudnn7                                       7.6.5.32-1+cuda10.0                             amd64        cuDNN runtime libraries
ii  libcudnn7-dev                                   7.6.5.32-1+cuda10.0                             amd64        cuDNN development libraries and headers
```
note that cudnn should match your cuda version, download them here: https://developer.nvidia.com/rdp/cudnn-download

@vilmara  Cpu only supports channel_last format model , I update channel_last pb model on https://zenodo.org/record/3708755#.XmszXqgzY2w.

> @vilmara Cpu only supports channel_last format model , I update channel_last pb model on https://zenodo.org/record/3708755#.XmszXqgzY2w.

accuracy validated mAP=22.761% on both cpu and gpu. Thanks for the help.

> @aaronzhongii - Any success on writing a tool for transposing the model?

Finally I was able to generate the script to convert the nhwc ssd model to nhwc by inserting transpose at designated locations. check https://github.com/mlperf/inference/pull/625 for details.