Repository: mlcommons/inference
Issue: TensorFlow Resnet50 v1.5 Pre-trained in NCHW data format
Root cause: Unimplemented Feature Gap

Hi, could you please provide the TensorFlow Resnet50 v1.5 Pre-trained in NCHW data format to run the inference located at inference/cloud/image_classification/?

Also could you please confirm what is the precision mode of the pre-trained model https://zenodo.org/record/2535873/files/resnet50_v1.pb


Thanks,

Vilmara

https://zenodo.org/record/2535873/files/resnet50_v1.pb was trained as fp32

Should be visible to publish a NCHW flavor, let me check with a few people first (it is easy for resnet50 but in theory we might need to do the same for other models).

When I try to run inference on with with a NCHW model I get the following error.
```
2019-05-04 13:45:18.523575: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x562c16a7e780 executing computations on platform Host. Devices:
2019-05-04 13:45:18.523609: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
ERROR:main:execute_parallel thread: Cannot feed value of shape (1, 224, 224, 3) for Tensor 'data:0', which has shape '(1, 3, 224, 224)'
ERROR:main:execute_parallel thread: Cannot feed value of shape (1, 224, 224, 3) for Tensor 'data:0', which has shape '(1, 3, 224, 224)'
```

I do understand it is complaining about the NCHW format while it expects the NHWC format but do not know where to change this. Changing the model could be one option, but I want to make mlperf work with NHWC  
Any help is much appreciated!

If you are using https://github.com/mlperf/inference/blob/master/cloud/image_classification/python/main.py it is pretty easy:
by default we use the backend's preferred data-format, ie. for tensorflow we use NHWC, for onnx we
use NCHW.

You can override it in the command line with ```--data-format NCHW```

hi @guschmue, I ran the test with the flag `--data-format=NCHW` but the system has thrown the below error **ERROR:main:execute_parallel thread: Cannot feed value of shape (1, 3, 224, 224) for Tensor 'input_tensor:0', which has shape '(?, 224, 224, 3)':**

```
ERROR:main:execute_parallel thread: Cannot feed value of shape (1, 3, 224, 224) for Tensor 'input_tensor:0', which has shape '(?, 224, 224, 3)'
Traceback (most recent call last):
  File "python/main.py", line 382, in <module>
    main()
  File "python/main.py", line 353, in main
    report_result("check_accuracy", 0., final_results, result_list, result_dict, check_acc=True)
  File "python/main.py", line 201, in report_result
    buckets = np.percentile(result_list, percentiles).tolist()
  File "/opt/anaconda3/lib/python3.6/site-packages/numpy/lib/function_base.py", line 3707, in percentile
    a, q, axis, out, overwrite_input, interpolation, keepdims)
  File "/opt/anaconda3/lib/python3.6/site-packages/numpy/lib/function_base.py", line 3826, in _quantile_unchecked
    interpolation=interpolation)
  File "/opt/anaconda3/lib/python3.6/site-packages/numpy/lib/function_base.py", line 3405, in _ureduce
    r = func(a, **kwargs)
  File "/opt/anaconda3/lib/python3.6/site-packages/numpy/lib/function_base.py", line 3941, in _quantile_ureduce_func
    x1 = take(ap, indices_below, axis=axis) * weights_below
  File "/opt/anaconda3/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 189, in take
    return _wrapfunc(a, 'take', indices, axis=axis, out=out, mode=mode)
  File "/opt/anaconda3/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 56, in _wrapfunc
    return getattr(obj, method)(*args, **kwds)
IndexError: cannot do a non-empty take from an empty axes.
```
Does MLPerf expect to release the TensorFlow Resnet50 v1.5 Pre-trained model in NCHW data format to run the benchmark with GPU support? 

syedaffanhamdani's issue:
```
Cannot feed value of shape (1, 224, 224, 3) for Tensor 'data:0', which has shape '(1, 3, 224, 224)'
he feeds NHWC but the model is NCHW

The fix for his model is --data-format NCHW
```
Your model:
```
Cannot feed value of shape (1, 3, 224, 224) for Tensor 'input_tensor:0', which has shape '(?, 224, 224, 3)
You feed NCHW bit the model wants NHWC

basically don't use  --data-format NCHW 
```

If you want a NCHW model we need to publish one.