Repository: mlcommons/inference
Issue: ValueError: Could not open '/home/root/model/resnet50_v1.tflite'.
Root cause: Validation Gap

HI,
How do I fix following error?

>  $ ./run_local.sh tflite --profile ssd-mobilenet-tf
> INFO:main:Namespace(accuracy=False, backend='tflite', cache=0, count=None, data_format=None, dataset='coco-300', dataset_list=None, dataset_path='/home/root/data/coco', debug=False, find_peak_performance=False, inputs=['image_tensor:0'], max_batchsize=32, max_latency=None, mlperf_conf='../../mlperf.conf', model='/home/root/model/resnet50_v1.tflite', model_name='ssd-mobilenet', output='/home/root/inference/vision/classification_and_detection/output/tflite-cpu/resnet50', outputs=['num_detections:0', 'detection_boxes:0', 'detection_scores:0', 'detection_classes:0'], profile='ssd-mobilenet-tf', qps=None, samples_per_query=None, scenario='SingleStream', threads=4, time=None, user_conf='user.conf')
> INFO:coco:reduced image list, 4997 images not found
> INFO:coco:loaded 3 images, cache=0, took=19.8sec
> /home/root/inference/vision/classification_and_detection/python/coco.py:115: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
>   self.label_list = np.array(self.label_list)
> Traceback (most recent call last):
>   File "python/main.py", line 558, in <module>
>     main()
>   File "python/main.py", line 436, in main
>     model = backend.load(args.model, inputs=args.inputs, outputs=args.outputs)
>   File "/home/root/inference/vision/classification_and_detection/python/backend_tflite.py", line 43, in load
>     self.sess = tflite.Interpreter(model_path=model_path)
>   File "/usr/lib/python3.8/site-packages/tflite_runtime/interpreter.py", line 348, in __init__
>     _interpreter_wrapper.CreateWrapperFromFile(
> ValueError: Could not open '/home/root/model/resnet50_v1.tflite'.

Where can I get **resnet50_v1.tflite**?

Thanks


@guschmue 

@rnaidu02 @guschmue 
any solution for this?

Thanks

TF Lite is not part of [reference models](https://github.com/mlcommons/inference/tree/master/vision/classification_and_detection) format for ResNet50 and also not main stream format for data center/edge submissions. @psyhtest mentioned during last week inference meeting that he might have links to TF lite models. @psyhtest Any suggestions to @Ankita-020696 ?

@Ankita-020696 According to the [metadata](https://github.com/krai/ck-mlperf/blob/master/package/model-tflite-mlperf-resnet/.cm/meta.json) of our TFLite ResNet50 model (which we converted from TF and used since v0.5) you can download it from this [URL](https://www.dropbox.com/s/cvv2zlfo80h54uz/resnet50_v1.tflite.gz).

> @Ankita-020696 According to the [metadata](https://github.com/krai/ck-mlperf/blob/master/package/model-tflite-mlperf-resnet/.cm/meta.json) of our TFLite ResNet50 model (which we converted from TF and used since v0.5) you can download it from this [URL](https://www.dropbox.com/s/cvv2zlfo80h54uz/resnet50_v1.tflite.gz).

@psyhtest after using given model get following error. Can you suggest solution for this?

> $ ./run_local.sh tflite --profile ssd-mobilenet-tf
> INFO:main:Namespace(accuracy=False, backend='tflite', cache=0, count=None, data_format=None, dataset='coco-300', dataset_list=None, dataset_path='/home/root/data/coco', debug=False, find_peak_performance=False, inputs=['image_tensor:0'], max_batchsize=32, max_latency=None, mlperf_conf='../../mlperf.conf', model='/home/root/model/resnet50_v1.tflite', model_name='ssd-mobilenet', output='/home/root/inference/vision/classification_and_detection/output/tflite-cpu/resnet50', outputs=['num_detections:0', 'detection_boxes:0', 'detection_scores:0', 'detection_classes:0'], profile='ssd-mobilenet-tf', qps=None, samples_per_query=None, scenario='SingleStream', threads=4, time=None, user_conf='user.conf')
> INFO:coco:reduced image list, 4997 images not found
> INFO:coco:loaded 3 images, cache=0, took=19.9sec
> /home/root/inference/vision/classification_and_detection/python/coco.py:115: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
>   self.label_list = np.array(self.label_list)
> Traceback (most recent call last):
>   File "python/main.py", line 558, in <module>
>     main()
>   File "python/main.py", line 468, in main
>     _ = backend.predict({backend.inputs[0]: img})
>   File "/home/root/inference/vision/classification_and_detection/python/backend_tflite.py", line 57, in predict
>     self.sess.set_tensor(v, feed[k])
>   File "/usr/lib/python3.8/site-packages/tflite_runtime/interpreter.py", line 572, in set_tensor
>     self._interpreter.SetTensor(tensor_index, value)
> ValueError: Cannot set tensor: Got value of type UINT8 but expected type FLOAT32 for input 2, name: input_tensor 


Thanks


Hi @Ankita-020696, have you tried using the corresponding profile for it, not `-- profile ssd-mobilenet-tf`?

What are you trying to achieve? Might be easier to use CK for benchmarking TFLite.

```--profile ssd-mobilenet-tf``` and ```resnet50``` don't go along well (ssd-mobilenet wants uint8 as input, resnet50 wants fp32).
Maybe there is something tflite is a 2nd class citizen
The following command line should work (at least it does for me):
```
python python/main.py --mlperf_conf ../../mlperf.conf --model /data/NN/mlperf/resnet50/resnet50_v1.tflite --dataset-path /mnt/sdc/imagenette2-320  --output /tmp/test --profile resnet50-tf
```


Closing down due to inactivity.