Repository: mlcommons/inference
Issue: Running Offline with tflite on mobilenet
Root cause: Configuration Error

![2](https://user-images.githubusercontent.com/55476509/79411592-2517b780-7fde-11ea-8293-709bf4fe1a01.png)

I tried to run Offline with tflite on mobilenet. 

I got message "Got 32 but expected 1 for dimension 0 of input 88"

I got similar message when I run Multi Stream scenario when I changed samples per query.

I guess 32 stands for number of query.

Q) I wonder if I let it be like that, it will be run and I can get result or it never return valid result because something is wrong. 

I let be like that pretty long and it takes long time. So I wonder what will be happen.

Thank you! 






32 is the batch size. Maybe after conversion the tflite model takes as input only batchsize 1?

@guschmue 
What do you mean after conversion~?
Maybe in this context tflite can not handle Offline and Multi Stream scenario?


when you convert to tflite you'd normally specify the shape, like here:
https://github.com/mlperf/inference/blob/master/v0.5/classification_and_detection/tools/resnet50-to-tflite.sh#L22
If the first dim (aka the batchsize) is not None, you can pass only that specific batchsize, else you get the error you see. You can pass in ```--batchsize 1``` into the mlperf app but that is basically the same as SingleStream.


@guschmue 
I used --max-batchsize. It seems working with --maxbatchsize 1 as you said.

dongkwan@tux ~/바탕화면/inference-master/tmp/inference/v0.5/classification_and_detection $ ./run_local.sh mobilenet cpu tflite --max-batchsize 64 --scenario Offline
model//mobilenet_v1_1.0_224.tflite
test from main
INFO:main:Namespace(accuracy=False, backend='tflite', cache=0, count=None, data_format=None, dataset='imagenet_mobilenet', dataset_list=None, dataset_path='imagenet_data/imagenet/imgs/', inputs=['input:0'], max_batchsize=64, max_latency=[0.01], model='model//mobilenet_v1_1.0_224.tflite', output='/home/dongkwan/바탕화면/inference-master/tmp/inference/v0.5/classification_and_detection/output/mobilenet-tflite-cpu/results.json', outputs=['MobilenetV1/Predictions/Reshape_1:0'], profile='mobilenet-tf', qps=10, queries_multi=24576, queries_offline=24576, queries_single=1024, scenario=[TestScenario.Offline], threads=4, time=None)
WARNING: Logging before flag parsing goes to stderr.
I0423 17:01:10.103033 139871337260352 imagenet.py:79] reduced image list, 386 images not found
I0423 17:01:10.103447 139871337260352 imagenet.py:82] loaded 200 images, cache=0, took=0.0sec
INFO: Initialized TensorFlow Lite runtime.
I0423 17:01:10.214162 139871337260352 main.py:569] starting TestScenario.Offline
E0423 17:01:10.458388 139869703632640 main.py:337] thread: failed on contentid=['imagenet_data/imagenet/imgs/ILSVRC2012_val_00000082.JPEG',
.
.
. 
'imagenet_data/imagenet/imgs/ILSVRC2012_val_00000059.JPEG'], Cannot set tensor: Dimension mismatch. Got 64 but expected 1 for dimension 0 of input 88.

Is there way I can change the shape of tflte offline? 
or maybe tflite only take input size as 1 ? 


@guschmue 
When I run tflite on other backend it said samples per query is 24576 and query was 1. I thought that was batchsize. Isn't it? 

I run it with --max-batchsize one and it said samples per query is 24576.

For Offline scenario all samples come in 1 query. By default we use a batchsize of 32 to send this to the runtime but if the model has a input shape [1,...] it supports only batchsize 1 and --max-batchsize will make us send that. Of course that won't do much good for your performance - one would need to check if that model can be converted with a None for batchsize.

@guschmue  Thank you always!

@guschmue I wonder the relationship maxbatchsize and samples per query 

they are independent from each other.