Issue 1:
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):no

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Ubuntu 16.04

TensorFlow installed from (source or binary):Source

TensorFlow version (use command below):1.8.0rc0

Python version:2.7.12

Bazel version (if compiling from source):0.12.0

GCC/Compiler version (if compiling from source):5.4.0

CUDA/cuDNN version:cuda-9.0/7.0

GPU model and memory:GeForce GTX 1080/8105MiB

Phone:xiaomi5 (Snapdragon 820)

Exact command to reproduce:

bazel run --config=opt //tensorflow/contrib/lite/toco:toco --

--input_file=/external_home/data/model/deeplabv3_mnv2_pascal_train_aug/frozen_inference_graph.pb

--output_file=/external_home/data/model/deeplabv3_mnv2_pascal_train_aug/kanul.tflite

--inference_type=QUANTIZED_UINT8

--input_shape=1,513,513,3

--input_array=sub_7

--output_array=logits/semantic/BiasAdd

Describe the problem

I have tried to quantize MobileNetV2 for deeplabV3+ with TFlite. But I fail to convert the model.

From the following issue, I saw that the operations were not supported for the option of quantization.

https://github.com/tensorflow/models/blob/master/research/deeplab/g3doc/model_zoo.md

Checkpoint name: mobilenetv2_coco_voc_trainaug

Who can explain and support to resolve the issue?

Source code / logs

bazel run --config=opt //tensorflow/contrib/lite/toco:toco --

--input_file=/external_home/data/model/deeplabv3_mnv2_pascal_train_aug/frozen_inference_graph.pb

--output_file=/external_home/data/model/deeplabv3_mnv2_pascal_train_aug/kanul.tflite

--inference_type=QUANTIZED_UINT8

--input_shape=1,513,513,3

--input_array=sub_7

--output_array=logits/semantic/BiasAdd

Unimplemented: this graph contains an operator of type SpaceToBatchND for which the quantized form is not yet implemented. Sorry, and patches welcome (that's a relatively fun patch to write, mostly providing the actual quantized arithmetic code for this op).

Issue 2:
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):no

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Ubuntu 16.04

TensorFlow installed from (source or binary):Source

TensorFlow version (use command below):1.9.0

Python version:2.7.12

Bazel version (if compiling from source):0.12.0

GCC/Compiler version (if compiling from source):5.4.0

CUDA/cuDNN version:cuda-9.0/7.0

GPU model and memory:GeForce GTX 1080/8105MiB

Phone:xiaomi5 (Snapdragon 820)

Exact command to reproduce:

bazel run --config=opt //tensorflow/contrib/lite/toco:toco --

--input_file=/external_home/data/model/deeplabv3_mnv2_pascal_train_aug/frozen_inference_graph.pb

--output_file=/external_home/data/model/deeplabv3_mnv2_pascal_train_aug/kanul.tflite

--inference_type=QUANTIZED_UINT8

--input_shape=1,513,513,3

--input_array=sub_7

--output_array=ResizeBilinear_3

Describe the problem

I have tried to quantize MobileNetV2 for deeplabV3+ with TFlite. But I fail to convert the model.

From the following issue, I saw that the operations were not supported for the option of quantization.

https://github.com/tensorflow/models/blob/master/research/deeplab/g3doc/model_zoo.md

Checkpoint name: mobilenetv2_coco_voc_trainaug

As we can see graphs from the tensorboard, there is one big problem.

In "import/MobilenetV2/expanded_conv_7/depthwise/depthwise",

the operation of depthwise consists of the subgraph with 3 nodes: (depthwise) and BatchToSpaceND, SpaceToBatchND.

But, in "import/MobilenetV2/expanded_conv_6/depthwise/depthwise",

the operation of depthwise is DepthwiseConv2dNative itself.

From the difference, we can not quantize deeplabv3 based on mobilenetv2.

The one thing is that MobilenetV2/expanded_conv_7~16 does not have min/max value to be needed for quantization with tflite.

Although I implement the needed min/max value in hardcode_min_max.cc,

This model does not run well in mobile environments.

The ultimate problem is caused by the fact that depthwise_conv_7~16 consist of 3 nodes with BatchToSpaceND and SpaceToBatchND.

I request you to notify the method to resolve above issues.

Source code / logs

bazel run --config=opt //tensorflow/contrib/lite/toco:toco --

--input_file=/external_home/data/model/deeplabv3_mnv2_pascal_train_aug/frozen_inference_graph.pb

--output_file=/external_home/data/model/deeplabv3_mnv2_pascal_train_aug/kanul.tflite

--inference_type=QUANTIZED_UINT8

--input_shape=1,513,513,3

--input_array=sub_7

--output_array=ResizeBilinear_3

2018-07-11 04:40:01.330069: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before pre-quantization graph transformations: 166 operators, 340 arrays (1 quantized)

2018-07-11 04:40:01.330711: W tensorflow/contrib/lite/toco/graph_transformations/hardcode_min_max.cc:339] Tweaking the MinMax of array ResizeBilinear_1, which is an input to {Concatenation operator with output concat}, because we want all inputs and outputs of a Concatenation operator to have the same MinMax so that it can be implemented as a pure byte-copy, no arithmetic.

2018-07-11 04:40:01.332983: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After pre-quantization graph transformations pass 1: 111 operators, 285 arrays (1 quantized)

2018-07-11 04:40:01.335731: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before quantization graph transformations: 111 operators, 285 arrays (1 quantized)

2018-07-11 04:40:01.337575: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_7/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.337670: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_7/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.337695: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_7/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.338553: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_8/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.338711: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_8/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.338786: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_8/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.339777: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_9/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.339918: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_9/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.339985: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_9/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.340933: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_10/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.341034: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_10/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.341059: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_10/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.342497: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_11/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.342593: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_11/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.342620: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_11/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.344311: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_12/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.344422: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_12/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.344452: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_12/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.345978: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_13/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.346094: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_13/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.346122: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_13/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.349163: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_14/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.349318: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_14/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.349351: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_14/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.353356: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_15/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.353511: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_15/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.353545: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_15/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.357264: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_16/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.357400: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_16/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

2018-07-11 04:40:01.357432: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_16/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.