Repository: tensorflow
Issue: TFLite Converter: how to skip Dequantize node in FP16 models for GPU Delegate?
Labels: stat:awaiting tensorflower,type:bug,TFLiteConverter,2.6.0

<em>Please make sure that this is a bug. As per our
[GitHub Policy](https://github.com/tensorflow/tensorflow/blob/master/ISSUES.md),
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template</em>

**System information**
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Yocto dunfell 
- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: aarch64
- TensorFlow installed from (source or binary):
- TensorFlow version (use command below): 2.6.1
- Python version: 3.9.9
- Bazel version (if compiling from source):
- GCC/Compiler version (if compiling from source):
- CUDA/cuDNN version:
- GPU model and memory: Mali G72


**Describe the current behavior**
I converted mobilenet_v2_1.0_224 [https://storage.googleapis.com/download.tensorflow.org/models/tflite_11_05_08/mobilenet_v2_1.0_224_quant.tgz](url) to fp16 tflite model using the steps mentioned here [https://www.tensorflow.org/lite/performance/post_training_quantization#float16_quantization](url)

the converted mobilenet_v2_fp16.tflite model has multiple Dequantize steps inserted within which seem to convert float16 to float32, even though the model is supposed to be quantized to float16.
Below is a snippet of the model visualization from netron. The inputs to the converted model are float32 and its outputs are also float32. 
 
![test](https://user-images.githubusercontent.com/78979784/155630251-9e58343d-95a9-420c-8101-86fed9165372.PNG)


All the dequantize nodes have float16 input and float32 output. Node 0 dequantize node properties are visualized below:

![image](https://user-images.githubusercontent.com/78979784/155630719-1c989e54-22d8-4423-846f-93eb911c22ff.png)

all dequantize nodes have similar properties where the inputs to the layer comprise of only weights and/or bias. 

now according to the official documentation, _"By default, a float16 quantized model will "dequantize" the weights values to float32 when run on the CPU. (Note that the GPU delegate will not perform this dequantization, since it can operate on float16 data.)"_ This means dequantize step is added just for cpu execution and gpu will not perform or override this node. But the question I have is, the dequantize node is already present, can a delegate simply skip a node in a model architecture during execution runtime, since the node is a part of the model graph and every node is executed with no op delegated to CPU? 


What is the main use of this dequantize node? And why is a dequantization step inserted during fp16 quantization? Is there a way to skip these dequantize nodes in the GPU delegate during post-training float16 quantization itself, for I want to run the model only on GPU and not CPU? The node seems to be executed by GPU delegate, as none of the actions are delegated to the CPU at runtime. Or are they being simply ignored by the GPU delegate?

mobilenet_v2_fp16.tflite model is attached for your reference.
[mobilenet_v2_fp16.zip](https://github.com/tensorflow/tensorflow/files/8137542/mobilenet_v2_fp16.zip)
 
thanks