Repository: tensorflow
Issue: Failing to convert MobileNetV3Large to TFLite w/ Integer q
Labels: stat:awaiting tensorflower,type:bug,comp:lite,TFLiteConverter,TF 2.16

### 1. System information

- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 and Windows 10 WSL
- TensorFlow installation (pip package or built from source):  2.10 (on Win 10) and 2.16.2 (on WSL)

### 2. Code

```
import tensorflow as tf
import numpy as np
from tensorflow.keras.applications import MobileNetV3Large
from tensorflow.keras.applications.mobilenet_v3 import preprocess_input
import matplotlib.pyplot as plt
from scipy.stats import pearsonr

# Generate one sample image for testing
test_image = np.random.normal(loc=127.5, scale=50, size=(1, 224, 224, 3))
test_image = np.clip(test_image, 0, 255).astype(np.float32)
preprocessed_image = preprocess_input(test_image.copy())

# Load model
model = MobileNetV3Large(
    weights='imagenet',
    include_top=True,
    input_shape=(224, 224, 3)
)

# Get original prediction
original_pred = model.predict(preprocessed_image, verbose=0)

# Convert to TFLite
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]  # Enable dynamic range quantization
converter.target_spec.supported_ops = [
    tf.lite.OpsSet.TFLITE_BUILTINS,
    tf.lite.OpsSet.SELECT_TF_OPS
]
converter._experimental_disable_per_channel = True
converter.experimental_new_converter = True

# Convert
tflite_model = converter.convert()

# Get TFLite prediction
interpreter = tf.lite.Interpreter(model_content=tflite_model)
interpreter.allocate_tensors()

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

interpreter.set_tensor(input_details[0]['index'], preprocessed_image)
interpreter.invoke()
tflite_pred = interpreter.get_tensor(output_details[0]['index'])

# Calculate correlation
correlation, _ = pearsonr(original_pred.flatten(), tflite_pred.flatten())

# Visualize
plt.figure(figsize=(10, 5))

# Scatter plot
plt.subplot(1, 2, 1)
plt.scatter(original_pred.flatten(), tflite_pred.flatten(), alpha=0.5)
plt.plot([original_pred.min(), original_pred.max()],
         [original_pred.min(), original_pred.max()],
         'r--', label=f'Perfect Correlation\nActual: {correlation:.4f}')
plt.title('Original vs Quantized Predictions')
plt.xlabel('Original Model')
plt.ylabel('Quantized Model')
plt.legend()

# Distribution plot
plt.subplot(1, 2, 2)
plt.hist(np.abs(original_pred.flatten() - tflite_pred.flatten()),
         bins=50, alpha=0.75, label='Prediction Differences')
plt.title('Distribution of Prediction Differences')
plt.xlabel('|Original - Quantized|')
plt.ylabel('Count')
plt.legend()

plt.tight_layout()
plt.show()

print(f"\nResults:")
print(f"Prediction correlation: {correlation:.4f}")
print(f"Original model size: {len(model.get_weights()) / 1024 / 1024:.2f} MB")
print(f"Quantized model size: {len(tflite_model) / 1024 / 1024:.2f} MB")
print(f"Size reduction: {(1 - len(tflite_model) / len(model.get_weights())) * 100:.1f}%")
```

### 3. Failure after conversion
1. TF 2.10 in Win10 Log:
Model produces wrong results. See plot made from code:
![image](https://github.com/user-attachments/assets/8823cbfc-88d9-4e2d-9ef7-c8a2adc3ef0a)

2. TF2.16 in WSL:
Model fails to convert. Gets error: `LLVM ERROR: Failed to infer result type(s).` (see log)


### 5. (optional) Any other info / logs
I ran this on 2 systems:

1. TF 2.10 in Win10 Log:
```
import sys; print('Python %s on %s' % (sys.version, sys.platform))
D:\code\ai_dev\venv\Scripts\python.exe "C:/Program Files/JetBrains/PyCharm 2023.2.4/plugins/python/helpers/pydev/pydevd.py" --multiprocess --qt-support=auto --client 127.0.0.1 --port 54366 --file C:\Users\Administrator\AppData\Roaming\JetBrains\PyCharm2023.2\scratches\tfmodel_tflite.py 
Connected to pydev debugger (build 232.10203.26)
2024-12-26 15:17:07.215039: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-26 15:17:07.670016: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 7423 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3080, pci bus id: 0000:0a:00.0, compute capability: 8.6
2024-12-26 15:17:11.111912: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8906
2024-12-26 15:17:12.036555: I tensorflow/stream_executor/cuda/cuda_blas.cc:1614] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 5 of 64). These functions will not be directly callable after loading.
2024-12-26 15:17:44.746406: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:362] Ignored output_format.
2024-12-26 15:17:44.746529: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:365] Ignored drop_control_dependency.
2024-12-26 15:17:44.747230: I tensorflow/cc/saved_model/reader.cc:45] Reading SavedModel from: C:\Users\ADMINI~1\AppData\Local\Temp\tmprjeve5nr
2024-12-26 15:17:44.771028: I tensorflow/cc/saved_model/reader.cc:89] Reading meta graph with tags { serve }
2024-12-26 15:17:44.771129: I tensorflow/cc/saved_model/reader.cc:130] Reading SavedModel debug info (if present) from: C:\Users\ADMINI~1\AppData\Local\Temp\tmprjeve5nr
2024-12-26 15:17:44.886049: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
2024-12-26 15:17:44.904668: I tensorflow/cc/saved_model/loader.cc:229] Restoring SavedModel bundle.
2024-12-26 15:17:45.275249: I tensorflow/cc/saved_model/loader.cc:213] Running initialization op on SavedModel bundle at path: C:\Users\ADMINI~1\AppData\Local\Temp\tmprjeve5nr
2024-12-26 15:17:45.402632: I tensorflow/cc/saved_model/loader.cc:305] SavedModel load for tags { serve }; Status: success: OK. Took 655396 microseconds.
2024-12-26 15:17:45.811466: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:268] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
```

3. TF 2.16.2 in WSL log:
```
/root/ai_dev/.venv/bin/python /root/.pycharm_helpers/pydev/pydevd.py --multiprocess --qt-support=auto --client 127.0.0.1 --port 55955 --file /mnt/c/Users/Administrator/AppData/Roaming/JetBrains/PyCharm2023.2/scratches/tfmodel_tflite.py 
Connected to pydev debugger (build 232.10203.26)
2024-12-26 15:18:56.434366: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-12-26 15:18:57.803991: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-12-26 15:18:58.473972: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:479] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-26 15:18:58.972456: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:10575] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-26 15:18:58.975123: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1442] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-26 15:18:59.910805: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-26 15:19:06.124533: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-12-26 15:19:15.989425: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:984] could not open file to read NUMA node: /sys/bus/pci/devices/0000:0a:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-26 15:19:17.013008: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2251] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
W0000 00:00:1735255170.947341  469943 tf_tfl_flatbuffer_helpers.cc:390] Ignored output_format.
W0000 00:00:1735255170.947401  469943 tf_tfl_flatbuffer_helpers.cc:393] Ignored drop_control_dependency.
2024-12-26 15:19:30.948060: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: /tmp/tmp9lkkwnp9
2024-12-26 15:19:30.953309: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve }
2024-12-26 15:19:30.953334: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: /tmp/tmp9lkkwnp9
2024-12-26 15:19:31.011901: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:388] MLIR V1 optimization pass is not enabled
2024-12-26 15:19:31.020594: I tensorflow/cc/saved_model/loader.cc:234] Restoring SavedModel bundle.
2024-12-26 15:19:31.231606: I tensorflow/cc/saved_model/loader.cc:218] Running initialization op on SavedModel bundle at path: /tmp/tmp9lkkwnp9
2024-12-26 15:19:31.297302: I tensorflow/cc/saved_model/loader.cc:317] SavedModel load for tags { serve }; Status: success: OK. Took 349244 microseconds.
2024-12-26 15:19:31.779723: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:268] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
loc(fused["ReadVariableOp:", callsite("MobileNetV3Large_1/conv_1/convolution/ReadVariableOp@__inference_serving_default_5035"("/root/.pycharm_helpers/pydev/pydevd.py":2199:1) at callsite("/root/.pycharm_helpers/pydev/pydevd.py":2181:1 at callsite("/root/.pycharm_helpers/pydev/pydevd.py":1493:1 at callsite("/root/.pycharm_helpers/pydev/pydevd.py":1500:1 at callsite("/root/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py":18:1 at callsite("/mnt/c/Users/Administrator/AppData/Roaming/JetBrains/PyCharm2023.2/scratches/tfmodel_tflite.py":34:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/tensorflow/lite/python/lite.py":1175:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/tensorflow/lite/python/lite.py":1129:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/tensorflow/lite/python/lite.py":1636:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/tensorflow/lite/python/lite.py":1614:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/tensorflow/lite/python/convert_phase.py":205:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/tensorflow/lite/python/lite.py":1537:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/backend/tensorflow/layer.py":58:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/backend/tensorflow/layer.py":112:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py":117:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/layers/layer.py":899:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py":117:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/ops/operation.py":46:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py":156:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/models/functional.py":182:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/ops/function.py":171:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/models/functional.py":632:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py":117:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/layers/layer.py":899:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py":117:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/ops/operation.py":46:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py":156:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/layers/convolutional/base_conv.py":243:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/layers/convolutional/base_conv.py":233:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/ops/nn.py":1183:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/backend/tensorflow/nn.py":301:1 at callsite("/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/backend/tensorflow/nn.py":274:1 at "/root/ai_dev/.venv/lib/python3.10/site-packages/keras/src/backend/tensorflow/core.py":85:1))))))))))))))))))))))))))))))))]): error: missing attribute 'value'
LLVM ERROR: Failed to infer result type(s).

Process finished with exit code 134
```