Unable to load/use TFWav2Vec2ForCTC TFLite-model 
Author: stefan-falk
Pull request: no

## Environment info

- `transformers` version:
- Platform: Ubuntu 20.04
- Python version: 3.6
- PyTorch version (GPU?): 1.9.1
- Tensorflow version (GPU?): 2.6.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

@patrickvonplaten @anton-l 

### Description

> **General question first:** 
> 
> Is there actually _any_ speech-to-text model on 🤗 that is TFLite-ready which I could use right away?

I am trying to save a TFWav2Vec2 model as a `.tflite` but I am unable to load and run it after saving. 

It appears that I am running into this problem once I use `tf.lite.OpsSet.TFLITE_BUILTINS`:

```python
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
```


See the following stacktrace:

```none
Traceback (most recent call last):
  File "/Users/sfalk/workspaces/git/stt/src/ml/speech/bin/example.py", line 90, in <module>
    main()
  File "/Users/sfalk/workspaces/git/stt/src/ml/speech/bin/example.py", line 84, in main
    run_model(model_fp, model_id, audio_fp)
  File "/Users/sfalk/workspaces/git/stt/src/ml/speech/bin/example.py", line 64, in run_model
    interpreter.invoke()
  File "/Users/sfalk/miniconda3/envs/stt/lib/python3.9/site-packages/tensorflow/lite/python/interpreter.py", line 875, in invoke
    self._interpreter.Invoke()
RuntimeError: tensorflow/lite/kernels/conv.cc:349 input->dims->data[3] != filter->dims->data[3] (768 != 48)Node number 90 (CONV_2D) failed to prepare.
```

### Dependencies

```toml
[tool.poetry.dependencies]
python = "~3.9"
pydantic = "~1.8"
torchaudio = "~0.9.1"
sentencepiece = "~0.1.96"
librosa = "~0.8.1"
# Tensorflow
tensorflow = "~2.6.0"
# Huggingface
datasets = "~1.15.1"
transformers= "~4.12.3"
```

### Reproduce

```python
import os
import torch
import librosa as lb
import tensorflow as tf
from transformers import TFWav2Vec2ForCTC, Wav2Vec2Processor
from tensorflow.keras import layers
from tensorflow import keras
import numpy as np


def save_model(model_fp, model_id):

    model = TFWav2Vec2ForCTC.from_pretrained(model_id)

    inputs = layers.Input(shape=(None,), dtype=tf.float32)
    x = model(inputs)

    model = keras.Model(inputs=inputs, outputs=x)

    converter = tf.lite.TFLiteConverter.from_keras_model(model)

    converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
    converter.target_spec.supported_types = [tf.float16]
    # converter.optimizations = [tf.lite.Optimize.DEFAULT]
    converter.experimental_new_converter = True

    tflite_model = converter.convert()

    print(f"Writing model {model_fp}")
    with tf.io.gfile.GFile(model_fp, "wb") as f:
        f.write(tflite_model)


def run_model(model_fp, model_id, audio_fp: str = None, sampling_rate = 16000):

    # Initialize the tokenizer
    tokenizer = Wav2Vec2Processor.from_pretrained(model_id, sampling_rate=sampling_rate)

    if audio_fp is not None:
        waveform, rate = lb.load(audio_fp, sr=sampling_rate)
    else:
        waveform = np.random.rand(sampling_rate)

    input_values = tokenizer(waveform, return_tensors="tf").input_values

    interpreter = tf.lite.Interpreter(model_path=model_fp)

    # Get input and output tensors.
    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()

    # Test model on random input data.
    b, _ = input_details[0]["shape"]
    input_shape = b, len(waveform)

    interpreter.resize_tensor_input(0, input_shape, strict=True)

    interpreter.allocate_tensors()

    input_data = input_values  # np.array(np.random.random_sample(input_shape), dtype=np.float32)
    interpreter.set_tensor(input_details[0]["index"], input_data)

    interpreter.invoke()

    # The function `get_tensor()` returns a copy of the tensor data.
    # Use `tensor()` in order to get a pointer to the tensor.
    output_data = interpreter.get_tensor(output_details[0]["index"])

    predicted_ids = torch.argmax(torch.Tensor(output_data), dim=-1)
    transcription = tokenizer.batch_decode(predicted_ids)

    print(transcription)


def main():
    model_fp = "/tmp/fb-wav2vec2.tflite"
    model_id = "facebook/wav2vec2-base-960h"

    audio_fp = None 

    save_model(model_fp, model_id)
    run_model(model_fp, model_id, audio_fp)

    print("All done.")


if __name__ == "__main__":
    main()
```


Comments:
Hi @stefan-falk! Generally we don't support or test models for TFLite compatibility due to its OpsSet limitations, but we're open to contributions and suggestions.

Also pinging @Rocketknight1 @merveenoyan in case we have some recent developments in the TFLite department :slightly_smiling_face: 

I'm afraid not - TFLite is very limited in my experience! We're not actively working on a way to port our models to it.

Alright, thanks for the quick response guys!

I'll close this issue then. 

@stefan-falk probably the following [page](https://pythonrepo.com/repo/vasudevgupta7-gsoc-wav2vec2-python-natural-language-processing) could be interesting for you.