T5 recipe ?
Hi
@SalomonMetre13
,
Congratulations on this unprecedented model, the BLEU scoring is impressive.
We are striving for the same translation quality for fula with https://huggingface.co/cawoylel
Would you consider sharing your approach in more details ?
Also is serving this model offline on edge (e.g. on mobile device) something you are interested in ?
There are a couple of ways to reach achieve this and wrap it in a dedicated app.
Hello
@flutter-painter
,
Thank you very much. Sure, I'd love to share my approach in more details.
I managed to serve the model on mobile device (an Android App) and on web & desktop through this progressive web app developed using Flutter.
But the app can only be used online so far.
Nice ! I saw some Dart in your github but did not expect you would also consider Flutter for the translation app !
On the flutter translation topic :
I would like to recommend you a github repo that also includes speech2text and text2speech that you might want to build upon.
On the fula translation topic :
I adapted this recipe on nllb and it worked perfectly : https://cointegrated.medium.com/how-to-fine-tune-a-nllb-200-model-for-translating-a-new-language-a37fc706b865
But the model is way to heavy. Where could I PM about you the recipe for finetuning t5 on a new language ?
On the offline first topic :
Having the app work fully offline will benefits both users (tool would be constantly available) and the owner (no running cost).
As you might be aware, tf-lite has some limits with flutter because of Flex ops. Though there are ways to overcome them, it does not seem solid enough to build upon (https://github.com/tensorflow/tensorflow/issues/45875).
I think ONNX is more robust, besides fonnx showed that using ffi it is possible to integrate elaborated models (e.g. whisper).
There is already a complete example of T5 inference in Java :
- https://github.com/microsoft/onnxruntime-inference-examples/issues/295
- please refer to https://onnxruntime.ai/docs/get-started/with-java.html
- and example code: https://github.com/microsoft/onnxruntime/blob/main/java/src/test/java/ai/onnxruntime/InferenceTest.java
I reckon porting this to Dart is not an easy task, but it would open a whole new realm for translation.