T2TT finetuning

#44
by vipinkatara - opened

Hello, can anyone point to where i can find article for finetuning m4t for T2TT task?

you can check T2TT task in this notebook: https://colab.research.google.com/github/ylacombe/scripts_and_notebooks/blob/main/v2_seamless_m4t_hugging_face.ipynb#scrollTo=l03ja7s5kT0Y
tuning: https://huggingface.co/docs/transformers/main/en/model_doc/seamless_m4t#transformers.SeamlessM4TModel

Otherwise, I can send this code for you.

import torch

device = "cuda:0" if torch.cuda.is_available() else "cpu"
model = model.to(device)

from transformers import SeamlessM4Tv2Model, AutoProcessor
from transformers import AutoProcessor

processor = AutoProcessor.from_pretrained("facebook/seamless-m4t-v2-large")
model = SeamlessM4Tv2Model.from_pretrained("facebook/seamless-m4t-v2-large")

process input

text_inputs = processor(text = "Hello, my dog is cute", src_lang="eng", return_tensors="pt").to(device)

generate translation

output_tokens = model.generate(**text_inputs, tgt_lang="fra", generate_speech=False)
translated_text_from_text = processor.decode(output_tokens[0].tolist()[0], skip_special_tokens=True)
print(f"Translation from text: {translated_text_from_text}")

Sign up or log in to comment