Skip a subset of tokens during translation

by anothersherlock - opened Dec 30, 2023

Dec 30, 2023

Is it possible to skip some systematically delineated tokens?
[CITY] is beautiful! -> [CITY] est belle!

anothersherlock changed discussion title from Skip a subset of tokens during translations to Skip a subset of tokens during translation Dec 30, 2023

jbochi

Owner Jan 2

This is not supported. The easiest approach is probably to replace [CITY] by a city name and then replace it back after translation.

You can also fine tune the model. Peft should work well. It supports t5 models.

jbochi changed discussion status to closed Jan 2

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment