Prune the model for certain languages
What is the most efficient way to prune model for one or two languages so it is smaller and less memory intensive?
If you only care about a subset of languages would recommend finetuning a new mt5 base using the released data filtered for only a subset of languages -- https://huggingface.co/datasets/CohereForAI/aya_collection. Typically models suffer from the curse of multilinguality -- so more capacity is needed to model more languages. If you reduce the number of languages and finetune on a smaller subset -- you will likely see bumps in performance for the languages you care about.
Modifying weights to selectively retain knowledge is an area of active research, but there are not yet very precise methods for ensuring when your prune you don't erode performance on languages you care about.