kaitchup

/

Llama-2-7b-mt-Czech-to-English

like 4

Hi,

Yes, I have a plan to fine-tune Mistral 7B for translation but I can only confirm that I will fine-tune for translation into English. For the translation into other languages such as Romanian, I have to confirm first that the translation quality is not too bad.
Note that I don't know when I will release these models (maybe in December).

Hello,
thank you for answer. On which dataset do you finetune a model for translation into a language ? For how long to achieve a good result ? How much do you estimate can cost such a finetuning of a mistral or zephyr base model into a bilingual translation model.
Thank you,
Catalin

I wrote an article explaining the entire process here:
https://kaitchup.substack.com/p/llama-2-mt-turn-llama-2-into-a-translation

"Moreover, since Llama 2 is pre-trained to generate English, I only considered fine-tuning Llama 2 to translate into English. In preliminary experiments, I tried some reverse directions, for instance, English to French, but I judged that the translation quality I obtained wasn’t good enough." - to overcome this you can try to use as base model for finetuning this one HuggingFaceH4/zephyr-7b-beta . Using zephyr it translate english text into romanian intelligible, but it invent some words ... something like what children does sometime...so I guess he have a limited vocabulary and it need more finetune on the translation task . I observed that Zephyr performs much better than Llama or mistral on translation tasks .

Hello, any news about training translation fine tuning with mistral or zephyr (zephyr I think is the best as it is showing good multilanguage skills natively , yet it needs to be polished )? In case you do not know yet I want to highlight for your attention a project that promise a substantial acceleration of fine tuning llama 2 models 2x-5x free version and up to 30x premium version at a future unspecified yet price . It also promise that it will reduce the amount of video memory needed with 40-60%. Here is their sales pitch:

"2-5x faster 50% less memory local LLM finetuning
Manual autograd engine - hand derived backprop steps.
2x to 5x faster than QLoRA. 50% less memory usage.
All kernels written in OpenAI's Triton language.
0% loss in accuracy - no approximation methods - all exact.
No change of hardware necessary. Supports NVIDIA GPUs since 2018+. Minimum CUDA Compute Capability 7.0 (V100, T4, Titan V, RTX 20, 30, 40x, A100, H100, L40 etc) Check your GPU
Flash Attention v2 support via Xformers.
NEW! Works on Linux and Windows via WSL.
NEW! Experimental support for DPO (Direct Preference Optimization)!
Supports 4bit and 16bit QLoRA / LoRA finetuning via bitsandbytes.
Train Slim Orca fully locally in 260 hours from 1301 hours (5x faster)."

• here is their project : https://github.com/unslothai/unsloth .

I want to mention that I have managed to fine tune on my laptop with 16 gb ram and nvidia rtx 4050 6 gb video ram a 7b llama 2 model . This looks amazing for me and I'm very excited about this ! I did not yet test to see how effective is this finetuning (I'm a noob and still learning this amazing field of AI :) ) BUT it is a huge step forward the ability to train locally a 7b model...or at least I feel it is so !

Thank you for your feedback!
I tried fine-tuning Mistral for translation into other languages than English. For now, my results are disappointing and not as good as with Llama 2. But I didn't give up yet.
I don't think fine-tuning Zephyr would work. Zephyr is already a fine-tuned model with a specific prompt format. To the best of my knowledge, Zephyr has only been fine-tuned on English data so it should have the same multilingual capabilities as Mistral.

Zephyr is the only 7b model with which I have a conversation in romanian language without any fine tuning (it still invent some words or make some mistakes BUT it is surprisingly good and make intelligible conversations). I read somewhere that it was finetuned with 104 languages dataset (70 % English or so BUT is a BIG step forward that it allready works in other languages) The 200k chat which was used to fine tune it I think it is a multilingual dataset. If you do not want to train it yourself can you please make a script or a notebook for training zephyr for translation (with ability to save checkpoints and reload the last checkpoint) and if possible taking advantage by the unsloth technology upgrade and post it for community to play with it ? Please! Thank you ! Catalin

The stakes are very high in this . If it is possible to have a good 7b model translator from english to any language => possibility to translate vast amount of good books of science in those respective languages and open the access to information for many peoples arround the world for whom language is a barrier .