| language: | |
| - ru | |
| license: apache-2.0 | |
| # Descripton: | |
| This is ruadapt version of upstage/SOLAR-10.7B-v1.0 model with tokenizer replacement. Additionally to previous work, the model was adapted in two stages: 1) vocabulary optimization, and 2) additional attention fine-tuning using LoRa. | |
| # How to cite: | |
| Tikhomirov M., Chernyshev D. Impact of Tokenization on LLaMa Russian Adaptation //arXiv preprint arXiv:2312.02598. – 2023. |