README.md

---
license: apache-2.0
language:
- bo

---

<!-- Provide a longer summary of what this model is. -->

# TiLamb-7B（Tibetan Large Language Model Base）

**TiLamb-7B** 是一款专注于藏文的大型语言模型基座模型，它使用了 26.43GB 的藏文语料库进行开发，并基于 LLaMA2-7B 模型，通过 LoRA 方法进行了增量预训练。该模型在 LLaMA2 的基础上扩展了词表，从原有的词表大小 32,000 扩充藏文词汇至 61,221 ，并对 embedding 和 lm_head 进行了均值扩充初始化。更多信息请访问 [TiLamb-7B GitHub 主页](https://github.com/NLP-Learning/TiLamb)。

**重要说明**：
- TiLamb-7B 是一个未经监督微调的基座模型，**不具备对话能力**。
- 要进行藏文对话和藏文 NLP 下游任务的适配（已验证的任务包括藏文新闻分类、藏文实体关系分类、藏文机器阅读理解、藏文分词、藏文摘要、藏文问题回答和藏文问题生成），建议使用 [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory/tree/main) 框架进行微调。

**使用须知**：
- 本项目基于 Meta 发布的 LLaMA2-7B 模型开发，使用时请严格遵守 LLaMA2-7B 的开源许可协议。
- 如果涉及使用第三方代码，请务必遵从相关的开源许可协议。
- 模型生成的内容准确性可能受到计算方法、随机因素等的影响，因此，我们不对模型输出的准确性提供任何保证，也不会对使用相关资源和输出结果产生的任何损失承担责任。
- 如果将相关模型用于商业用途，开发者应遵守当地法律法规，确保模型输出内容的合规性。本项目不对任何由此衍生的产品或服务承担责任。

# TiLamb-7B (Tibetan Large Language Model Base)

**TiLamb-7B** is a large-scale language model base focused on the Tibetan language, developed using a 26.43GB Tibetan corpus, and incrementally pre-trained through the LoRA method based on the LLaMA2-7B model. This model expands the vocabulary from the original size of 32,000 to 61,221 Tibetan entries, and initializes the embedding and lm_head with mean expansion. For more information, please visit the [TiLamb-7B GitHub page](https://github.com/NLP-Learning/TiLamb).

**Important Notes**:
- TiLamb-7B is an unsupervised fine-tuned base model, **lacking conversational capabilities**.
- For adaptation to Tibetan dialogue and Tibetan NLP downstream tasks (verified tasks include Tibetan news classification, Tibetan entity relation classification, Tibetan machine reading comprehension, Tibetan word segmentation, Tibetan summarization, Tibetan question answering, and Tibetan question generation), it is recommended to use the [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory/tree/main) framework for fine-tuning.

**Usage Notice**:
- This project is developed based on the LLaMA2-7B model released by Meta, and its use must strictly adhere to the open-source license agreement of LLaMA2-7B.
- If third-party code is involved, it is essential to comply with the relevant open-source license agreements.
- The accuracy of the content generated by the model may be affected by computational methods, random factors, etc., therefore, we do not provide any guarantee for the accuracy of the model outputs, nor will we bear any responsibility for losses arising from the use of related resources and results.
- If the related models are used for commercial purposes, developers must comply with local laws and regulations to ensure the compliance of the model output content. This project will not bear any responsibility for any products or services derived from it.