yinsong1986
commited on
Commit
•
5cc44e3
1
Parent(s):
90fed38
Update README.md
Browse files
README.md
CHANGED
@@ -5,9 +5,9 @@ inference: false
|
|
5 |
|
6 |
# MistralLite Model
|
7 |
|
8 |
-
MistralLite is a fine-tuned [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) language model, with enhanced capblities of processing long context (up to 32K tokens). By utilizing an adapted Rotary Embedding and sliding window during fine-tuning,
|
9 |
|
10 |
-
|
11 |
|Model|Fine-tuned on long contexts| Max context length| RotaryEmbedding adaptation| Sliding Window Size|
|
12 |
|----------|-------------:|------------:|-----------:|-----------:|
|
13 |
| Mistral-7B-v0.1 | up to 8K tokens | 32K | rope_theta = 10000 | 4096 |
|
|
|
5 |
|
6 |
# MistralLite Model
|
7 |
|
8 |
+
MistralLite is a fine-tuned [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) language model, with enhanced capblities of processing long context (up to 32K tokens). By utilizing an adapted Rotary Embedding and sliding window during fine-tuning, MistralLite is able to **perform signficantly better on several long context retrieve and answering tasks**, while keeping the simple model structure of the original model. MistralLite is useful for applications such as long context line and topic retrieval, summarization, question-answering, and etc. MistralLite can be deployed on a single AWS `g5.2x` instance with Sagemaker [Huggingface Text Generation Inference (TGI)](https://github.com/huggingface/text-generation-inference) endpoint, making it suitable for applications that require high performance in resource-constrained environments. You can also serve the MistralLite model directly using TGI docker containers. Also, MistralLite supports other ways of serving like [vLLM](https://github.com/vllm-project/vllm), and you can use MistralLite in Python by using the [HuggingFace transformers](https://huggingface.co/docs/transformers/index) and [FlashAttention-2](https://github.com/Dao-AILab/flash-attention) library.
|
9 |
|
10 |
+
MistralLite is similar to [Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1), and their similarities and differences are summarized below:
|
11 |
|Model|Fine-tuned on long contexts| Max context length| RotaryEmbedding adaptation| Sliding Window Size|
|
12 |
|----------|-------------:|------------:|-----------:|-----------:|
|
13 |
| Mistral-7B-v0.1 | up to 8K tokens | 32K | rope_theta = 10000 | 4096 |
|