Commit
•
0c97327
1
Parent(s):
1dcc0f0
Update README.md
Browse files
README.md
CHANGED
|
@@ -7,8 +7,15 @@ license: apache-2.0
|
|
| 7 |
---
|
| 8 |
|
| 9 |
**Paper**: [Adapting Language Models to Compress Contexts](https://arxiv.org/abs/2305.14788)
|
|
|
|
| 10 |
**Code**: https://github.com/princeton-nlp/AutoCompressors
|
| 11 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
---
|
| 13 |
|
| 14 |
FullAttention-Llama-2-7b-6k is a model fine-tuned from [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) and used as baseline in [Adapting Language Models to Compress Contexts](https://arxiv.org/abs/2305.14788).
|
|
|
|
| 7 |
---
|
| 8 |
|
| 9 |
**Paper**: [Adapting Language Models to Compress Contexts](https://arxiv.org/abs/2305.14788)
|
| 10 |
+
|
| 11 |
**Code**: https://github.com/princeton-nlp/AutoCompressors
|
| 12 |
|
| 13 |
+
**Models**:
|
| 14 |
+
- Llama-2-7b fine-tuned models: [AutoCompressor-Llama-2-7b-6k](https://huggingface.co/princeton-nlp/AutoCompressor-Llama-2-7b-6k/), [FullAttention-Llama-2-7b-6k](https://huggingface.co/princeton-nlp/FullAttention-Llama-2-7b-6k)
|
| 15 |
+
- OPT-2.7b fine-tuned models: [AutoCompressor-2.7b-6k](https://huggingface.co/princeton-nlp/AutoCompressor-2.7b-6k), [AutoCompressor-2.7b-30k](https://huggingface.co/princeton-nlp/AutoCompressor-2.7b-30k), [RMT-2.7b-8k](https://huggingface.co/princeton-nlp/RMT-2.7b-8k)
|
| 16 |
+
- OPT-1.3b fine-tuned models: [AutoCompressor-1.3b-30k](https://huggingface.co/princeton-nlp/AutoCompressor-1.3b-30k), [RMT-1.3b-30k](https://huggingface.co/princeton-nlp/RMT-1.3b-30k)
|
| 17 |
+
|
| 18 |
+
|
| 19 |
---
|
| 20 |
|
| 21 |
FullAttention-Llama-2-7b-6k is a model fine-tuned from [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) and used as baseline in [Adapting Language Models to Compress Contexts](https://arxiv.org/abs/2305.14788).
|