Feature(LLMLingua): add exaplain
Browse files
app.py
CHANGED
|
@@ -4,7 +4,9 @@ from llmlingua import PromptCompressor
|
|
| 4 |
llm_lingua = PromptCompressor("lgaalves/gpt2-dolly", device_map="cpu")
|
| 5 |
|
| 6 |
INTRO = """
|
| 7 |
-
# LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
|
|
|
|
|
|
|
| 8 |
This is an early demo of the prompt compression method LLMLingua.
|
| 9 |
|
| 10 |
It should be noted that due to limited resources, we only provide the **GPT2-Small** size language model in this demo. Using the **LLaMA2-7B** as a small language model would result in a significant performance improvement, especially at high compression ratios.
|
|
|
|
| 4 |
llm_lingua = PromptCompressor("lgaalves/gpt2-dolly", device_map="cpu")
|
| 5 |
|
| 6 |
INTRO = """
|
| 7 |
+
# LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models (EMNLP 2023) [paper]()
|
| 8 |
+
_Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang and Lili Qiu_
|
| 9 |
+
|
| 10 |
This is an early demo of the prompt compression method LLMLingua.
|
| 11 |
|
| 12 |
It should be noted that due to limited resources, we only provide the **GPT2-Small** size language model in this demo. Using the **LLaMA2-7B** as a small language model would result in a significant performance improvement, especially at high compression ratios.
|