iofu728 commited on
Commit
85cbdde
β€’
1 Parent(s): a7d425a

Feature(LLMLingua): add note

Browse files
Files changed (1) hide show
  1. app.py +3 -0
app.py CHANGED
@@ -6,6 +6,9 @@ llm_lingua = PromptCompressor("lgaalves/gpt2-dolly", device_map="cpu")
6
  INTRO = """
7
  # LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
8
  This is an early demo of the prompt compression method LLMLingua.
 
 
 
9
  To use it, upload your prompt and set the compression target.
10
  1. βœ… Set the different components of the prompt separately, including instruction, context, and question. Leave the corresponding field empty if a particular component does not exist.
11
  - Question: This refers to the directives given by the user to the LLMs, such as inquiries, questions, or requests. Positioned after the instruction and context modules, the question module has a high sensitivity to compression.
 
6
  INTRO = """
7
  # LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
8
  This is an early demo of the prompt compression method LLMLingua.
9
+
10
+ It should be noted that due to limited resources, we only provide the **GPT2-Small** size language model in this demo. Using the **LLaMA2-7B** as a small language model would result in a significant performance improvement, especially at high compression ratios.
11
+
12
  To use it, upload your prompt and set the compression target.
13
  1. βœ… Set the different components of the prompt separately, including instruction, context, and question. Leave the corresponding field empty if a particular component does not exist.
14
  - Question: This refers to the directives given by the user to the LLMs, such as inquiries, questions, or requests. Positioned after the instruction and context modules, the question module has a high sensitivity to compression.