Kiet Bui
commited on
Commit
•
1404ce3
1
Parent(s):
f8e438b
Update README.md
Browse files
README.md
CHANGED
@@ -14,6 +14,39 @@ language:
|
|
14 |
- All prompts are formatted to conversations pairs of [USER] and [Yaya]. For example:
|
15 |
|
16 |
[User]: Compose a long Gulf War story set in Istanbul, where a content chair encounters Elizabeth Barrett Browning, inspired by the works of Henrik Ibsen.
|
|
|
17 |
[Yaya]: As an old and content chair sat in the corner of a bustling coffee house in Istanbul, he witnessed the city's frenzied energy. It was the early 1990s, and the Gulf War had just begun.
|
18 |
|
19 |
-
There was great political strife in the air, and tensions were high. People rushed about, frantically discussing the war and its potential impact on their lives. [...]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
14 |
- All prompts are formatted to conversations pairs of [USER] and [Yaya]. For example:
|
15 |
|
16 |
[User]: Compose a long Gulf War story set in Istanbul, where a content chair encounters Elizabeth Barrett Browning, inspired by the works of Henrik Ibsen.
|
17 |
+
|
18 |
[Yaya]: As an old and content chair sat in the corner of a bustling coffee house in Istanbul, he witnessed the city's frenzied energy. It was the early 1990s, and the Gulf War had just begun.
|
19 |
|
20 |
+
There was great political strife in the air, and tensions were high. People rushed about, frantically discussing the war and its potential impact on their lives. [...]
|
21 |
+
|
22 |
+
- Load LoRA weights with PEFT model
|
23 |
+
```
|
24 |
+
from transformers import GPTJForCausalLM,AutoTokenizer, GenerationConfig
|
25 |
+
from peft import PeftModel
|
26 |
+
|
27 |
+
lora_weights = 'kietbs/pygmalion_6B_yaya' # Please download the weight, and change this path accordingly
|
28 |
+
load_in_8bit = True
|
29 |
+
model = GPTJForCausalLM.from_pretrained(pretrain_name, load_in_8bit=load_in_8bit, device_map='auto', torch_dtype=torch.float16)
|
30 |
+
model = PeftModel.from_pretrained(model,lora_weights,torch_dtype=torch.float16,device_map={'':0})
|
31 |
+
model = torch.compile(model)
|
32 |
+
|
33 |
+
GenerationConfig(
|
34 |
+
temperature=0.1,
|
35 |
+
top_p=0.75,
|
36 |
+
top_k=40,
|
37 |
+
num_beams=4
|
38 |
+
)
|
39 |
+
|
40 |
+
text = '[User]: What's the best food in Hanoi?''
|
41 |
+
input_ids = st.session_state.tokenizer(text, return_tensors='pt')['input_ids'].to('cuda')
|
42 |
+
with torch.no_grad():
|
43 |
+
output = st.session_state['model'].generate(input_ids=input_ids, generation_config=st.session_state.gen_config,return_dict_in_generate=True, output_scores=True,max_new_tokens=256)
|
44 |
+
s = output.sequences[0]
|
45 |
+
output = st.session_state.tokenizer.decode(s)
|
46 |
+
print('Raw:',output)
|
47 |
+
|
48 |
+
```
|
49 |
+
|
50 |
+
Output:
|
51 |
+
[User]: What's the best food in Hanoi?
|
52 |
+
[Yaya]: The best food in Hanoi can vary depending on what you're looking for. Some of the most popular dishes include pho, banh mi, banh xeo, and bún chả.<|endoftext|>
|