Text Generation
Transformers
PyTorch
Chinese
English
llama
text-generation-inference
fireballoon commited on
Commit
e1c582c
1 Parent(s): f7bff14

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -1
README.md CHANGED
@@ -25,11 +25,30 @@ tokenizer = AutoTokenizer.from_pretrained("fireballoon/baichuan-vicuna-7b", use_
25
  model = AutoModelForCausalLM.from_pretrained("fireballoon/baichuan-vicuna-7b")
26
  ```
27
 
28
- # Inference with FastChat
 
29
  ```
30
  python3 -m fastchat.serve.cli --model-path fireballoon/baichuan-vicuna-7b
31
  ```
32
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  # Test examples on FastChat Eval
34
  Test example on [FastChat Eval](https://github.com/lm-sys/FastChat/blob/main/fastchat/eval/table/question.jsonl)
35
  ```
 
25
  model = AutoModelForCausalLM.from_pretrained("fireballoon/baichuan-vicuna-7b")
26
  ```
27
 
28
+ # Inference
29
+ Inference with FastChat:
30
  ```
31
  python3 -m fastchat.serve.cli --model-path fireballoon/baichuan-vicuna-7b
32
  ```
33
 
34
+ Inference with Transformers:
35
+ ```ipython
36
+ >>> from transformers import AutoTokenizer, AutoModelForCausalLM, TextStreamer
37
+ >>> tokenizer = AutoTokenizer.from_pretrained("fireballoon/baichuan-vicuna-7b", use_fast=False)
38
+ >>> model = AutoModelForCausalLM.from_pretrained("fireballoon/baichuan-vicuna-7b").half().cuda()
39
+ >>> instruction = "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: {} ASSISTANT:"
40
+ >>> prompt = instruction.format("five tips to help with sleep") # user message
41
+ >>> generate_ids = model.generate(tokenizer(prompt, return_tensors='pt').input_ids.cuda(), max_new_tokens=2048, streamer=streamer)
42
+ '''
43
+ 1. Create a relaxing bedtime routine, such as winding down with a warm bath or reading a book before bed.
44
+ 2. Avoid caffeine and alcohol close to bedtime, as they can disrupt sleep.
45
+ 3. Use blackout curtains or shades to block out light from the outside, which can interfere with your body's natural sleep rhythms.
46
+ 4. Keep your bedroom cool and dark, and avoid using electronic devices at least an hour before bed.
47
+ 5. Regular exercise and a healthy diet can also help improve sleep quality.
48
+ '''
49
+ ```
50
+
51
+
52
  # Test examples on FastChat Eval
53
  Test example on [FastChat Eval](https://github.com/lm-sys/FastChat/blob/main/fastchat/eval/table/question.jsonl)
54
  ```