DeepMount00 commited on
Commit
9798446
1 Parent(s): 48feee2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md CHANGED
@@ -27,6 +27,33 @@ The Mistral-7B-v0.1 model is a transformer-based model that can handle a variety
27
  ## How to Use
28
  How to utilize my Mistral for Italian text generation
29
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  ---
31
  ## Developer
32
  [Michele Montebovi]
 
27
  ## How to Use
28
  How to utilize my Mistral for Italian text generation
29
 
30
+ ```python
31
+ import transformers
32
+ from transformers import TextStreamer
33
+ import torch
34
+
35
+ model_name = "DeepMount00/Mistral-Ita-7b"
36
+
37
+ tokenizer = transformers.AutoTokenizer.from_pretrained(model_name)
38
+ model = transformers.LlamaForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto").eval()
39
+
40
+ def stream(user_prompt):
41
+ runtimeFlag = "cuda:0"
42
+ system_prompt = '\n'
43
+ B_INST, E_INST = "[INST]", "[/INST]"
44
+ prompt = f"{system_prompt}{B_INST}{user_prompt.strip()}\n{E_INST}"
45
+ inputs = tokenizer([prompt], return_tensors="pt").to(runtimeFlag)
46
+ streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
47
+ _ = model.generate(**inputs, streamer=streamer, max_new_tokens=100, num_return_sequences=1)
48
+
49
+ domanda = """individua il giorno presente nel contesto\n"""
50
+ contesto = """
51
+ La mononucleosi è spesso chiamata la malattia del bacio perché si trasmette principalmente attraverso la saliva ale ore 17 del giorno 21 ottobre.
52
+ """
53
+ prompt = domanda + contesto
54
+
55
+ stream(prompt)
56
+ ```
57
  ---
58
  ## Developer
59
  [Michele Montebovi]