fernandofinardi commited on
Commit
9095279
1 Parent(s): da52493

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +59 -1
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- library_name: peft
3
  base_model: codellama/CodeLlama-7b-Instruct-hf
4
  license: apache-2.0
5
  datasets:
@@ -42,6 +42,64 @@ Input : Text
42
 
43
  Output : Text (Code)
44
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
 
46
 
47
  **Params**
 
1
  ---
2
+ library_name: transformers
3
  base_model: codellama/CodeLlama-7b-Instruct-hf
4
  license: apache-2.0
5
  datasets:
 
42
 
43
  Output : Text (Code)
44
 
45
+
46
+ **Usage**
47
+
48
+ Using Transformers
49
+ ```python
50
+ #Import required libraries
51
+ import torch
52
+ from transformers import (
53
+ AutoModelForCausalLM,
54
+ AutoTokenizer
55
+ )
56
+
57
+ #Load Model
58
+ model_name = "semantixai/LloroV2"
59
+ base_model = AutoModelForCausalLM.from_pretrained(
60
+ model_name,
61
+ return_dict=True,
62
+ torch_dtype=torch.float16,
63
+ device_map="auto",
64
+ )
65
+
66
+ #Load Tokenizer
67
+ tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
68
+
69
+
70
+ #Define Prompt
71
+ user_prompt = "Desenvolva um algoritmo em Python para calcular a média e a mediana dos preços de vendas por tipo de material do produto."
72
+ system = "Provide answers in Python without explanations, only the code"
73
+ prompt_template = f"[INST] <<SYS>>\\n{system}\\n<</SYS>>\\n\\n{user_prompt}[/INST]"
74
+
75
+ #Call the model
76
+ input_ids = tokenizer([prompt_template], return_tensors="pt")["input_ids"].to("cuda")
77
+
78
+
79
+ outputs = base_model.generate(
80
+ input_ids,
81
+ do_sample=True,
82
+ top_p=0.95,
83
+ max_new_tokens=1024,
84
+ temperature=0.1,
85
+ )
86
+
87
+ #Decode and retrieve Output
88
+ output_text = tokenizer.batch_decode(outputs, skip_prompt=True, skip_special_tokens=False)
89
+ display(output_text)
90
+ ```
91
+
92
+ Using an OpenAI compatible inference server (like [vLLM](https://docs.vllm.ai/en/latest/index.html))
93
+ ```python
94
+ from openai import OpenAI
95
+
96
+ client = OpenAI(
97
+ api_key="EMPTY",
98
+ base_url="http://localhost:8000/v1",
99
+ )
100
+ user_prompt = "Desenvolva um algoritmo em Python para calcular a média e a mediana dos preços de vendas por tipo de material do produto."
101
+ completion = client.chat.completions.create(temperature=0.1,frequency_penalty=0.1,model="semantixai/LloroV2",messages=[{"role":"system","content":"Provide answers in Python without explanations, only the code"},{"role":"user","content":user_prompt}])
102
+ ```
103
 
104
 
105
  **Params**