Update README.md

README.md

@@ -16,6 +16,56 @@ thumbnail: https://h2o.ai/etc.clientlibs/h2o/clientlibs/clientlib-site/resources
 This model was trained using [H2O LLM Studio](https://github.com/h2oai/h2o-llmstudio).
 - Base model: [EleutherAI/pythia-70m-deduped-v0](https://huggingface.co/EleutherAI/pythia-70m-deduped-v0)
 
+## Usage on CPU
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+def generate_response(prompt, model_name):
+    tokenizer = AutoTokenizer.from_pretrained(
+        model_name,
+        use_fast=True,
+        trust_remote_code=True,
+    )
+
+    model = AutoModelForCausalLM.from_pretrained(
+        model_name,
+        torch_dtype=torch.float32,
+        device_map={"": "cpu"},
+        trust_remote_code=True,
+    )
+    model.cpu().eval()
+
+    inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to("cpu")
+
+    tokens = model.generate(
+        input_ids=inputs["input_ids"],
+        attention_mask=inputs["attention_mask"],
+        min_new_tokens=2,
+        max_new_tokens=500,
+        do_sample=False,
+        num_beams=2,
+        temperature=float(0.0),
+        repetition_penalty=float(1.0),
+        renormalize_logits=True
+    )[0]
+
+    tokens = tokens[inputs["input_ids"].shape[1]:]
+    answer = tokenizer.decode(tokens, skip_special_tokens=True)
+
+    return answer
+```
+
+After defining the function, you can define the prompt and the model from another file
+
+```python
+model_name = "diegomiranda/text-to-cypher"
+prompt = "Create a Cypher statement to answer the following question:Retorne os processos de Direito Tributário que se baseiam em lei 939 de 1992?<|endoftext|>"
+response = generate_response(prompt, model_name)
+print(response)
+```
+
 
 ## Usage