File size: 1,107 Bytes
ebd33aa |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
---
license: apache-2.0
language:
- en
tags:
- medical
---
#### This Model is optimized version of "alibidaran/llama-2-7b-virtual_doctor" for executing on CPU and GPU. It can be easily used on CPU and personal computers.
## Uses
In order to use this model on the CPU, you need to install a few libraries.
```python
!pip install ctransformers
```
In the next step you can use this model in your project by using codes below:
```python
from ctransformers import AutoModelForCausalLM
from transformers import AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained("alibidaran/llama-2-7b-virtual_doctor-gguf",hf=True)
tokenizer = AutoTokenizer.from_pretrained("alibidaran/llama-2-7b-virtual_doctor")
prompt = " Hi doctor, I have nose running, I have fever, I often feel tired, what should i do?"
text=f"<s> ###Human: {prompt} ###Asistant: "
inputs=tokenizer(text,return_tensors='pt').to('cpu')
with torch.no_grad():
outputs=model.generate(**inputs,max_new_tokens=200,do_sample=True,top_p=0.92,top_k=10,temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
``` |