|
--- |
|
license: other |
|
language: |
|
- en |
|
pipeline_tag: text-generation |
|
inference: false |
|
tags: |
|
- transformers |
|
- gguf |
|
- imatrix |
|
- Asclepius-Llama2-13B |
|
--- |
|
Quantizations of https://huggingface.co/starmpcc/Asclepius-Llama2-13B |
|
|
|
|
|
### Inference Clients/UIs |
|
* [llama.cpp](https://github.com/ggerganov/llama.cpp) |
|
* [JanAI](https://github.com/janhq/jan) |
|
* [KoboldCPP](https://github.com/LostRuins/koboldcpp) |
|
* [text-generation-webui](https://github.com/oobabooga/text-generation-webui) |
|
* [ollama](https://github.com/ollama/ollama) |
|
* [GPT4All](https://github.com/nomic-ai/gpt4all) |
|
|
|
--- |
|
|
|
# From original readme |
|
|
|
## How to Get Started with the Model |
|
|
|
```python |
|
prompt = """You are an intelligent clinical languge model. |
|
Below is a snippet of patient's discharge summary and a following instruction from healthcare professional. |
|
Write a response that appropriately completes the instruction. |
|
The response should provide the accurate answer to the instruction, while being concise. |
|
|
|
[Discharge Summary Begin] |
|
{note} |
|
[Discharge Summary End] |
|
|
|
[Instruction Begin] |
|
{question} |
|
[Instruction End] |
|
""" |
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
tokenizer = AutoTokenizer.from_pretrained("starmpcc/Asclepius-Llama2-13B", use_fast=False) |
|
model = AutoModelForCausalLM.from_pretrained("starmpcc/Asclepius-Llama13-7B") |
|
|
|
note = "This is a sample note" |
|
question = "What is the diagnosis?" |
|
|
|
model_input = prompt.format(note=note, question=question) |
|
input_ids = tokenizer(model_input, return_tensors="pt").input_ids |
|
output = model.generate(input_ids) |
|
print(tokenizer.decode(output[0])) |
|
``` |