QIOvO/maids

介绍 (Introduction)

🤗Evaluation and Enhancement of Large Language Models for In-Patient Diagnostic Support🤗

Requirements

• python 3.9 and above
• pytorch 1.12 and above, 2.0 and above are recommended
• CUDA 11.8 and above are recommended
  

Dependency

To run maids, please make sure you meet the above requirements, and then execute the following pip commands to install the dependent libraries.

pip install transformers==4.32.0 accelerate tiktoken einops scipy transformers_stream_generator==0.0.4 peft deepspeed

Quickstart

You can easily call the model with the following code:

import transformers
import torch

model_id = "QIOvO/maids"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map=0,
)

def test(query):
    messages = [
    {"role": "system", "content": '''You are a clinic expert.'''},
    {"role": "user", "content": query},
]

    terminators = [
        pipeline.tokenizer.eos_token_id,
        pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
    ]

    outputs = pipeline(
        messages,
        max_new_tokens=4096,
        eos_token_id=terminators,
        do_sample=True,
        temperature=0.6,
        top_p=0.9,
    )
    return outputs[0]["generated_text"][-1]["content"]