Text Generation
Transformers
Safetensors
English
llama
mergekit
Merge
medical
conversational
Eval Results
Inference Endpoints
text-generation-inference
sethuiyer commited on
Commit
e6ccaa3
1 Parent(s): 5094eea

Code sample added

Browse files
Files changed (1) hide show
  1. README.md +41 -16
README.md CHANGED
@@ -7,24 +7,12 @@ library_name: transformers
7
  tags:
8
  - mergekit
9
  - merge
10
-
 
 
11
  ---
12
- # merge
13
-
14
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
-
16
- ## Merge Details
17
- ### Merge Method
18
-
19
- This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [Locutusque/llama-3-neural-chat-v1-8b](https://huggingface.co/Locutusque/llama-3-neural-chat-v1-8b) as a base.
20
 
21
- ### Models Merged
22
-
23
- The following models were included in the merge:
24
- * [Undi95/Llama-3-Unholy-8B](https://huggingface.co/Undi95/Llama-3-Unholy-8B)
25
- * [ruslanmv/Medical-Llama3-8B-16bit](https://huggingface.co/ruslanmv/Medical-Llama3-8B-16bit)
26
-
27
- ### Configuration
28
 
29
  The following YAML configuration was used to produce this model:
30
 
@@ -47,3 +35,40 @@ parameters:
47
  dtype: bfloat16
48
 
49
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  tags:
8
  - mergekit
9
  - merge
10
+ license: llama2
11
+ language:
12
+ - en
13
  ---
 
 
 
 
 
 
 
 
14
 
15
+ ### Medichat-Llama3-8B
 
 
 
 
 
 
16
 
17
  The following YAML configuration was used to produce this model:
18
 
 
35
  dtype: bfloat16
36
 
37
  ```
38
+
39
+ ### Usage:
40
+ ```python
41
+ from transformers import AutoTokenizer, AutoModelForCausalLM
42
+
43
+ # Load tokenizer and model
44
+ tokenizer = AutoTokenizer.from_pretrained("sethuiyer/Medichat-Llama3-8B")
45
+ model = AutoModelForCausalLM.from_pretrained("sethuiyer/Medichat-Llama3-8B").to("cuda")
46
+
47
+ # Function to format and generate response with prompt engineering using a chat template
48
+ def askme(question):
49
+ sys_message = '''
50
+ You are an AI Medical Assistant trained on a vast dataset of health information. Please be thorough and
51
+ provide an informative answer. If you don't know the answer to a specific medical inquiry, advise seeking professional help.
52
+ '''
53
+
54
+ # Create messages structured for the chat template
55
+ messages = [{"role": "system", "content": sys_message}, {"role": "user", "content": question}]
56
+
57
+ # Applying chat template
58
+ prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
59
+ inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
60
+ outputs = model.generate(**inputs, max_new_tokens=512, use_cache=True) # Adjust max_new_tokens for longer responses
61
+
62
+ # Extract and return the generated text
63
+ answer = tokenizer.batch_decode(outputs)[0].strip()
64
+ return answer
65
+
66
+ # Example usage
67
+ question = '''
68
+ Symptoms:
69
+ Dizziness, headache and nausea.
70
+
71
+ What is the differnetial diagnosis?
72
+ '''
73
+ print(askme(question))
74
+ ```