Update README.md
Browse files
README.md
CHANGED
@@ -24,13 +24,13 @@ Trying to get better at medical Q & A
|
|
24 |
|
25 |
|
26 |
- **Developed by:** [Tonic](https://huggingface.co/Tonic)
|
27 |
-
- **Shared by
|
28 |
- **Model type:** Mistral Fine-Tune
|
29 |
- **Language(s) (NLP):** English
|
30 |
- **License:** MIT2.0
|
31 |
-
- **Finetuned from model
|
32 |
|
33 |
-
### Model Sources
|
34 |
|
35 |
|
36 |
- **Repository:** [Tonic/mistralmed](https://huggingface.co/Tonic/mistralmed)
|
@@ -45,7 +45,7 @@ This model can be used the same way you normally use mistral
|
|
45 |
|
46 |
This model can do better in medical question and answer scenarios.
|
47 |
|
48 |
-
### Downstream Use
|
49 |
|
50 |
This model is intended to be further fine tuned.
|
51 |
|
@@ -63,18 +63,16 @@ Users (both direct and downstream) should be made aware of the risks, biases and
|
|
63 |
|
64 |
Use the code below to get started with the model.
|
65 |
|
66 |
-
[
|
67 |
|
68 |
```python
|
69 |
-
from transformers import AutoTokenizer, MistralForCausalLM
|
70 |
-
import torch
|
71 |
-
import gradio as gr
|
72 |
-
import random
|
73 |
-
from textwrap import wrap
|
74 |
from transformers import AutoConfig, AutoTokenizer, AutoModelForSeq2SeqLM, AutoModelForCausalLM, MistralForCausalLM
|
75 |
from peft import PeftModel, PeftConfig
|
76 |
import torch
|
77 |
import gradio as gr
|
|
|
|
|
|
|
78 |
|
79 |
# Functions to Wrap the Prompt Correctly
|
80 |
def wrap_text(text, width=90):
|
@@ -195,14 +193,16 @@ iface.launch()
|
|
195 |
|
196 |
### Training Procedure
|
197 |
|
|
|
198 |
Dataset({
|
199 |
features: ['qtype', 'Question', 'Answer'],
|
200 |
num_rows: 16407
|
201 |
})
|
202 |
-
|
203 |
|
204 |
#### Preprocessing [optional]
|
205 |
|
|
|
206 |
MistralForCausalLM(
|
207 |
(model): MistralModel(
|
208 |
(embed_tokens): Embedding(32000, 4096)
|
@@ -229,11 +229,12 @@ MistralForCausalLM(
|
|
229 |
)
|
230 |
(lm_head): Linear(in_features=4096, out_features=32000, bias=False)
|
231 |
)
|
232 |
-
|
233 |
|
234 |
#### Training Hyperparameters
|
235 |
|
236 |
- **Training regime:**
|
|
|
237 |
config = LoraConfig(
|
238 |
r=8,
|
239 |
lora_alpha=16,
|
@@ -251,6 +252,7 @@ config = LoraConfig(
|
|
251 |
lora_dropout=0.05, # Conventional
|
252 |
task_type="CAUSAL_LM",
|
253 |
)
|
|
|
254 |
|
255 |
#### Speeds, Sizes, Times [optional]
|
256 |
|
@@ -260,7 +262,6 @@ config = LoraConfig(
|
|
260 |
|
261 |
## Environmental Impact
|
262 |
|
263 |
-
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
264 |
|
265 |
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
266 |
|
@@ -300,6 +301,7 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
|
|
300 |
|
301 |
### Model Architecture and Objective
|
302 |
|
|
|
303 |
PeftModelForCausalLM(
|
304 |
(base_model): LoraModel(
|
305 |
(model): MistralForCausalLM(
|
@@ -434,6 +436,8 @@ PeftModelForCausalLM(
|
|
434 |
)
|
435 |
)
|
436 |
)
|
|
|
|
|
437 |
#### Hardware
|
438 |
|
439 |
A100
|
|
|
24 |
|
25 |
|
26 |
- **Developed by:** [Tonic](https://huggingface.co/Tonic)
|
27 |
+
- **Shared by :** [Tonic](https://huggingface.co/Tonic)
|
28 |
- **Model type:** Mistral Fine-Tune
|
29 |
- **Language(s) (NLP):** English
|
30 |
- **License:** MIT2.0
|
31 |
+
- **Finetuned from model :** [mistralai/Mistral-7B-v0.1](https://huggingface.com/Mistralai/Mistral-7B-v0.1)
|
32 |
|
33 |
+
### Model Sources
|
34 |
|
35 |
|
36 |
- **Repository:** [Tonic/mistralmed](https://huggingface.co/Tonic/mistralmed)
|
|
|
45 |
|
46 |
This model can do better in medical question and answer scenarios.
|
47 |
|
48 |
+
### Downstream Use
|
49 |
|
50 |
This model is intended to be further fine tuned.
|
51 |
|
|
|
63 |
|
64 |
Use the code below to get started with the model.
|
65 |
|
66 |
+
[pseudolab/MistralMED_Chat](https://huggingface.co/spaces/pseudolab/MistralMED_Chat)
|
67 |
|
68 |
```python
|
|
|
|
|
|
|
|
|
|
|
69 |
from transformers import AutoConfig, AutoTokenizer, AutoModelForSeq2SeqLM, AutoModelForCausalLM, MistralForCausalLM
|
70 |
from peft import PeftModel, PeftConfig
|
71 |
import torch
|
72 |
import gradio as gr
|
73 |
+
import random
|
74 |
+
from textwrap import wrap
|
75 |
+
|
76 |
|
77 |
# Functions to Wrap the Prompt Correctly
|
78 |
def wrap_text(text, width=90):
|
|
|
193 |
|
194 |
### Training Procedure
|
195 |
|
196 |
+
```json
|
197 |
Dataset({
|
198 |
features: ['qtype', 'Question', 'Answer'],
|
199 |
num_rows: 16407
|
200 |
})
|
201 |
+
```
|
202 |
|
203 |
#### Preprocessing [optional]
|
204 |
|
205 |
+
```json
|
206 |
MistralForCausalLM(
|
207 |
(model): MistralModel(
|
208 |
(embed_tokens): Embedding(32000, 4096)
|
|
|
229 |
)
|
230 |
(lm_head): Linear(in_features=4096, out_features=32000, bias=False)
|
231 |
)
|
232 |
+
```
|
233 |
|
234 |
#### Training Hyperparameters
|
235 |
|
236 |
- **Training regime:**
|
237 |
+
```json
|
238 |
config = LoraConfig(
|
239 |
r=8,
|
240 |
lora_alpha=16,
|
|
|
252 |
lora_dropout=0.05, # Conventional
|
253 |
task_type="CAUSAL_LM",
|
254 |
)
|
255 |
+
```
|
256 |
|
257 |
#### Speeds, Sizes, Times [optional]
|
258 |
|
|
|
262 |
|
263 |
## Environmental Impact
|
264 |
|
|
|
265 |
|
266 |
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
267 |
|
|
|
301 |
|
302 |
### Model Architecture and Objective
|
303 |
|
304 |
+
```json
|
305 |
PeftModelForCausalLM(
|
306 |
(base_model): LoraModel(
|
307 |
(model): MistralForCausalLM(
|
|
|
436 |
)
|
437 |
)
|
438 |
)
|
439 |
+
```
|
440 |
+
|
441 |
#### Hardware
|
442 |
|
443 |
A100
|