PEFT
English
medical
Tonic commited on
Commit
4fee1a9
1 Parent(s): 32cca07

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -13
README.md CHANGED
@@ -24,13 +24,13 @@ Trying to get better at medical Q & A
24
 
25
 
26
  - **Developed by:** [Tonic](https://huggingface.co/Tonic)
27
- - **Shared by [optional]:** [Tonic](https://huggingface.co/Tonic)
28
  - **Model type:** Mistral Fine-Tune
29
  - **Language(s) (NLP):** English
30
  - **License:** MIT2.0
31
- - **Finetuned from model [optional]:** [mistralai/Mistral-7B-v0.1](https://huggingface.com/Mistralai/Mistral-7B-v0.1)
32
 
33
- ### Model Sources [optional]
34
 
35
 
36
  - **Repository:** [Tonic/mistralmed](https://huggingface.co/Tonic/mistralmed)
@@ -45,7 +45,7 @@ This model can be used the same way you normally use mistral
45
 
46
  This model can do better in medical question and answer scenarios.
47
 
48
- ### Downstream Use [optional]
49
 
50
  This model is intended to be further fine tuned.
51
 
@@ -63,18 +63,16 @@ Users (both direct and downstream) should be made aware of the risks, biases and
63
 
64
  Use the code below to get started with the model.
65
 
66
- [Tonic/MistralMED_Chat](https://huggingface.co/Tonic/MistralMED_Chat)
67
 
68
  ```python
69
- from transformers import AutoTokenizer, MistralForCausalLM
70
- import torch
71
- import gradio as gr
72
- import random
73
- from textwrap import wrap
74
  from transformers import AutoConfig, AutoTokenizer, AutoModelForSeq2SeqLM, AutoModelForCausalLM, MistralForCausalLM
75
  from peft import PeftModel, PeftConfig
76
  import torch
77
  import gradio as gr
 
 
 
78
 
79
  # Functions to Wrap the Prompt Correctly
80
  def wrap_text(text, width=90):
@@ -195,14 +193,16 @@ iface.launch()
195
 
196
  ### Training Procedure
197
 
 
198
  Dataset({
199
  features: ['qtype', 'Question', 'Answer'],
200
  num_rows: 16407
201
  })
202
-
203
 
204
  #### Preprocessing [optional]
205
 
 
206
  MistralForCausalLM(
207
  (model): MistralModel(
208
  (embed_tokens): Embedding(32000, 4096)
@@ -229,11 +229,12 @@ MistralForCausalLM(
229
  )
230
  (lm_head): Linear(in_features=4096, out_features=32000, bias=False)
231
  )
232
-
233
 
234
  #### Training Hyperparameters
235
 
236
  - **Training regime:**
 
237
  config = LoraConfig(
238
  r=8,
239
  lora_alpha=16,
@@ -251,6 +252,7 @@ config = LoraConfig(
251
  lora_dropout=0.05, # Conventional
252
  task_type="CAUSAL_LM",
253
  )
 
254
 
255
  #### Speeds, Sizes, Times [optional]
256
 
@@ -260,7 +262,6 @@ config = LoraConfig(
260
 
261
  ## Environmental Impact
262
 
263
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
264
 
265
  Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
266
 
@@ -300,6 +301,7 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
300
 
301
  ### Model Architecture and Objective
302
 
 
303
  PeftModelForCausalLM(
304
  (base_model): LoraModel(
305
  (model): MistralForCausalLM(
@@ -434,6 +436,8 @@ PeftModelForCausalLM(
434
  )
435
  )
436
  )
 
 
437
  #### Hardware
438
 
439
  A100
 
24
 
25
 
26
  - **Developed by:** [Tonic](https://huggingface.co/Tonic)
27
+ - **Shared by :** [Tonic](https://huggingface.co/Tonic)
28
  - **Model type:** Mistral Fine-Tune
29
  - **Language(s) (NLP):** English
30
  - **License:** MIT2.0
31
+ - **Finetuned from model :** [mistralai/Mistral-7B-v0.1](https://huggingface.com/Mistralai/Mistral-7B-v0.1)
32
 
33
+ ### Model Sources
34
 
35
 
36
  - **Repository:** [Tonic/mistralmed](https://huggingface.co/Tonic/mistralmed)
 
45
 
46
  This model can do better in medical question and answer scenarios.
47
 
48
+ ### Downstream Use
49
 
50
  This model is intended to be further fine tuned.
51
 
 
63
 
64
  Use the code below to get started with the model.
65
 
66
+ [pseudolab/MistralMED_Chat](https://huggingface.co/spaces/pseudolab/MistralMED_Chat)
67
 
68
  ```python
 
 
 
 
 
69
  from transformers import AutoConfig, AutoTokenizer, AutoModelForSeq2SeqLM, AutoModelForCausalLM, MistralForCausalLM
70
  from peft import PeftModel, PeftConfig
71
  import torch
72
  import gradio as gr
73
+ import random
74
+ from textwrap import wrap
75
+
76
 
77
  # Functions to Wrap the Prompt Correctly
78
  def wrap_text(text, width=90):
 
193
 
194
  ### Training Procedure
195
 
196
+ ```json
197
  Dataset({
198
  features: ['qtype', 'Question', 'Answer'],
199
  num_rows: 16407
200
  })
201
+ ```
202
 
203
  #### Preprocessing [optional]
204
 
205
+ ```json
206
  MistralForCausalLM(
207
  (model): MistralModel(
208
  (embed_tokens): Embedding(32000, 4096)
 
229
  )
230
  (lm_head): Linear(in_features=4096, out_features=32000, bias=False)
231
  )
232
+ ```
233
 
234
  #### Training Hyperparameters
235
 
236
  - **Training regime:**
237
+ ```json
238
  config = LoraConfig(
239
  r=8,
240
  lora_alpha=16,
 
252
  lora_dropout=0.05, # Conventional
253
  task_type="CAUSAL_LM",
254
  )
255
+ ```
256
 
257
  #### Speeds, Sizes, Times [optional]
258
 
 
262
 
263
  ## Environmental Impact
264
 
 
265
 
266
  Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
267
 
 
301
 
302
  ### Model Architecture and Objective
303
 
304
+ ```json
305
  PeftModelForCausalLM(
306
  (base_model): LoraModel(
307
  (model): MistralForCausalLM(
 
436
  )
437
  )
438
  )
439
+ ```
440
+
441
  #### Hardware
442
 
443
  A100