Update README.md
Browse files
README.md
CHANGED
@@ -34,9 +34,9 @@ Trying to get better at medical Q & A
|
|
34 |
### Model Sources [optional]
|
35 |
|
36 |
|
37 |
-
- **Repository:** [Tonic/mistralmed]
|
38 |
-
- **
|
39 |
-
- **Demo
|
40 |
|
41 |
## Uses
|
42 |
|
@@ -79,7 +79,8 @@ Dataset({
|
|
79 |
num_rows: 16407
|
80 |
})
|
81 |
|
82 |
-
|
|
|
83 |
|
84 |
MistralForCausalLM(
|
85 |
(model): MistralModel(
|
@@ -108,10 +109,6 @@ MistralForCausalLM(
|
|
108 |
(lm_head): Linear(in_features=4096, out_features=32000, bias=False)
|
109 |
)
|
110 |
|
111 |
-
#### Preprocessing [optional]
|
112 |
-
|
113 |
-
[More Information Needed]
|
114 |
-
|
115 |
|
116 |
#### Training Hyperparameters
|
117 |
|
@@ -136,9 +133,9 @@ config = LoraConfig(
|
|
136 |
|
137 |
#### Speeds, Sizes, Times [optional]
|
138 |
|
139 |
-
trainable params: 21260288 || all params: 3773331456 || trainable%: 0.5634354746703705
|
140 |
-
TrainOutput(global_step=1000, training_loss=0.47226515007019043, metrics={'train_runtime': 3143.4141, 'train_samples_per_second': 2.545, 'train_steps_per_second': 0.318, 'total_flos': 1.75274075357184e+17, 'train_loss': 0.47226515007019043, 'epoch': 0.49})
|
141 |
-
|
142 |
|
143 |
## Environmental Impact
|
144 |
|
@@ -146,11 +143,11 @@ TrainOutput(global_step=1000, training_loss=0.47226515007019043, metrics={'train
|
|
146 |
|
147 |
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
148 |
|
149 |
-
- **Hardware Type:**
|
150 |
-
- **Hours used:**
|
151 |
-
- **Cloud Provider:**
|
152 |
-
- **Compute Region:**
|
153 |
-
- **Carbon Emitted:**
|
154 |
|
155 |
## Training Results
|
156 |
|
|
|
34 |
### Model Sources [optional]
|
35 |
|
36 |
|
37 |
+
- **Repository:** [Tonic/mistralmed](https://huggingface.co/Tonic/mistralmed)
|
38 |
+
- **Code :** [github](https://github.com/Josephrp/mistralmed/blob/main/finetuning.py)
|
39 |
+
- **Demo :** [Tonic/MistralMed_Chat](https://huggingface.co/Tonic/MistralMed_Chat)
|
40 |
|
41 |
## Uses
|
42 |
|
|
|
79 |
num_rows: 16407
|
80 |
})
|
81 |
|
82 |
+
|
83 |
+
#### Preprocessing [optional]
|
84 |
|
85 |
MistralForCausalLM(
|
86 |
(model): MistralModel(
|
|
|
109 |
(lm_head): Linear(in_features=4096, out_features=32000, bias=False)
|
110 |
)
|
111 |
|
|
|
|
|
|
|
|
|
112 |
|
113 |
#### Training Hyperparameters
|
114 |
|
|
|
133 |
|
134 |
#### Speeds, Sizes, Times [optional]
|
135 |
|
136 |
+
- trainable params: 21260288 || all params: 3773331456 || trainable%: 0.5634354746703705
|
137 |
+
- TrainOutput(global_step=1000, training_loss=0.47226515007019043, metrics={'train_runtime': 3143.4141, 'train_samples_per_second': 2.545, 'train_steps_per_second': 0.318, 'total_flos': 1.75274075357184e+17, 'train_loss': 0.47226515007019043, 'epoch': 0.49})
|
138 |
+
|
139 |
|
140 |
## Environmental Impact
|
141 |
|
|
|
143 |
|
144 |
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
145 |
|
146 |
+
- **Hardware Type:** A100
|
147 |
+
- **Hours used:** 1
|
148 |
+
- **Cloud Provider:** Google
|
149 |
+
- **Compute Region:** East1
|
150 |
+
- **Carbon Emitted:** 0.09
|
151 |
|
152 |
## Training Results
|
153 |
|