HuggingSara
commited on
Commit
•
e6c4c22
1
Parent(s):
bc9eb0f
Update README.md
Browse files
README.md
CHANGED
@@ -42,9 +42,9 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
|
42 |
```
|
43 |
|
44 |
### Training Procedure
|
45 |
-
- **Dataset:**
|
46 |
-
- **
|
47 |
-
- **
|
48 |
|
49 |
### Model Performance
|
50 |
- **Benchmarks:** Outperforms the baseline model and Jais-30B in medical evaluations.
|
@@ -62,10 +62,9 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
|
62 |
### Safety and Ethical Considerations
|
63 |
- **Usage:** Research purposes only.
|
64 |
|
65 |
-
|
66 |
### Accessibility
|
67 |
- **Availability:** [BiMediX GitHub Repository](https://github.com/mbzuai-oryx/BiMediX).
|
68 |
|
69 |
### Authors
|
70 |
-
Sara Pieri, Sahal Shaji Mullappilly, Fahad Shahbaz Khan, Rao Muhammad Anwer Salman Khan, Timothy Baldwin, Hisham Cholakkal
|
71 |
-
|
|
|
42 |
```
|
43 |
|
44 |
### Training Procedure
|
45 |
+
- **Dataset:** BiMed1.3M, 632 million healthcare specialized tokens.
|
46 |
+
- **QLoRA Adaptation:** Implements a low-rank adaptation technique, incorporating learnable low-rank adapter weights into the experts and the routing network. This results in training about 4% of the original parameters.
|
47 |
+
- **Training Resources:** The model underwent training on approximately 632 million tokens from the Arabic-English corpus, including 288 million tokens exclusively for English.
|
48 |
|
49 |
### Model Performance
|
50 |
- **Benchmarks:** Outperforms the baseline model and Jais-30B in medical evaluations.
|
|
|
62 |
### Safety and Ethical Considerations
|
63 |
- **Usage:** Research purposes only.
|
64 |
|
|
|
65 |
### Accessibility
|
66 |
- **Availability:** [BiMediX GitHub Repository](https://github.com/mbzuai-oryx/BiMediX).
|
67 |
|
68 |
### Authors
|
69 |
+
Sara Pieri, Sahal Shaji Mullappilly, Fahad Shahbaz Khan, Rao Muhammad Anwer Salman Khan, Timothy Baldwin, Hisham Cholakkal
|
70 |
+
**Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI)**
|