Avnish Kumar
commited on
Commit
•
8a0cf16
1
Parent(s):
8c8b3a3
Update README.md
Browse files
README.md
CHANGED
|
@@ -4,6 +4,14 @@ library_name: peft
|
|
| 4 |
|
| 5 |
Open source falcon 7b large language model fine tuned on SQuAD dataset for question and answering.
|
| 6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
## Training procedure
|
| 8 |
|
| 9 |
|
|
|
|
| 4 |
|
| 5 |
Open source falcon 7b large language model fine tuned on SQuAD dataset for question and answering.
|
| 6 |
|
| 7 |
+
QLoRA technique used for fine tuning the model on consumer grade GPU
|
| 8 |
+
SFTTrainer is also used.
|
| 9 |
+
|
| 10 |
+
Dataset used: SQuAD
|
| 11 |
+
Dataset Size: 87278
|
| 12 |
+
Training Steps: 500
|
| 13 |
+
|
| 14 |
+
|
| 15 |
## Training procedure
|
| 16 |
|
| 17 |
|