Update README.md
Browse files
README.md
CHANGED
@@ -6,10 +6,10 @@ datasets:
|
|
6 |
- appvoid/no-prompt-15k
|
7 |
pipeline_tag: text-generation
|
8 |
---
|
9 |
-
![palmer](https://huggingface.co/appvoid/
|
10 |
-
#
|
11 |
-
### a
|
12 |
-
|
13 |
|
14 |
### evaluation
|
15 |
|Model| ARC_C| HellaSwag| PIQA| Winogrande|
|
@@ -20,7 +20,7 @@ palmer is a series of ~1b parameters language models fine-tuned to be used as ba
|
|
20 |
|no-prompt-1.3b| 0.3157| **0.6022**| 0.7334| 0.5864|
|
21 |
|falcon-rw-1b-instruct-openorca (sota) | **0.3362**| 0.5997| **0.7394**| **0.6148**|
|
22 |
|
23 |
-
This model was trained on less than 25% of the dataset yet achieves competitive performance to current sota on open llm leaderboard.
|
24 |
|
25 |
### training
|
26 |
Training took ~5 P100 gpu hours. It was trained on 15,000 gpt-4 shuffled samples. palmer was fine-tuned using lower learning rates ensuring it keeps as much general knowledge as possible.
|
|
|
6 |
- appvoid/no-prompt-15k
|
7 |
pipeline_tag: text-generation
|
8 |
---
|
9 |
+
![palmer](https://huggingface.co/appvoid/no-prompt-1.3b/resolve/main/_ccd1a5dd-2ddc-4d5a-8163-fd6d1b39f5f4.jpeg?download=true)
|
10 |
+
# no-prompt
|
11 |
+
### a sheared-llama-1.3b fine-tuning
|
12 |
+
This model uses an 1.3 billion parameters model as base to be further fine-tuned on the same data as palmer. It works pretty good and even surpasses sota model on `hellaswag`.
|
13 |
|
14 |
### evaluation
|
15 |
|Model| ARC_C| HellaSwag| PIQA| Winogrande|
|
|
|
20 |
|no-prompt-1.3b| 0.3157| **0.6022**| 0.7334| 0.5864|
|
21 |
|falcon-rw-1b-instruct-openorca (sota) | **0.3362**| 0.5997| **0.7394**| **0.6148**|
|
22 |
|
23 |
+
This model was trained on less than 25% of the dataset yet achieves competitive performance to current sota on open llm leaderboard.
|
24 |
|
25 |
### training
|
26 |
Training took ~5 P100 gpu hours. It was trained on 15,000 gpt-4 shuffled samples. palmer was fine-tuned using lower learning rates ensuring it keeps as much general knowledge as possible.
|