adding in End Learning Ratio
Browse files
README.md
CHANGED
|
@@ -146,6 +146,7 @@ We trained BloomChat with SambaStudio, a platform built on SambaNova's in-house
|
|
| 146 |
- Learning Rate: 1e-5
|
| 147 |
- Learning Rate Scheduler: Cosine Schedule with Warmup
|
| 148 |
- Warmup Steps: 0
|
|
|
|
| 149 |
- Weight decay: 0.1
|
| 150 |
|
| 151 |
**Instruction-tuned Training on Dolly 2.0 and Oasst1**
|
|
@@ -159,6 +160,7 @@ We trained BloomChat with SambaStudio, a platform built on SambaNova's in-house
|
|
| 159 |
- Learning Rate: 1e-5
|
| 160 |
- Learning Rate Scheduler: Cosine Schedule with Warmup
|
| 161 |
- Warmup Steps: 0
|
|
|
|
| 162 |
- Weight decay: 0.1
|
| 163 |
|
| 164 |
|
|
|
|
| 146 |
- Learning Rate: 1e-5
|
| 147 |
- Learning Rate Scheduler: Cosine Schedule with Warmup
|
| 148 |
- Warmup Steps: 0
|
| 149 |
+
- End Learning Ratio: 0.1
|
| 150 |
- Weight decay: 0.1
|
| 151 |
|
| 152 |
**Instruction-tuned Training on Dolly 2.0 and Oasst1**
|
|
|
|
| 160 |
- Learning Rate: 1e-5
|
| 161 |
- Learning Rate Scheduler: Cosine Schedule with Warmup
|
| 162 |
- Warmup Steps: 0
|
| 163 |
+
- End Learning Ratio: 0.1
|
| 164 |
- Weight decay: 0.1
|
| 165 |
|
| 166 |
|