Update README.md
Browse filesReplace the scores to ones with batch size 1
README.md
CHANGED
|
@@ -48,13 +48,13 @@ In addition, Sarashina2.2-3B outperforms Sarashina2-70B in Japanese math and cod
|
|
| 48 |
|
| 49 |
#### Evaluation in Japanese tasks
|
| 50 |
|
| 51 |
-
| Model
|
| 52 |
-
|------------------
|
| 53 |
-
| [Sarashina2-7B](https://huggingface.co/sbintuitions/sarashina2-7b)
|
| 54 |
-
| [Sarashina2-70B](https://huggingface.co/sbintuitions/sarashina2-70b)
|
| 55 |
-
|**[Sarashina2.2-0.5B](https://huggingface.co/sbintuitions/sarashina2.2-0.5b)**|
|
| 56 |
-
|**[Sarashina2.2-1B](https://huggingface.co/sbintuitions/sarashina2.2-1b)**| 47.2
|
| 57 |
-
|**[Sarashina2.2-3B](https://huggingface.co/sbintuitions/sarashina2.2-3b)**|
|
| 58 |
|
| 59 |
|
| 60 |
## Ethical Considerations and Limitations
|
|
|
|
| 48 |
|
| 49 |
#### Evaluation in Japanese tasks
|
| 50 |
|
| 51 |
+
| Model | NIILC | JMMLU | MGSM-ja | JHumanEval |
|
| 52 |
+
|--------------------------------------------------------------------------------|------------|-----------|------------|-------------|
|
| 53 |
+
| [Sarashina2-7B](https://huggingface.co/sbintuitions/sarashina2-7b) | 61.4 | 42.5 | 8.4 | 12.8 |
|
| 54 |
+
| [Sarashina2-70B](https://huggingface.co/sbintuitions/sarashina2-70b) | **65.4** | **62.7** | 54.0 | 22.0 |
|
| 55 |
+
| **[Sarashina2.2-0.5B](https://huggingface.co/sbintuitions/sarashina2.2-0.5b)** | 33.9 | 28.8 | 21.6 | 15.2 |
|
| 56 |
+
| **[Sarashina2.2-1B](https://huggingface.co/sbintuitions/sarashina2.2-1b)** | 47.2 | 38.2 | 39.6 | 20.7 |
|
| 57 |
+
| **[Sarashina2.2-3B](https://huggingface.co/sbintuitions/sarashina2.2-3b)** | 63.0 | 52.7 | **63.6** | **39.0** |
|
| 58 |
|
| 59 |
|
| 60 |
## Ethical Considerations and Limitations
|