File size: 2,660 Bytes
ca2c22c cf187b1 ca2c22c cf187b1 ca2c22c 3168332 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
---
language:
- ja
tags:
- japanese-stablelm
- causal-lm
pipeline_tag: text-generation
base_model: stabilityai/japanese-stablelm-base-gamma-7b
license: apache-2.0
extra_gated_fields:
Name: text
Email: text
Country: text
Organization or Affiliation: text
I allow Stability AI to contact me about information related to its models and research: checkbox
---
# Reproduced Japanese Stable LM Instruct Gamma 7B
## Model Description
This is a reproduction of 7B-parameter decoder-only Japanese language model fine-tuned on instruction-following datasets, built on top of the base model [Japanese Stable LM Base Gamma 7B](https://huggingface.co/stabilityai/japanese-stablelm-base-gamma-7b).
This model is trained with [notus](https://github.com/argilla-io/notus) code base.
*If you are in search of the official model, please check [Japanese Stable LM Instruct Gamma 7B](https://huggingface.co/stabilityai/japanese-stablelm-instruct-gamma-7b).*
## Model Details
### Training Datasets
- [Japanese translation of the Databricks Dolly-15k dataset](https://huggingface.co/datasets/kunishou/databricks-dolly-15k-ja)
- [Japanese translation of the subset of the Anthropic HH dataset](https://huggingface.co/datasets/fujiki/japanese_hh-rlhf-49k)
- [Wikinews](https://ja.wikinews.org/wi) [subset](https://huggingface.co/datasets/fujiki/llm-japanese-dataset_wikinews) of the [izumi-lab/llm-japanese-dataset](https://huggingface.co/datasets/izumi-lab/llm-japanese-dataset)
### Benchmarks
The result is evaluated by [Nejumi-leaderboard Neo](https://github.com/wandb/llm-leaderboard/tree/b2723944d4955768cb93c18ffe162a8ff4e88955).
- llm-jp-eval:
|AVG |EL |FA |MC |MR |NLI |QA |RC |chabsa|jamp |janli|jcommonsenseqa|jemhopqa|jnli |jsem |jsick|jsquad |mawps |niilc |wiki_coreference|wiki_dependency|wiki_ner|wiki_pas|wiki_reading|
|------|---|----|-----|----|------|-------|-------|------|-----|-----|--------------|--------|-----|-----|-----|-------|------|------|----------------|---------------|--------|--------|------------|
|0.26 |0 |0.14|0.27 |0.1 |0.302 |0.2619 |0.7464 |0.0 |0.15 |0.5 |0.27 |0.2528 |0.04 |0.67 |0.15 |0.7464 |0.1 |0.271 |0.0 |0.0 |0.0 |0.0 |0.7 |
- Japanese Mt-Bench:
|coding|extraction|humanities|math|reasoning|roleplay|stem|writing|
|------|----------|----------|----|---------|--------|----|-------|
|1.3 |1.75 |2.35 |1.45|3.4 |5.8 |4.3 |3.1 |
- Overall Average: 0.283125
## Credits
The training was carried out by [Hwigeon Oh](https://huggingface.co/ohwi) and [Fujiki Nakamura](https://huggingface.co/fujiki). |