|
--- |
|
base_model: |
|
- nitky/Oumuamua-7b-base |
|
- stabilityai/japanese-stablelm-base-gamma-7b |
|
- tokyotech-llm/Swallow-MS-7b-v0.1 |
|
- DataPilot/ArrowPro-7B-KillerWhale |
|
- nitky/RP-7b-instruct |
|
- mistralai/Mistral-7B-Instruct-v0.3 |
|
- mistralai/Mistral-7B-v0.1 |
|
library_name: transformers |
|
tags: |
|
- mergekit |
|
- merge |
|
language: |
|
- ja |
|
- en |
|
pipeline_tag: text-generation |
|
license: apache-2.0 |
|
--- |
|
# Oumuamua-7b-instruct |
|
|
|
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). |
|
|
|
## Output example |
|
|
|
``` |
|
[INST] <<SYS>> |
|
あなたは日本語を話す優秀なアシスタントです。回答には必ず日本語で答えてください。また考える過程も出力してください。 |
|
<</SYS>> |
|
|
|
道路で泣いている子供がいます。その子供には何があったと考えられますか？ [/INST] 道路で泣いている子供がいる場合、考えられる理由はいくつかあります。 |
|
|
|
1. 怪我をした: 転んで怪我をしたり、何かにぶつかって痛みを感じている可能性があります。 |
|
2. 迷子になった: 親や保護者とはぐれてしまった、または道に迷って不安になっている可能性があります。 |
|
3. お腹が空いた: 長時間食事をしていない、またはお腹が空いているために泣いている可能性があります。 |
|
4. 疲れた: 長時間歩いていたり、遊び疲れて疲れている可能性があります。 |
|
5. 寂しかった: 一人でいることに寂しさを感じている、または親や友達がいなくて寂しいと感じている可能性があります。 |
|
6. 怖い思いをした: 車や不審者に遭遇した、または何かに驚いた可能性があります。 |
|
7. 体調が悪い: 発熱や頭痛、吐き気などの体調不良を感じている可能性があります。 |
|
8. トイレに行きたい: おむつが濡れていたり、トイレに行きたくて泣いている可能性があります。 |
|
9. 何かを求めている: 注意を引きたかったり、助けを求めている可能性があります。 |
|
|
|
子供の状況をよく観察し、声をかけて話を聞いてみることが大切です。必要に応じて、保護者や警察、救急車を呼ぶなどの対応をする必要があるかもしれません。 |
|
``` |
|
|
|
## Test environment |
|
|
|
This model was tested using [text-generation-webui](https://github.com/oobabooga/text-generation-webui/tree/main). I use preset `min_p` and `Null preset` with temperature=0.3 for Generation. |
|
|
|
|
|
## Usage |
|
|
|
This format must be adhered to strictly, as deviations may result in less optimal outputs from the model. |
|
|
|
The template used to construct a prompt for the Instruct model is specified as follows: |
|
|
|
``` |
|
<s>[INST] <<SYS>>\n{SYSTEM_PROMPT}\n<</SYS>>\n\n{USER_MESSAGE_1} [/INST] {BOT_MESSAGE_1}</s>[INST] {USER_MESSAGE_2} [/INST] |
|
``` |
|
|
|
|
|
Please be aware that ``<s>`` and ``</s>`` are special tokens used for the beginning of string (BOS) and end of string (EOS), respectively, while [INST] and [/INST] are considered regular strings. |
|
|
|
For the "{SYSTEM_PROMPT}" part, We recommend using "あなたは日本語を話す優秀なアシスタントです。回答には必ず日本語で答えてください。また考える過程も出力してください。" |
|
|
|
For the "{USER_MESSAGE_1}" part, We recommend using {instruction}\n{input} |
|
|
|
In other words, We recommend the following: |
|
|
|
``` |
|
<s>[INST] <<SYS>>\nあなたは日本語を話す優秀なアシスタントです。回答には必ず日本語で答えてください。また考える過程も出力してください。\n<</SYS>>\n\n{instruction1}\n{input1} [/INST] {BOT_MESSAGE_1}</s>[INST] {instruction2}\n{input2} [/INST] |
|
``` |
|
|
|
### Use the instruct model |
|
|
|
```python |
|
import torch |
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
model_name = "nitky/Oumuamua-7b-instruct" |
|
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto") |
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
|
device = "cuda" |
|
|
|
messages = [ |
|
{"role": "system", "content": "あなたは日本語を話す優秀なアシスタントです。回答には必ず日本語で答えてください。また考える過程も出力してください。"}, |
|
{"role": "user", "content": "道路で泣いている子供がいます。その子供には何があったと考えられますか？"} |
|
] |
|
|
|
encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt") |
|
|
|
model_inputs = encodeds.to(device) |
|
model.to(device) |
|
|
|
generated_ids = model.generate(model_inputs, max_new_tokens=256, do_sample=True, temperature=0.3) |
|
decoded = tokenizer.batch_decode(generated_ids) |
|
print(decoded[0]) |
|
``` |
|
|
|
## Merge Details |
|
### Merge Method |
|
|
|
This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [nitky/Oumuamua-7b-base](https://huggingface.co/nitky/Oumuamua-7b-base) as a base. |
|
|
|
### Models Merged |
|
|
|
The following models were included in the merge: |
|
* [stabilityai/japanese-stablelm-base-gamma-7b](https://huggingface.co/stabilityai/japanese-stablelm-base-gamma-7b) |
|
* [tokyotech-llm/Swallow-MS-7b-v0.1](https://huggingface.co/tokyotech-llm/Swallow-MS-7b-v0.1) |
|
* [DataPilot/ArrowPro-7B-KillerWhale](https://huggingface.co/DataPilot/ArrowPro-7B-KillerWhale) |
|
* [nitky/RP-7b-instruct](https://huggingface.co/nitky/RP-7b-instruct) |
|
* [mistralai/Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) |
|
* [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) |
|
|
|
### Configuration |
|
|
|
The following YAML configuration was used to produce this model: |
|
|
|
```yaml |
|
merge_method: task_arithmetic |
|
base_model: mistralai/Mistral-7B-v0.1 |
|
models: |
|
- model: tokyotech-llm/Swallow-MS-7b-v0.1 |
|
parameters: |
|
weight: |
|
- filter: embed_tokens |
|
value: 1.0 |
|
- value: 0 |
|
dtype: bfloat16 |
|
tokenizer_source: model:tokyotech-llm/Swallow-MS-7b-v0.1 |
|
name: Mistral-7B-v0.1-VE-Swallow-MS |
|
--- |
|
merge_method: task_arithmetic |
|
base_model: stabilityai/japanese-stablelm-base-gamma-7b |
|
models: |
|
- model: tokyotech-llm/Swallow-MS-7b-v0.1 |
|
parameters: |
|
weight: |
|
- filter: embed_tokens |
|
value: 1.0 |
|
- value: 0 |
|
dtype: bfloat16 |
|
tokenizer_source: model:tokyotech-llm/Swallow-MS-7b-v0.1 |
|
name: japanese-stablelm-base-gamma-7b-VE-Swallow-MS |
|
--- |
|
merge_method: task_arithmetic |
|
base_model: DataPilot/ArrowPro-7B-KillerWhale |
|
models: |
|
- model: tokyotech-llm/Swallow-MS-7b-v0.1 |
|
parameters: |
|
weight: |
|
- filter: embed_tokens |
|
value: 1.0 |
|
- value: 0 |
|
dtype: bfloat16 |
|
tokenizer_source: model:tokyotech-llm/Swallow-MS-7b-v0.1 |
|
name: ArrowPro-7B-KillerWhale-VE-Swallow-MS |
|
--- |
|
merge_method: task_arithmetic |
|
base_model: nitky/RP-7b-instruct |
|
models: |
|
- model: tokyotech-llm/Swallow-MS-7b-v0.1 |
|
parameters: |
|
weight: |
|
- filter: embed_tokens |
|
value: 1.0 |
|
- value: 0 |
|
dtype: bfloat16 |
|
tokenizer_source: model:tokyotech-llm/Swallow-MS-7b-v0.1 |
|
name: RP-7b-instruct-VE-Swallow-MS |
|
--- |
|
merge_method: task_arithmetic |
|
base_model: mistralai/Mistral-7B-Instruct-v0.3 |
|
models: |
|
- model: tokyotech-llm/Swallow-MS-7b-v0.1 |
|
parameters: |
|
weight: |
|
- filter: embed_tokens |
|
value: 1.0 |
|
- value: 0 |
|
dtype: bfloat16 |
|
tokenizer_source: model:tokyotech-llm/Swallow-MS-7b-v0.1 |
|
name: Mistral-7B-Instruct-v0.3-VE-Swallow-MS |
|
--- |
|
merge_method: linear |
|
models: |
|
- model: nitky/Oumuamua-7b-base |
|
parameters: |
|
weight: 1.0 |
|
- model: tokyotech-llm/Swallow-MS-7b-instruct-v0.1 |
|
parameters: |
|
weight: 0.8 |
|
- model: tokyotech-llm/Swallow-MS-7b-v0.1 |
|
parameters: |
|
weight: -0.8 |
|
- model: Mistral-7B-Instruct-v0.3-VE-Swallow-MS |
|
parameters: |
|
weight: 0.8 |
|
- model: Mistral-7B-v0.1-VE-Swallow-MS |
|
parameters: |
|
weight: -0.8 |
|
dtype: bfloat16 |
|
name: Oumuamua-7b-instruct-alpha |
|
--- |
|
merge_method: linear |
|
models: |
|
- model: nitky/Oumuamua-7b-base |
|
parameters: |
|
weight: 1.0 |
|
- model: ArrowPro-7B-KillerWhale-VE-Swallow-MS # include Mistral-7B-Instruct-v0.3 |
|
parameters: |
|
weight: 0.8 |
|
- model: japanese-stablelm-base-gamma-7b-VE-Swallow-MS |
|
parameters: |
|
weight: -0.8 |
|
dtype: bfloat16 |
|
name: Oumuamua-7b-instruct-beta |
|
--- |
|
merge_method: linear |
|
models: |
|
- model: nitky/Oumuamua-7b-base |
|
parameters: |
|
weight: 1.0 |
|
- model: RP-7b-instruct-VE-Swallow-MS |
|
parameters: |
|
weight: 0.8 |
|
- model: japanese-stablelm-base-gamma-7b-VE-Swallow-MS |
|
parameters: |
|
weight: -0.8 |
|
- model: Mistral-7B-Instruct-v0.3-VE-Swallow-MS |
|
parameters: |
|
weight: 0.8 |
|
- model: Mistral-7B-v0.1-VE-Swallow-MS |
|
parameters: |
|
weight: -0.8 |
|
dtype: bfloat16 |
|
name: Oumuamua-7b-instruct-gamma |
|
--- |
|
merge_method: model_stock |
|
base_model: nitky/Oumuamua-7b-base |
|
models: |
|
- model: Oumuamua-7b-instruct-alpha |
|
- model: Oumuamua-7b-instruct-beta |
|
- model: Oumuamua-7b-instruct-gamma |
|
dtype: bfloat16 |
|
name: Oumuamua-7b-instruct |
|
``` |
|
|
