Text Generation
Transformers
Safetensors
Japanese
mistral
roleplay
Merge
mergekit
Inference Endpoints
text-generation-inference
Aratako commited on
Commit
284bf0e
1 Parent(s): 777f74a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +53 -42
README.md CHANGED
@@ -1,42 +1,53 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # Ninja-RP-MS
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using ./Ninja-v1-NSFW-RP as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * ./Ninja-v1-NSFW-RP-SiliconMaid
22
- * ./Ninja-v1-NSFW-RP-LoyalMacaroniMaid
23
- * ./Ninja-v1-NSFW-RP-WestLake
24
- * ./Ninja-v1-NSFW-RP-Kunoichi
25
-
26
- ### Configuration
27
-
28
- The following YAML configuration was used to produce this model:
29
-
30
- ```yaml
31
- models:
32
- - model: ./Ninja-v1-NSFW-RP
33
- - model: ./Ninja-v1-NSFW-RP-Kunoichi
34
- - model: ./Ninja-v1-NSFW-RP-SiliconMaid
35
- - model: ./Ninja-v1-NSFW-RP-WestLake
36
- - model: ./Ninja-v1-NSFW-RP-LoyalMacaroniMaid
37
- merge_method: model_stock
38
- base_model: ./Ninja-v1-NSFW-RP
39
- dtype: bfloat16
40
- tokenizer_source: union
41
-
42
- ```
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - Aratako/Rosebleu-1on1-Dialogues-RP
5
+ - Aratako/LimaRP-augmented-ja-karakuri
6
+ - grimulkan/LimaRP-augmented
7
+ - Aratako/Bluemoon_Top50MB_Sorted_Fixed_ja
8
+ - SicariusSicariiStuff/Bluemoon_Top50MB_Sorted_Fixed
9
+ - OmniAICreator/Japanese-Roleplay
10
+ language:
11
+ - ja
12
+ library_name: transformers
13
+ tags:
14
+ - roleplay
15
+ base_model:
16
+ - Aratako/Ninja-v1-RP-WIP
17
+ ---
18
+
19
+ # Ninja-v1-RP
20
+
21
+ ## 概要
22
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
23
+
24
+ [Aratako/Ninja-v1-RP-WIP](https://huggingface.co/Aratako/Ninja-v1-RP-WIP)をベースに、Task Vectorの加算・Model Stockによるマージを行い指示追従能力と表現力を強化したロールプレイ用モデルです。
25
+
26
+ ## マージの詳細
27
+ まず、[Aratako/Ninja-v1-RP-WIP](https://huggingface.co/Aratako/Ninja-v1-RP-WIP)に対し、以下4モデルのTask Vectorを0.8倍して加算し、4種類、Task Vector加算モデルを作成しました。
28
+
29
+ - [senseable/WestLake-7B-v2](https://huggingface.co/senseable/WestLake-7B-v2)
30
+ - [SanjiWatsuki/Kunoichi-DPO-v2-7B](https://huggingface.co/SanjiWatsuki/Kunoichi-DPO-v2-7B)
31
+ - [SanjiWatsuki/Silicon-Maid-7B](https://huggingface.co/SanjiWatsuki/Silicon-Maid-7B)
32
+ - [SanjiWatsuki/Loyal-Macaroni-Maid-7B](https://huggingface.co/SanjiWatsuki/Loyal-Macaroni-Maid-7B)
33
+
34
+ 各モデルのTask Vectorの加算の式は以下の通りです。
35
+ ```
36
+ new_model = Ninja-v1-RP-WIP + 0.8 * (target_model - Mistral-7B-v0.1)
37
+ ```
38
+
39
+ 次に、このTask Vector加算によってできた4モデルと元のモデルを、Model Stockという手法を用い以下のようなconfigを使ってmergekitでマージし、このモデルを作成しました。
40
+
41
+ ```yaml
42
+ models:
43
+ - model: ./Ninja-v1-RP-WIP
44
+ - model: ./Ninja-v1-RP-WIP-Kunoichi
45
+ - model: ./Ninja-v1-RP-WIP-SiliconMaid
46
+ - model: ./Ninja-v1-RP-WIP-WestLake
47
+ - model: ./Ninja-v1-RP-WIP-LoyalMacaroniMaid
48
+ merge_method: model_stock
49
+ base_model: ./Ninja-v1-RP-WIP
50
+ dtype: bfloat16
51
+ tokenizer_source: union
52
+
53
+ ```