Hugging Face's logo

MrRobotoAI
/
MrRoboto-ProLong-8b-v2h

Text Generation
Transformers
Safetensors
llama
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
MrRobotoAI commited on
Commit
5527c77
•
1 Parent(s): f7b966a

Upload folder using huggingface_hub

Browse files
Files changed (7) hide show
  1. README.md +31 -35
  2. config.json +2 -2
  3. mergekit_config.yml +13 -31
  4. model-00001-of-00004.safetensors +2 -2
  5. model-00002-of-00004.safetensors +2 -2
  6. model-00003-of-00004.safetensors +2 -2
  7. model-00004-of-00004.safetensors +2 -2
README.md CHANGED
@@ -1,8 +1,16 @@
1
  ---
2
  base_model:
3
- - MrRobotoAI/MrRoboto-ProLong-8b-v2e
 
 
 
 
 
 
 
 
4
  - MrRobotoAI/MrRoboto-ProLong-8b-v2g
5
- - MrRobotoAI/MrRoboto-ProLong-8b-v1n
6
  library_name: transformers
7
  tags:
8
  - mergekit
@@ -16,51 +24,39 @@ This is a merge of pre-trained language models created using [mergekit](https://
16
  ## Merge Details
17
  ### Merge Method
18
 
19
- This model was merged using the linear [DARE](https://arxiv.org/abs/2311.03099) merge method using [MrRobotoAI/MrRoboto-ProLong-8b-v1n](https://huggingface.co/MrRobotoAI/MrRoboto-ProLong-8b-v1n) as a base.
20
 
21
  ### Models Merged
22
 
23
  The following models were included in the merge:
24
- * [MrRobotoAI/MrRoboto-ProLong-8b-v2e](https://huggingface.co/MrRobotoAI/MrRoboto-ProLong-8b-v2e)
 
 
 
 
 
25
  * [MrRobotoAI/MrRoboto-ProLong-8b-v2g](https://huggingface.co/MrRobotoAI/MrRoboto-ProLong-8b-v2g)
 
26
 
27
  ### Configuration
28
 
29
  The following YAML configuration was used to produce this model:
30
 
31
  ```yaml
32
- merge_method: dare_linear
33
  models:
34
- - model: MrRobotoAI/MrRoboto-ProLong-8b-v2e
35
- parameters:
36
- weight:
37
- - filter: v_proj
38
- value: [0.2, 0.2, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.2, 0.2]
39
- - filter: o_proj
40
- value: [0.2, 0.2, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.2, 0.2]
41
- - filter: up_proj
42
- value: [0.2, 0.2, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.2, 0.2]
43
- - filter: gate_proj
44
- value: [0.2, 0.2, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.2, 0.2]
45
- - filter: down_proj
46
- value: [0.2, 0.2, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.2, 0.2]
47
- - value: 1
48
  - model: MrRobotoAI/MrRoboto-ProLong-8b-v2g
49
- parameters:
50
- weight:
51
- - filter: v_proj
52
- value: [0.8, 0.8, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.8, 0.8]
53
- - filter: o_proj
54
- value: [0.8, 0.8, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.8, 0.8]
55
- - filter: up_proj
56
- value: [0.8, 0.8, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.8, 0.8]
57
- - filter: gate_proj
58
- value: [0.8, 0.8, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.8, 0.8]
59
- - filter: down_proj
60
- value: [0.8, 0.8, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.8, 0.8]
61
- - value: 0
62
- base_model: MrRobotoAI/MrRoboto-ProLong-8b-v1n
63
- tokenizer_source: base
64
- dtype: bfloat16
65
 
66
  ```
 
1
  ---
2
  base_model:
3
+ - akjindal53244/Llama-3.1-Storm-8B
4
+ - nothingiisreal/L3-8B-Stheno-Horny-v3.3-32K
5
+ - Blackroot/Llama-3-LongStory-LORA
6
+ - cgato/L3-TheSpice-8b-v0.8.3
7
+ - Blackroot/Llama-3-8B-Abomination-LORA
8
+ - DeepAutoAI/ldm_soup_Llama-3.1-8B-Inst
9
+ - MrRobotoAI/MrRoboto-ProLong-8b-v2l
10
+ - v000000/L3.1-Sthenorm-8B
11
+ - OpenBuddy/openbuddy-llama3.1-8b-v22.2-131k
12
  - MrRobotoAI/MrRoboto-ProLong-8b-v2g
13
+ - MrRobotoAI/Thor-v1.4-8b-DARK-FICTION
14
  library_name: transformers
15
  tags:
16
  - mergekit
 
24
  ## Merge Details
25
  ### Merge Method
26
 
27
+ This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [MrRobotoAI/MrRoboto-ProLong-8b-v2l](https://huggingface.co/MrRobotoAI/MrRoboto-ProLong-8b-v2l) as a base.
28
 
29
  ### Models Merged
30
 
31
  The following models were included in the merge:
32
+ * [akjindal53244/Llama-3.1-Storm-8B](https://huggingface.co/akjindal53244/Llama-3.1-Storm-8B)
33
+ * [nothingiisreal/L3-8B-Stheno-Horny-v3.3-32K](https://huggingface.co/nothingiisreal/L3-8B-Stheno-Horny-v3.3-32K) + [Blackroot/Llama-3-LongStory-LORA](https://huggingface.co/Blackroot/Llama-3-LongStory-LORA)
34
+ * [cgato/L3-TheSpice-8b-v0.8.3](https://huggingface.co/cgato/L3-TheSpice-8b-v0.8.3) + [Blackroot/Llama-3-8B-Abomination-LORA](https://huggingface.co/Blackroot/Llama-3-8B-Abomination-LORA)
35
+ * [DeepAutoAI/ldm_soup_Llama-3.1-8B-Inst](https://huggingface.co/DeepAutoAI/ldm_soup_Llama-3.1-8B-Inst)
36
+ * [v000000/L3.1-Sthenorm-8B](https://huggingface.co/v000000/L3.1-Sthenorm-8B)
37
+ * [OpenBuddy/openbuddy-llama3.1-8b-v22.2-131k](https://huggingface.co/OpenBuddy/openbuddy-llama3.1-8b-v22.2-131k)
38
  * [MrRobotoAI/MrRoboto-ProLong-8b-v2g](https://huggingface.co/MrRobotoAI/MrRoboto-ProLong-8b-v2g)
39
+ * [MrRobotoAI/Thor-v1.4-8b-DARK-FICTION](https://huggingface.co/MrRobotoAI/Thor-v1.4-8b-DARK-FICTION)
40
 
41
  ### Configuration
42
 
43
  The following YAML configuration was used to produce this model:
44
 
45
  ```yaml
 
46
  models:
47
+
48
+ - model: OpenBuddy/openbuddy-llama3.1-8b-v22.2-131k
49
+ - model: akjindal53244/Llama-3.1-Storm-8B
50
+ - model: v000000/L3.1-Sthenorm-8B
51
+ - model: DeepAutoAI/ldm_soup_Llama-3.1-8B-Inst
52
+ - model: cgato/L3-TheSpice-8b-v0.8.3+Blackroot/Llama-3-8B-Abomination-LORA
 
 
 
 
 
 
 
 
53
  - model: MrRobotoAI/MrRoboto-ProLong-8b-v2g
54
+ - model: nothingiisreal/L3-8B-Stheno-Horny-v3.3-32K+Blackroot/Llama-3-LongStory-LORA
55
+ - model: MrRobotoAI/Thor-v1.4-8b-DARK-FICTION
56
+
57
+ merge_method: model_stock
58
+ base_model: MrRobotoAI/MrRoboto-ProLong-8b-v2l
59
+ normalize: true
60
+ dtype: float16
 
 
 
 
 
 
 
 
 
61
 
62
  ```
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "MrRobotoAI/MrRoboto-ProLong-8b-v1n",
3
  "architectures": [
4
  "LlamaForCausalLM"
5
  ],
@@ -23,7 +23,7 @@
23
  "rope_scaling": null,
24
  "rope_theta": 2804339835.0,
25
  "tie_word_embeddings": false,
26
- "torch_dtype": "bfloat16",
27
  "transformers_version": "4.46.2",
28
  "use_cache": true,
29
  "vocab_size": 128256
 
1
  {
2
+ "_name_or_path": "MrRobotoAI/MrRoboto-ProLong-8b-v2l",
3
  "architectures": [
4
  "LlamaForCausalLM"
5
  ],
 
23
  "rope_scaling": null,
24
  "rope_theta": 2804339835.0,
25
  "tie_word_embeddings": false,
26
+ "torch_dtype": "float16",
27
  "transformers_version": "4.46.2",
28
  "use_cache": true,
29
  "vocab_size": 128256
mergekit_config.yml CHANGED
@@ -1,33 +1,15 @@
1
- merge_method: dare_linear
2
  models:
3
- - model: MrRobotoAI/MrRoboto-ProLong-8b-v2e
4
- parameters:
5
- weight:
6
- - filter: v_proj
7
- value: [0.2, 0.2, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.2, 0.2]
8
- - filter: o_proj
9
- value: [0.2, 0.2, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.2, 0.2]
10
- - filter: up_proj
11
- value: [0.2, 0.2, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.2, 0.2]
12
- - filter: gate_proj
13
- value: [0.2, 0.2, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.2, 0.2]
14
- - filter: down_proj
15
- value: [0.2, 0.2, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.2, 0.2]
16
- - value: 1
17
  - model: MrRobotoAI/MrRoboto-ProLong-8b-v2g
18
- parameters:
19
- weight:
20
- - filter: v_proj
21
- value: [0.8, 0.8, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.8, 0.8]
22
- - filter: o_proj
23
- value: [0.8, 0.8, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.8, 0.8]
24
- - filter: up_proj
25
- value: [0.8, 0.8, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.8, 0.8]
26
- - filter: gate_proj
27
- value: [0.8, 0.8, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.8, 0.8]
28
- - filter: down_proj
29
- value: [0.8, 0.8, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.55, 0.8, 0.8]
30
- - value: 0
31
- base_model: MrRobotoAI/MrRoboto-ProLong-8b-v1n
32
- tokenizer_source: base
33
- dtype: bfloat16
 
 
1
  models:
2
+
3
+ - model: OpenBuddy/openbuddy-llama3.1-8b-v22.2-131k
4
+ - model: akjindal53244/Llama-3.1-Storm-8B
5
+ - model: v000000/L3.1-Sthenorm-8B
6
+ - model: DeepAutoAI/ldm_soup_Llama-3.1-8B-Inst
7
+ - model: cgato/L3-TheSpice-8b-v0.8.3+Blackroot/Llama-3-8B-Abomination-LORA
 
 
 
 
 
 
 
 
8
  - model: MrRobotoAI/MrRoboto-ProLong-8b-v2g
9
+ - model: nothingiisreal/L3-8B-Stheno-Horny-v3.3-32K+Blackroot/Llama-3-LongStory-LORA
10
+ - model: MrRobotoAI/Thor-v1.4-8b-DARK-FICTION
11
+
12
+ merge_method: model_stock
13
+ base_model: MrRobotoAI/MrRoboto-ProLong-8b-v2l
14
+ normalize: true
15
+ dtype: float16
 
 
 
 
 
 
 
 
 
model-00001-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b49c7786a0438d8c75dd2db6a3b2b9db6038dd74463cc8a0e37da2c9cfc21981
3
- size 4953586384
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:87cb93d68ff5e647e19bad5adf052d3700396a4d29917583b2f36b9173011e96
3
+ size 4953586328
model-00002-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:184dab4aa343b306cf17aedeec57adae78ead74c436d134ef72046387fed606c
3
- size 4999819336
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d428e29c0d524c6e7dcf960f6eec01201748bd092bdf32e61d63486841e0b2f6
3
+ size 4999819232
model-00003-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fd10bf731074047bc67a435eeef404d1c8dd99a832c1d0e4fd3ce43a8669029c
3
- size 4915916144
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c0e6e0ff38e4cbb18530d3f585fd2f4aedd6a16da7a2f071634dc888ebee2ca0
3
+ size 4915916048
model-00004-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0bf4d79d52512f7cda4779f4ff0ac40fd6e3d9f9e3815534dd10ee181febbe99
3
- size 1191234472
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f7a93c6812ac2d967509bb31248b0145eacf0d4adc4eead3b6dd5fe7eec09675
3
+ size 1191234448