Hugging Face's logo

denru
/
Monstral-123B-v2x2

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
denru commited on
Commit
615cff3
·
verified ·
1 Parent(s): 766c877

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +39 -0
  2. config.json +27 -0
  3. mergekit_config.yml +10 -0
  4. model-00001-of-00071.safetensors +3 -0
  5. model-00002-of-00071.safetensors +3 -0
  6. model-00003-of-00071.safetensors +3 -0
  7. model-00004-of-00071.safetensors +3 -0
  8. model-00005-of-00071.safetensors +3 -0
  9. model-00006-of-00071.safetensors +3 -0
  10. model-00007-of-00071.safetensors +3 -0
  11. model-00008-of-00071.safetensors +3 -0
  12. model-00009-of-00071.safetensors +3 -0
  13. model-00010-of-00071.safetensors +3 -0
  14. model-00011-of-00071.safetensors +3 -0
  15. model-00012-of-00071.safetensors +3 -0
  16. model-00013-of-00071.safetensors +3 -0
  17. model-00014-of-00071.safetensors +3 -0
  18. model-00015-of-00071.safetensors +3 -0
  19. model-00016-of-00071.safetensors +3 -0
  20. model-00017-of-00071.safetensors +3 -0
  21. model-00018-of-00071.safetensors +3 -0
  22. model-00019-of-00071.safetensors +3 -0
  23. model-00020-of-00071.safetensors +3 -0
  24. model-00021-of-00071.safetensors +3 -0
  25. model-00022-of-00071.safetensors +3 -0
  26. model-00023-of-00071.safetensors +3 -0
  27. model-00024-of-00071.safetensors +3 -0
  28. model-00025-of-00071.safetensors +3 -0
  29. model-00026-of-00071.safetensors +3 -0
  30. model-00027-of-00071.safetensors +3 -0
  31. model-00028-of-00071.safetensors +3 -0
  32. model-00029-of-00071.safetensors +3 -0
  33. model-00030-of-00071.safetensors +3 -0
  34. model-00031-of-00071.safetensors +3 -0
  35. model-00032-of-00071.safetensors +3 -0
  36. model-00033-of-00071.safetensors +3 -0
  37. model-00034-of-00071.safetensors +3 -0
  38. model-00035-of-00071.safetensors +3 -0
  39. model-00036-of-00071.safetensors +3 -0
  40. model-00037-of-00071.safetensors +3 -0
  41. model-00038-of-00071.safetensors +3 -0
  42. model-00039-of-00071.safetensors +3 -0
  43. model-00040-of-00071.safetensors +3 -0
  44. model-00041-of-00071.safetensors +3 -0
  45. model-00042-of-00071.safetensors +3 -0
  46. model-00043-of-00071.safetensors +3 -0
  47. model-00044-of-00071.safetensors +3 -0
  48. model-00045-of-00071.safetensors +3 -0
  49. model-00046-of-00071.safetensors +3 -0
  50. model-00047-of-00071.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - MarsupialAI/Monstral-123B-v2
4
+ library_name: transformers
5
+ tags:
6
+ - mergekit
7
+ - merge
8
+
9
+ ---
10
+ # merged_model
11
+
12
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
13
+
14
+ ## Merge Details
15
+ ### Merge Method
16
+
17
+ This model was merged using the passthrough merge method.
18
+
19
+ ### Models Merged
20
+
21
+ The following models were included in the merge:
22
+ * [MarsupialAI/Monstral-123B-v2](https://huggingface.co/MarsupialAI/Monstral-123B-v2)
23
+
24
+ ### Configuration
25
+
26
+ The following YAML configuration was used to produce this model:
27
+
28
+ ```yaml
29
+ slices:
30
+ - sources:
31
+ - model: MarsupialAI/Monstral-123B-v2
32
+ layer_range: [0, 61]
33
+ - sources:
34
+ - model: MarsupialAI/Monstral-123B-v2
35
+ layer_range: [27, 88]
36
+ merge_method: passthrough
37
+ dtype: float16
38
+ name: monstral
39
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "MarsupialAI/Monstral-123B-v2",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 12288,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 96,
17
+ "num_hidden_layers": 122,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "float16",
24
+ "transformers_version": "4.47.1",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ slices:
2
+ - sources:
3
+ - model: MarsupialAI/Monstral-123B-v2
4
+ layer_range: [0, 61]
5
+ - sources:
6
+ - model: MarsupialAI/Monstral-123B-v2
7
+ layer_range: [27, 88]
8
+ merge_method: passthrough
9
+ dtype: float16
10
+ name: monstral
model-00001-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6352ac86f29fdadb4aaec1e95647db2a60255cb9872f3458bdf4196f311cccd
3
+ size 4378928488
model-00002-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e17a9b2a1e4500c5a9fd7a219e880a8670bf688b68dc9ab29618a0aec74e83b
3
+ size 4907411072
model-00003-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1b6590c5b4e6d37323ce3d6666881e987e4a2321d3d9621820bb0319079defe
3
+ size 4806747888
model-00004-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:220eb35df35d5c24c31528ecbeb3570bc7ee464ccf4b1e50654e50dae2a719b9
3
+ size 4831938528
model-00005-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:063e5db6771b651c709190ef1d0ffef89dae2204648044ccaad30d258d23cc95
3
+ size 4831938536
model-00006-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d54039dd70f461b164c0905f9203f8aaf9bf0921da3ccb6a48003ec5da824f4
3
+ size 4907411080
model-00007-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0752ad241695d36a76a973463ffecba679e19c621863283f029c22f55481b225
3
+ size 4806747888
model-00008-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:168640080680cb0f534d8e89ccf2e51e24526147c5f387fe1e52f5f43119e789
3
+ size 4831938520
model-00009-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c28a21aecb702c8fc049b8ae20d4f57672f09bd392cd58e815d4a0efeabc56f
3
+ size 4831938536
model-00010-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5291e0cabd1baaa6b7871a64207c367e633817c95bb3bffe8f965b72474d7d50
3
+ size 4907411080
model-00011-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e902aa69bafc0553661efd2fbb12e6a8564a4bc33b30567680ca39a38e26c2b
3
+ size 4806747888
model-00012-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:540d7141babd99b521f69456a29b2c8f1d118b530561d14dad9edbbe0ef8930b
3
+ size 4831963216
model-00013-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6b2bd20df8a9bcd7d9dd6cd4b55e4af251f73bcb569c1fcd38386b2c00d9f54
3
+ size 4831938536
model-00014-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e437274e55686a042273618e8f412f8e857ec571e61e5dfbcd4c3ab5f0449b3
3
+ size 4882220448
model-00015-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17e07e7d8ad08960299ad262d22d0c147ab8c26278b07028fa16855a10124d5a
3
+ size 4932601704
model-00016-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:707cd1f4990809d41b9b8a3860f7a9eae58dee5a26614c2f247b366cf5d082ce
3
+ size 4731275336
model-00017-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9042a76fde5477b979eff4add51c45f25c42af6498c97d0b5418351b554b682b
3
+ size 4831938536
model-00018-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e9c1bd012770b202b66c5849e95be2aee7df46c3c630c30cf79200860b9b2ffa
3
+ size 4882220448
model-00019-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a0f35f37f60138a9efdc5ce602a6df1bc425f794ca15f1f2ff22faa9d3b30dd
3
+ size 4932601704
model-00020-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84209bb04856251d1cf57fec1c3c2c8d4056c1c5a344aea00330985a8bc3aa40
3
+ size 4781557256
model-00021-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:caaf0e5b78824d063f93f23bf69bea319280826d51db02cd25459f951d1fca93
3
+ size 4831938536
model-00022-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c79514a9eae5307df4419b54ed65cec5bc70d63cb9e66df3fb970938591a7da
3
+ size 4831938528
model-00023-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a0768851cf27792d8691fa6013b0424e14b75d03a9138d8a9e61f98be3d3db5
3
+ size 4831938536
model-00024-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:47ea008f886132c305b3f0a8cadb44a3a595b0d685bb2da9c986ca237817161f
3
+ size 4831938536
model-00025-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6064d7cbaac8ce4051de8457086e3aae07f2f60b1a6f8a2aebd0c7958f47255d
3
+ size 4831938536
model-00026-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:02d1ed3f7c40d4e3e421eaf5e9702f024cc4cd5ee5dd8600acf2e62df8110e6f
3
+ size 4882220448
model-00027-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7030be8a2bd52b33eaa4d6a09884be63461207fcb397755ac88369d63c99a65c
3
+ size 4932601704
model-00028-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5aa30be050e7b15ef0d186006783cc15e3fdabb637d990a469c888fa54c65356
3
+ size 4731275336
model-00029-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:105bccee2286bd53a6f72dbcad5210dab36cff42009ff872cd5e992b25908b9a
3
+ size 4831938536
model-00030-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d77d747b371d2dbc7ce3fa687579a5adbebf8af7d20e9e9100edae29caa12a34
3
+ size 4882220448
model-00031-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:610f4319c904614899249b88a070001c038976b5f27360af1bbcd983d89c60d1
3
+ size 4932601704
model-00032-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6a0fdb6cd3f18e9341b26fe03c767ae7629c6db323a6118032495106f151ee40
3
+ size 4781557256
model-00033-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe1d715d097e2f3f2b229542a44fada6cde0668fb09bfcc0fd9d75dec8e9216c
3
+ size 4831938536
model-00034-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0e7b6c30e5f88972a25d95769ed63913beffba7ccda97473b253b43cfb3e892e
3
+ size 4831938528
model-00035-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6da3fd77bd7d28155fb96c20f0bac37b0cd616fe40928cb63214491c9e99835c
3
+ size 4831938536
model-00036-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:251fc71d96bf4671d3be73e7fff7b588831afbcdab277c047da3bfd6cd2ca867
3
+ size 4831938536
model-00037-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a0447e7cb00497647a33b6ad7bfee92d893a48ee45ad992cd79395dfdd1e91ed
3
+ size 4831938536
model-00038-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5196575076b78de2b8af6d32ffe47feb64e0a542469acb55346ac848ccb99d40
3
+ size 4882220448
model-00039-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff184d4db9e783fc1705aebf245e30a23369b76019f155a347ca1227d619238b
3
+ size 4932601704
model-00040-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62d10225758b85a1fff57ed1d967453967c67f2240dc53a06bb3c9d22c099e49
3
+ size 4731275336
model-00041-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9ea732d139aac5655a67242a1b05fb5c420d5fe89a2e05c8dee7f455d47e6e50
3
+ size 4831938536
model-00042-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:986d61b48da617965019b0ff367b07d16e438891c26d7701289f2ebe03537662
3
+ size 4882220448
model-00043-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60f8599504d4599ad3f18b23d00203b55c1cf506f0c38a1261852b1120f0eedc
3
+ size 4932601704
model-00044-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:34e5b723442ac5243fb5460094ec844fbe38da4b871fb3de90dc8b2f78b8fba4
3
+ size 4781557256
model-00045-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9041b8217fb0f9c51760bcd1a34ef15e9825248d29412579668dd206db4e21eb
3
+ size 4831938536
model-00046-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9c59f549c52a9337302d91b687ab24a5c463afba1bc813ace062e14d3bd0a4e8
3
+ size 4831938528
model-00047-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ca2bc2ca74370ec23e25246e769b96c0caa140fcaeececfd3890515512879e21
3
+ size 4831938536