Hugging Face's logo

mashleburneded
/
Mobiusv0.2

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
mashleburneded commited on
Commit
b16bf33
•
1 Parent(s): 5fc784f

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +56 -0
  2. config.json +27 -0
  3. mergekit_config.yml +18 -0
  4. model-00001-of-00051.safetensors +3 -0
  5. model-00002-of-00051.safetensors +3 -0
  6. model-00003-of-00051.safetensors +3 -0
  7. model-00004-of-00051.safetensors +3 -0
  8. model-00005-of-00051.safetensors +3 -0
  9. model-00006-of-00051.safetensors +3 -0
  10. model-00007-of-00051.safetensors +3 -0
  11. model-00008-of-00051.safetensors +3 -0
  12. model-00009-of-00051.safetensors +3 -0
  13. model-00010-of-00051.safetensors +3 -0
  14. model-00011-of-00051.safetensors +3 -0
  15. model-00012-of-00051.safetensors +3 -0
  16. model-00013-of-00051.safetensors +3 -0
  17. model-00014-of-00051.safetensors +3 -0
  18. model-00015-of-00051.safetensors +3 -0
  19. model-00016-of-00051.safetensors +3 -0
  20. model-00017-of-00051.safetensors +3 -0
  21. model-00018-of-00051.safetensors +3 -0
  22. model-00019-of-00051.safetensors +3 -0
  23. model-00020-of-00051.safetensors +3 -0
  24. model-00021-of-00051.safetensors +3 -0
  25. model-00022-of-00051.safetensors +3 -0
  26. model-00023-of-00051.safetensors +3 -0
  27. model-00024-of-00051.safetensors +3 -0
  28. model-00025-of-00051.safetensors +3 -0
  29. model-00026-of-00051.safetensors +3 -0
  30. model-00027-of-00051.safetensors +3 -0
  31. model-00028-of-00051.safetensors +3 -0
  32. model-00029-of-00051.safetensors +3 -0
  33. model-00030-of-00051.safetensors +3 -0
  34. model-00031-of-00051.safetensors +3 -0
  35. model-00032-of-00051.safetensors +3 -0
  36. model-00033-of-00051.safetensors +3 -0
  37. model-00034-of-00051.safetensors +3 -0
  38. model-00035-of-00051.safetensors +3 -0
  39. model-00036-of-00051.safetensors +3 -0
  40. model-00037-of-00051.safetensors +3 -0
  41. model-00038-of-00051.safetensors +3 -0
  42. model-00039-of-00051.safetensors +3 -0
  43. model-00040-of-00051.safetensors +3 -0
  44. model-00041-of-00051.safetensors +3 -0
  45. model-00042-of-00051.safetensors +3 -0
  46. model-00043-of-00051.safetensors +3 -0
  47. model-00044-of-00051.safetensors +3 -0
  48. model-00045-of-00051.safetensors +3 -0
  49. model-00046-of-00051.safetensors +3 -0
  50. model-00047-of-00051.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - mistralai/Codestral-22B-v0.1
4
+ - nvidia/Mistral-NeMo-Minitron-8B-Instruct
5
+ - mistralai/Mistral-Large-Instruct-2407
6
+ - mistralai/Mistral-Nemo-Base-2407
7
+ - mistralai/Mathstral-7B-v0.1
8
+ library_name: transformers
9
+ tags:
10
+ - mergekit
11
+ - merge
12
+
13
+ ---
14
+ # merge
15
+
16
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
17
+
18
+ ## Merge Details
19
+ ### Merge Method
20
+
21
+ This model was merged using the passthrough merge method.
22
+
23
+ ### Models Merged
24
+
25
+ The following models were included in the merge:
26
+ * [mistralai/Codestral-22B-v0.1](https://huggingface.co/mistralai/Codestral-22B-v0.1)
27
+ * [nvidia/Mistral-NeMo-Minitron-8B-Instruct](https://huggingface.co/nvidia/Mistral-NeMo-Minitron-8B-Instruct)
28
+ * [mistralai/Mistral-Large-Instruct-2407](https://huggingface.co/mistralai/Mistral-Large-Instruct-2407)
29
+ * [mistralai/Mistral-Nemo-Base-2407](https://huggingface.co/mistralai/Mistral-Nemo-Base-2407)
30
+ * [mistralai/Mathstral-7B-v0.1](https://huggingface.co/mistralai/Mathstral-7B-v0.1)
31
+
32
+ ### Configuration
33
+
34
+ The following YAML configuration was used to produce this model:
35
+
36
+ ```yaml
37
+ dtype: float16
38
+ merge_method: passthrough
39
+ slices:
40
+ - sources:
41
+ - layer_range: [0, 8]
42
+ model: mistralai/Mistral-Large-Instruct-2407
43
+ - sources:
44
+ - layer_range: [4, 12]
45
+ model: nvidia/Mistral-NeMo-Minitron-8B-Instruct
46
+ - sources:
47
+ - layer_range: [8, 16]
48
+ model: mistralai/Mistral-Nemo-Base-2407
49
+ - sources:
50
+ - layer_range: [12, 20]
51
+ model: mistralai/Codestral-22B-v0.1
52
+ - sources:
53
+ - layer_range: [16, 24]
54
+ model: mistralai/Mathstral-7B-v0.1
55
+
56
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "mistralai/Codestral-22B-v0.1",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 6144,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 16384,
14
+ "max_position_embeddings": 32768,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 48,
17
+ "num_hidden_layers": 40,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "float16",
24
+ "transformers_version": "4.46.2",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ dtype: float16
2
+ merge_method: passthrough
3
+ slices:
4
+ - sources:
5
+ - layer_range: [0, 8]
6
+ model: mistralai/Mistral-Large-Instruct-2407
7
+ - sources:
8
+ - layer_range: [4, 12]
9
+ model: nvidia/Mistral-NeMo-Minitron-8B-Instruct
10
+ - sources:
11
+ - layer_range: [8, 16]
12
+ model: mistralai/Mistral-Nemo-Base-2407
13
+ - sources:
14
+ - layer_range: [12, 20]
15
+ model: mistralai/Codestral-22B-v0.1
16
+ - sources:
17
+ - layer_range: [16, 24]
18
+ model: mistralai/Mathstral-7B-v0.1
model-00001-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c8d2bbe13a2b0867d821474c0fc762b2d20f19da6ce78128ba231c5e539f9f6f
3
+ size 268435584
model-00002-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:32e63a7779ef715a54c1cdb1567ff706ba81cf4e7d8712bf3df6de8fc4d4e3ca
3
+ size 805331192
model-00003-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e5febf2b1c1dd8cce1bcca40aaf29bc35c1ff118c121491636a5f979b1aeec5
3
+ size 704643224
model-00004-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84d8ed92f4ff7b10bb2424bb1845177f464a8c7e395e45280ff0afb6405c5ed4
3
+ size 704643224
model-00005-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2661636ef58a5f18d097375908238bb9db1d6a4ff5d799bc17299920f960f07c
3
+ size 729833848
model-00006-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:250a1e92daae5d34419b2a6e1486c643998326a4e05dbf79986b6662bbded147
3
+ size 629170664
model-00007-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99c8a7d3082421e592c9be4d1545e277111b6bc8c34f1d546ba3c95c13087ecb
3
+ size 704643224
model-00008-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17b0eb6aa3346357759240b697e0bb27b23cc1e5326f267c952038838b3eba1e
3
+ size 704643224
model-00009-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb2a04a197d57294fd8928adcae0a21bf7cea72a21c545ff97abe50ad69b5d49
3
+ size 729833848
model-00010-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ea2fc6c915ae709ef1c890e34f95be2e53be01ede3c4d2b6a26765516f6560f1
3
+ size 870337336
model-00011-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:130e198b8978277acd9d1c050048372b2ed670c0cc78b22aa602068b1f34eecf
3
+ size 912300064
model-00012-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dac13acfc1353f5ee4f7bd70cef2214e232c2839a3ec50ebb82c3a8663482dfb
3
+ size 872458160
model-00013-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:26c9a249ef6e3347fee63f1dce83542bc312e50d1df4d11401a8e0ab0c0c80f9
3
+ size 983588112
model-00014-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10cb08d8e1cd85d7683da924774520c6a97d353ef335689e20e0b5af7d56178f
3
+ size 836787336
model-00015-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:235a1f61bd30faab6ba6deae0b8af95c033198607ddcaf9911ef24663ce8063c
3
+ size 977319744
model-00016-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aed4f41072c3e3e8d19b329f5daa7f8e146b2a9cc97143911b593702ee6eb7d8
3
+ size 977319744
model-00017-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8e79117ad70024586fc4008e9c61b84a4cc815235ea89811663c86b03aedb7d1
3
+ size 897581656
model-00018-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af628563a1ab363836fbf80a0051da4d8dbd9687e3f7934cdc65de0c9ca4d91e
3
+ size 864071488
model-00019-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c7587f54481f0d013c32ab3d6395e507bc2b7999d37327daff0e85c65b91148c
3
+ size 897623872
model-00020-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a4bce9d19f9649482eeff19aa79e39c64034ce0668a8da14e7f51b12640fd9fc
3
+ size 968906000
model-00021-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e2522381ad473cbe163d7c13dd5e841f9cf29af7b6f9935e67bf7095256df88
3
+ size 885019896
model-00022-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:59a0d20dc8580b68db9c5d4ef45600eab8bbdd96878c3877bdf4fbaed5b7b475
3
+ size 897623872
model-00023-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d08c7323270f99ee5a865f9a3b4616524836af46101fe652ff34cf4019399054
3
+ size 780187224
model-00024-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:59fd8400f786c8e9c6d418985760fe2a73cfbe8f8c0413b6d3ab8f592fb5c1c5
3
+ size 704643224
model-00025-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:03ac324299df9a62fd8e4b9a16b7005fcd6f09ab8507ad338c054a85994a2deb
3
+ size 704643224
model-00026-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63e677bf2c1bf5ce5e0bf8b374a7f7fdd8ebd8069dd87b5b5cbc1b2a15a1d288
3
+ size 729833848
model-00027-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a06aeeae07a5a1af23ce789d896036d8b9f533a88f22e3ef589657c67d62869a
3
+ size 989873192
model-00028-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a483f4888bb3bd330d9bfaac1a4c7f3137f53d2cff95b6002ae747812cfcb266
3
+ size 947956192
model-00029-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a10d7cc3e37ef29f3099a71339bb57b3cdbd91d5c7c831896f3403048fd6c4b3
3
+ size 436241432
model-00030-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0a1864c9a5b18c1757c6b66fd3c94cbd60f8c80b9d6089de3f6ceea67b1ae672
3
+ size 704643224
model-00031-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a236cc03345e39aae925979d672c367147c48de2acfd840ab8c8d04da3dfffa6
3
+ size 704643224
model-00032-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7de8b1fb4b96f62c35b169e8a686b73d1f13f60962475768dffa9dc7bb784865
3
+ size 729833848
model-00033-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6dce8230c3c4e9502cc76ae982af237b92b3bbe0b6038d5ca81921826d4e695
3
+ size 723550920
model-00034-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0b282bae5fcd9ffa1d4b830faa4c05492f9ed5352946358cbfcc192622b29203
3
+ size 799015176
model-00035-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:13a97fe95c9bb92314ab7c5fd8b97770efe3d8a600c4b44e125a38cb570216c5
3
+ size 799015168
model-00036-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:86de010894f7fa553271f96ea0d45f0a2e4b71284b0e5854ef5e082fb2662d3f
3
+ size 771785432
model-00037-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4508cf45b7e9c9726dc6b2b38a584f5e669aba38afc02153a2675dddeda26cb6
3
+ size 765494184
model-00038-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:23bcefa82701e95d98368c12b2ff03526227e9881f9898d34d45ffcaee57f9de
3
+ size 799015176
model-00039-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2ea6cb9b2d321da4b11fb18bf4c7cb064e0dafbf6bf676d26928c32346fa35b2
3
+ size 799015168
model-00040-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db2aa5d8ee7e05560e4858381aeb175f9ec8a4806ba6db786958e80b69d8237f
3
+ size 771785432
model-00041-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4f59d619e9d4ab9a6385c5dddafea4e80ba544c2c41f6a4b3217a23a9a5900f
3
+ size 765494176
model-00042-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e9b326404bc78ab0dc3cef868ac069bb478894b468ad79a1cc5dfd262ed413a
3
+ size 799015168
model-00043-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6da8371362818c8032b71af502070ab9661f9a374dd5af581983eaef7125bfee
3
+ size 799015168
model-00044-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b1e4f56861ea30804f1723c03ffd70b37997004f24bee0e61f232a498cc19609
3
+ size 771785416
model-00045-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d7864ae0b43a380a5549b6fadf52e592396c2bc5f70096d40b944d72edcd26c5
3
+ size 765494184
model-00046-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6b21e422053826188aa7969de308793373be1db571f1910645c3707232c16c38
3
+ size 799015168
model-00047-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:446b33c02e85cb2270ba82c206f690b7cd0aa5d410ee418a8da2b293a8768394
3
+ size 799015168