Hugging Face's logo

mpasila
/
Mistral-7B-Holodeck-1-Instruct-32k-exl2-4bpw

Text Generation
Transformers
mistral
mergekit
Merge
Inference Endpoints
text-generation-inference
Model card Files Community
mpasila commited on
Commit
7eb96be
•
1 Parent(s): 4984bc8

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +51 -0
  2. cal_data.safetensors +3 -0
  3. config.json +26 -0
  4. hidden_states.safetensors +3 -0
  5. job_new.json +0 -0
  6. measurement.json +0 -0
  7. out_tensor/lm_head.safetensors +3 -0
  8. out_tensor/model.layers.0.mlp.down_proj.safetensors +3 -0
  9. out_tensor/model.layers.0.mlp.gate_proj.safetensors +3 -0
  10. out_tensor/model.layers.0.mlp.up_proj.safetensors +3 -0
  11. out_tensor/model.layers.0.self_attn.k_proj.safetensors +3 -0
  12. out_tensor/model.layers.0.self_attn.o_proj.safetensors +3 -0
  13. out_tensor/model.layers.0.self_attn.q_proj.safetensors +3 -0
  14. out_tensor/model.layers.0.self_attn.v_proj.safetensors +3 -0
  15. out_tensor/model.layers.1.mlp.down_proj.safetensors +3 -0
  16. out_tensor/model.layers.1.mlp.gate_proj.safetensors +3 -0
  17. out_tensor/model.layers.1.mlp.up_proj.safetensors +3 -0
  18. out_tensor/model.layers.1.self_attn.k_proj.safetensors +3 -0
  19. out_tensor/model.layers.1.self_attn.o_proj.safetensors +3 -0
  20. out_tensor/model.layers.1.self_attn.q_proj.safetensors +3 -0
  21. out_tensor/model.layers.1.self_attn.v_proj.safetensors +3 -0
  22. out_tensor/model.layers.10.mlp.down_proj.safetensors +3 -0
  23. out_tensor/model.layers.10.mlp.gate_proj.safetensors +3 -0
  24. out_tensor/model.layers.10.mlp.up_proj.safetensors +3 -0
  25. out_tensor/model.layers.10.self_attn.k_proj.safetensors +3 -0
  26. out_tensor/model.layers.10.self_attn.o_proj.safetensors +3 -0
  27. out_tensor/model.layers.10.self_attn.q_proj.safetensors +3 -0
  28. out_tensor/model.layers.10.self_attn.v_proj.safetensors +3 -0
  29. out_tensor/model.layers.11.mlp.down_proj.safetensors +3 -0
  30. out_tensor/model.layers.11.mlp.gate_proj.safetensors +3 -0
  31. out_tensor/model.layers.11.mlp.up_proj.safetensors +3 -0
  32. out_tensor/model.layers.11.self_attn.k_proj.safetensors +3 -0
  33. out_tensor/model.layers.11.self_attn.o_proj.safetensors +3 -0
  34. out_tensor/model.layers.11.self_attn.q_proj.safetensors +3 -0
  35. out_tensor/model.layers.11.self_attn.v_proj.safetensors +3 -0
  36. out_tensor/model.layers.12.mlp.down_proj.safetensors +3 -0
  37. out_tensor/model.layers.12.mlp.gate_proj.safetensors +3 -0
  38. out_tensor/model.layers.12.mlp.up_proj.safetensors +3 -0
  39. out_tensor/model.layers.12.self_attn.k_proj.safetensors +3 -0
  40. out_tensor/model.layers.12.self_attn.o_proj.safetensors +3 -0
  41. out_tensor/model.layers.12.self_attn.q_proj.safetensors +3 -0
  42. out_tensor/model.layers.12.self_attn.v_proj.safetensors +3 -0
  43. out_tensor/model.layers.13.mlp.down_proj.safetensors +3 -0
  44. out_tensor/model.layers.13.mlp.gate_proj.safetensors +3 -0
  45. out_tensor/model.layers.13.mlp.up_proj.safetensors +3 -0
  46. out_tensor/model.layers.13.self_attn.k_proj.safetensors +3 -0
  47. out_tensor/model.layers.13.self_attn.o_proj.safetensors +3 -0
  48. out_tensor/model.layers.13.self_attn.q_proj.safetensors +3 -0
  49. out_tensor/model.layers.13.self_attn.v_proj.safetensors +3 -0
  50. out_tensor/model.layers.14.mlp.down_proj.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - alpindale/Mistral-7B-v0.2-hf
4
+ - mistralai/Mistral-7B-Instruct-v0.2
5
+ - KoboldAI/Mistral-7B-Holodeck-1
6
+ library_name: transformers
7
+ tags:
8
+ - mergekit
9
+ - merge
10
+
11
+ ---
12
+ # Mistral-7B-Holodeck-1-Instruct-32k
13
+
14
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
+
16
+ Merge script copied from this [ichigoberry/pandafish-2-7b-32k](https://huggingface.co/ichigoberry/pandafish-2-7b-32k).
17
+
18
+ ## Merge Details
19
+ ### Merge Method
20
+
21
+ This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [alpindale/Mistral-7B-v0.2-hf](https://huggingface.co/alpindale/Mistral-7B-v0.2-hf) as a base.
22
+
23
+ ### Models Merged
24
+
25
+ The following models were included in the merge:
26
+ * [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)
27
+ * [KoboldAI/Mistral-7B-Holodeck-1](https://huggingface.co/KoboldAI/Mistral-7B-Holodeck-1)
28
+
29
+ ### Configuration
30
+
31
+ The following YAML configuration was used to produce this model:
32
+
33
+ ```yaml
34
+ models:
35
+ - model: alpindale/Mistral-7B-v0.2-hf
36
+ # No parameters necessary for base model
37
+ - model: mistralai/Mistral-7B-Instruct-v0.2
38
+ parameters:
39
+ density: 0.53
40
+ weight: 0.4
41
+ - model: KoboldAI/Mistral-7B-Holodeck-1
42
+ parameters:
43
+ density: 0.53
44
+ weight: 0.4
45
+ merge_method: dare_ties
46
+ base_model: alpindale/Mistral-7B-v0.2-hf
47
+ parameters:
48
+ int8_mask: true
49
+ dtype: bfloat16
50
+
51
+ ```
cal_data.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:754a0ca18db223cbcdd94ecaa1d916a8945f3896fc8823e39d9b542f4c322bad
3
+ size 1884248
config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "alpindale/Mistral-7B-v0.2-hf",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "hidden_act": "silu",
10
+ "hidden_size": 4096,
11
+ "initializer_range": 0.02,
12
+ "intermediate_size": 14336,
13
+ "max_position_embeddings": 32768,
14
+ "model_type": "mistral",
15
+ "num_attention_heads": 32,
16
+ "num_hidden_layers": 32,
17
+ "num_key_value_heads": 8,
18
+ "rms_norm_eps": 1e-05,
19
+ "rope_theta": 1000000.0,
20
+ "sliding_window": null,
21
+ "tie_word_embeddings": false,
22
+ "torch_dtype": "bfloat16",
23
+ "transformers_version": "4.38.2",
24
+ "use_cache": true,
25
+ "vocab_size": 32000
26
+ }
hidden_states.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:694886cc776ae2d44f96e768d890dcf88ad4897eaa119d5c9199cbd47cdedfd1
3
+ size 1929389952
job_new.json ADDED
The diff for this file is too large to render. See raw diff
 
measurement.json ADDED
The diff for this file is too large to render. See raw diff
 
out_tensor/lm_head.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4886c177666a6f6bad8292292c6af403e09ac22d1ff1ef8878116fd2397369e4
3
+ size 103953008
out_tensor/model.layers.0.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da24b421344231d86e50cf070efeaa8c3a4fb78ba420e924d32d747e3b01ea65
3
+ size 39546008
out_tensor/model.layers.0.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:405163f3dd69724e160d816cfebafc9bf930363603fa7301b640ee6f60303661
3
+ size 38380824
out_tensor/model.layers.0.mlp.up_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e627c38fbcf999d26797d6d276646542d96d26f0ebee7ff24721d7c55eddf3e8
3
+ size 39470352
out_tensor/model.layers.0.self_attn.k_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:873a47e3996de93fe7c81e73e83a165d0b0d342f481b922db20768edee51898e
3
+ size 2757920
out_tensor/model.layers.0.self_attn.o_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:71686f08025fad98d07d5b0ce2fb8166523cc42f54f8b906b846946ca251b312
3
+ size 10978600
out_tensor/model.layers.0.self_attn.q_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cafbd2b3e06315cc1dcbeeac3a8bc9597694cadc9696795e9cea7666bb4174cb
3
+ size 10978600
out_tensor/model.layers.0.self_attn.v_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d417d7e18ff3c7945466c5640a591f929e64266272c569e6fed01b60782bce36
3
+ size 3228960
out_tensor/model.layers.1.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3e8a0cbdb145a10bad8948f4aeff5a0f13787b0f1a0b0b14575496d3567fd563
3
+ size 25619608
out_tensor/model.layers.1.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b43a248f9cfed4473c12beb1b60ad5b71d715548e3616d43730ac4b99d5947df
3
+ size 23700760
out_tensor/model.layers.1.mlp.up_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e6199fdcbb7e226d1cc6e4b0ca2f31f1409c49d94fa9b87b2206243c1669440d
3
+ size 24790288
out_tensor/model.layers.1.self_attn.k_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:07a923d76f4ec9e864eaf6e4e94e890d2c30b5ea075e5576c7e0aec441f3851c
3
+ size 3228960
out_tensor/model.layers.1.self_attn.o_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb64c7bbf040529529e6d3a271294b28d2234d4d7e0847f15d5871c9767c57ec
3
+ size 12862760
out_tensor/model.layers.1.self_attn.q_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fcd9bd2515c9f245fc8ce9820afeb42311b148ac2e289ea03127ef8441025810
3
+ size 12862760
out_tensor/model.layers.1.self_attn.v_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:254b6b35ccbf857ae479146758d30756825f7688181b572078302bcdc680ead0
3
+ size 4277536
out_tensor/model.layers.10.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aee2e3a0e525b9b6d22016f329c9b6535aaf2aec14e81e43b18d84aaa2a26719
3
+ size 25619616
out_tensor/model.layers.10.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:66d32f4311aafef387b303593df750802e2254fe724a0068784c942ba3599d5b
3
+ size 23700768
out_tensor/model.layers.10.mlp.up_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8ca9f29d9281eba5f7e8843a758876bf7047093be5257b4770f8808bb444b68b
3
+ size 24790288
out_tensor/model.layers.10.self_attn.k_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a806f34fd12153625fdd78a9b2cef8b64bfe13ea88dbdd38dda1b0e58a87884
3
+ size 1671904
out_tensor/model.layers.10.self_attn.o_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:092768796f7d0a962b65b3f368b9884aa78d05c8b143bc90e06efd36be6730cf
3
+ size 6636256
out_tensor/model.layers.10.self_attn.q_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f92d316e9b73ae8889cc5776ba96b1b27f1c8762cb369cb6de33dcef03d921a
3
+ size 6636256
out_tensor/model.layers.10.self_attn.v_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:530e945dac3383773b9f61e7b7b6cf9431eb75138ac635ec12beb5ec64b363e9
3
+ size 1680288
out_tensor/model.layers.11.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:82d427c0c8f9a1c105aee746cdb276e707d0d493ff8398b7e57eb48ee028f55e
3
+ size 31192364
out_tensor/model.layers.11.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e6fc15aaf0a82dde963f3302db42d85199a99992a8a8e55b69898d0c0f0c8fe
3
+ size 23700768
out_tensor/model.layers.11.mlp.up_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:18ea490a6f7234a88db6196a15e6f6735093c5f40c913afd51d83efc99a7ef75
3
+ size 24790288
out_tensor/model.layers.11.self_attn.k_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc57dc06380bf8ebef85de68cfd06db1198e2187d376f71268b68b3e40ad5def
3
+ size 1671904
out_tensor/model.layers.11.self_attn.o_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:757eb522d0c922c71983b1a73b8015bf8687e8697884f7ce0b56f9c15aa41037
3
+ size 6636256
out_tensor/model.layers.11.self_attn.q_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f7786ab794cde2b7f72ae0b2f8c6b88ccdd323eeaec28336fd4f22d7949974f5
3
+ size 6636256
out_tensor/model.layers.11.self_attn.v_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2bb928ad6a8d42a3550765237bc84d02346b50bb2d5adea0acf846e258a29fc
3
+ size 1680288
out_tensor/model.layers.12.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ba5ee6e65713605577daf13b529da12d5d2a1a00c506158957edd750ca6b7a4a
3
+ size 31845536
out_tensor/model.layers.12.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b506622bb353d951b40b62275efde8a8d0def4f254b961cdf0b3878b6c334ae3
3
+ size 30295328
out_tensor/model.layers.12.mlp.up_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b425f82c5ac9cb62ae0b3a76affbda739043f0732309af80e5f95fe205aedc81
3
+ size 30295312
out_tensor/model.layers.12.self_attn.k_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d05082d5d3f08848e1554b0b827ce481af1fbd286ae4a095c9c05b9e5e2935a5
3
+ size 1671904
out_tensor/model.layers.12.self_attn.o_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cca18e1d91efdba422f622f56357bca604f10a5ee0e51f567892124b29731ec6
3
+ size 6636256
out_tensor/model.layers.12.self_attn.q_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36492fb335233ef2c39a43dc82a38679f1f1a61fa880baf2b83bdc8a6f6cf9f7
3
+ size 6636256
out_tensor/model.layers.12.self_attn.v_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bbc3449442f4ecb56a9d479f4dd7b0b1f1a401cb3800514199f9b6045c08ed18
3
+ size 1680288
out_tensor/model.layers.13.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d43c5244fbdacdeb69cf027f9c10483017ea7f4a8f5354110c75d0c959bebc8
3
+ size 31845536
out_tensor/model.layers.13.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f918644d1d827e61213f5550c35bd96d2e59ba5ad35d72f4de4cb3d3372b3be
3
+ size 30295328
out_tensor/model.layers.13.mlp.up_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a80807171c6cbe31efa0ccdbca568164c042ac6610a04cb85117b782c69195af
3
+ size 30295312
out_tensor/model.layers.13.self_attn.k_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc155c9813f59228c07f91fc07d0fa8cf2b5e02c9c8206b38a2f327ef8a404eb
3
+ size 1671904
out_tensor/model.layers.13.self_attn.o_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4d2797ca38cb857407569107c33b00a0b3dd080ab7d7eb4c0c78bde537f519fe
3
+ size 6636256
out_tensor/model.layers.13.self_attn.q_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:485cf58b2445364290c4b6145674f215d7d7785bb178593baa931d2002e6bfce
3
+ size 6636256
out_tensor/model.layers.13.self_attn.v_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:368775c4c72f7a239e3e5c8bf22d4076c1fd415b2b202db36e78318b6497979a
3
+ size 1680288
out_tensor/model.layers.14.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:43908ec528c0fc04225c1fc507a0a32fa967df2037c6d835c9d758bbea04fe58
3
+ size 31845536