Hugging Face's logo

vicgalle
/
Merge-Mixtral-Prometheus-8x7B

Text Generation
Transformers
Safetensors
mixtral
Merge
mergekit
lazymergekit
prometheus-eval/prometheus-8x7b-v2.0
mistralai/Mixtral-8x7B-Instruct-v0.1
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
vicgalle commited on
Commit
a4dbafe
•
1 Parent(s): 91689cc

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +56 -0
  2. config.json +30 -0
  3. mergekit_config.yml +10 -0
  4. model-00001-of-00048.safetensors +3 -0
  5. model-00002-of-00048.safetensors +3 -0
  6. model-00003-of-00048.safetensors +3 -0
  7. model-00004-of-00048.safetensors +3 -0
  8. model-00005-of-00048.safetensors +3 -0
  9. model-00006-of-00048.safetensors +3 -0
  10. model-00007-of-00048.safetensors +3 -0
  11. model-00008-of-00048.safetensors +3 -0
  12. model-00009-of-00048.safetensors +3 -0
  13. model-00010-of-00048.safetensors +3 -0
  14. model-00011-of-00048.safetensors +3 -0
  15. model-00012-of-00048.safetensors +3 -0
  16. model-00013-of-00048.safetensors +3 -0
  17. model-00014-of-00048.safetensors +3 -0
  18. model-00015-of-00048.safetensors +3 -0
  19. model-00016-of-00048.safetensors +3 -0
  20. model-00017-of-00048.safetensors +3 -0
  21. model-00018-of-00048.safetensors +3 -0
  22. model-00019-of-00048.safetensors +3 -0
  23. model-00020-of-00048.safetensors +3 -0
  24. model-00021-of-00048.safetensors +3 -0
  25. model-00022-of-00048.safetensors +3 -0
  26. model-00023-of-00048.safetensors +3 -0
  27. model-00024-of-00048.safetensors +3 -0
  28. model-00025-of-00048.safetensors +3 -0
  29. model-00026-of-00048.safetensors +3 -0
  30. model-00027-of-00048.safetensors +3 -0
  31. model-00028-of-00048.safetensors +3 -0
  32. model-00029-of-00048.safetensors +3 -0
  33. model-00030-of-00048.safetensors +3 -0
  34. model-00031-of-00048.safetensors +3 -0
  35. model-00032-of-00048.safetensors +3 -0
  36. model-00033-of-00048.safetensors +3 -0
  37. model-00034-of-00048.safetensors +3 -0
  38. model-00035-of-00048.safetensors +3 -0
  39. model-00036-of-00048.safetensors +3 -0
  40. model-00037-of-00048.safetensors +3 -0
  41. model-00038-of-00048.safetensors +3 -0
  42. model-00039-of-00048.safetensors +3 -0
  43. model-00040-of-00048.safetensors +3 -0
  44. model-00041-of-00048.safetensors +3 -0
  45. model-00042-of-00048.safetensors +3 -0
  46. model-00043-of-00048.safetensors +3 -0
  47. model-00044-of-00048.safetensors +3 -0
  48. model-00045-of-00048.safetensors +3 -0
  49. model-00046-of-00048.safetensors +3 -0
  50. model-00047-of-00048.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - merge
4
+ - mergekit
5
+ - lazymergekit
6
+ - prometheus-eval/prometheus-8x7b-v2.0
7
+ - mistralai/Mixtral-8x7B-Instruct-v0.1
8
+ base_model:
9
+ - prometheus-eval/prometheus-8x7b-v2.0
10
+ - mistralai/Mixtral-8x7B-Instruct-v0.1
11
+ ---
12
+
13
+ # test-merge-3
14
+
15
+ test-merge-3 is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
16
+ * [prometheus-eval/prometheus-8x7b-v2.0](https://huggingface.co/prometheus-eval/prometheus-8x7b-v2.0)
17
+ * [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)
18
+
19
+ ## 🧩 Configuration
20
+
21
+ ```yaml
22
+ models:
23
+ - model: prometheus-eval/prometheus-8x7b-v2.0
24
+ parameters:
25
+ weight: 1.0
26
+ - model: mistralai/Mixtral-8x7B-Instruct-v0.1
27
+ parameters:
28
+ weight: 1.0
29
+ merge_method: linear
30
+ dtype: bfloat16
31
+ ```
32
+
33
+ ## 💻 Usage
34
+
35
+ ```python
36
+ !pip install -qU transformers accelerate
37
+
38
+ from transformers import AutoTokenizer
39
+ import transformers
40
+ import torch
41
+
42
+ model = "vicgalle/test-merge-3"
43
+ messages = [{"role": "user", "content": "What is a large language model?"}]
44
+
45
+ tokenizer = AutoTokenizer.from_pretrained(model)
46
+ prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
47
+ pipeline = transformers.pipeline(
48
+ "text-generation",
49
+ model=model,
50
+ torch_dtype=torch.float16,
51
+ device_map="auto",
52
+ )
53
+
54
+ outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
55
+ print(outputs[0]["generated_text"])
56
+ ```
config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "mistralai/Mixtral-8x7B-Instruct-v0.1",
3
+ "architectures": [
4
+ "MixtralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "hidden_act": "silu",
10
+ "hidden_size": 4096,
11
+ "initializer_range": 0.02,
12
+ "intermediate_size": 14336,
13
+ "max_position_embeddings": 32768,
14
+ "model_type": "mixtral",
15
+ "num_attention_heads": 32,
16
+ "num_experts_per_tok": 2,
17
+ "num_hidden_layers": 32,
18
+ "num_key_value_heads": 8,
19
+ "num_local_experts": 8,
20
+ "output_router_logits": false,
21
+ "rms_norm_eps": 1e-05,
22
+ "rope_theta": 1000000.0,
23
+ "router_aux_loss_coef": 0.02,
24
+ "sliding_window": null,
25
+ "tie_word_embeddings": false,
26
+ "torch_dtype": "bfloat16",
27
+ "transformers_version": "4.39.3",
28
+ "use_cache": true,
29
+ "vocab_size": 32000
30
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ models:
3
+ - model: prometheus-eval/prometheus-8x7b-v2.0
4
+ parameters:
5
+ weight: 1.0
6
+ - model: mistralai/Mixtral-8x7B-Instruct-v0.1
7
+ parameters:
8
+ weight: 1.0
9
+ merge_method: linear
10
+ dtype: bfloat16
model-00001-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4d71879f3fb6e961a55ad3b2476b8079e858bff9965674c4c9982aece8861972
3
+ size 1933575936
model-00002-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9832b157fb137aa627a65931980c8162bb7be98b6dcae90909c171396682a82c
3
+ size 1963019128
model-00003-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6636f541528828ee64fe3e6f72363f743563ed4d5570f79797aedcd13abf2ca3
3
+ size 1996490952
model-00004-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8253efaeb4f4d9062ce504167cffed0d73d3dd6208b24b4cc3a5062f43ba720b
3
+ size 1963019128
model-00005-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4dd511c4dbe65b315b60827cedc421fd1eff18483a3c50a5c4e469b6e880ed15
3
+ size 1963019144
model-00006-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:505a680242fbaad74b3825450b021d32c094dcdef1d7ed6e58041d6add488f5c
3
+ size 1996490968
model-00007-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd80eedb8509b15650abd813797178df1f85a19729d1ebe652be3db74a219682
3
+ size 1963019136
model-00008-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0977c1828af8a10fc743811e3f72070e2b0ad0ef9a8e67845ca6ea2c2d25be68
3
+ size 1963019144
model-00009-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d36c4664f2789cdf0005e55060c425e790b513e7da2272fd5ce3d01e2d0e6ed
3
+ size 1996490968
model-00010-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:74ad827fc4dedbf6e1ad66c8791fed70c5d50ab9dc82c847592bd444bc8daf5a
3
+ size 1963019136
model-00011-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b741e1e4ae7886307d65c9e3f0b6dae383add844a87fdce5eabe5c394b69dc66
3
+ size 1963019144
model-00012-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f38b1fc554f42c86b3a1ee89745fb7422ed91be1d537e175b880b09cb9ab99e
3
+ size 1996573240
model-00013-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd3f1d33224de5bece6e9a205a9fa46ad4b630ed91eeaa9bada216efdf0afd67
3
+ size 1962936856
model-00014-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:13d924b3d1b17df451c18b9232425067f9deee883ed43021b2e9d593461ef95c
3
+ size 1963019136
model-00015-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d922ec99e5bf52da771d26e49891e0b48fc16c7387d823342817be2f30ae996
3
+ size 1963019144
model-00016-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b7ee073bd34425665e33c3d323a31590571540d97273cc5ca16c409c54a2fc36
3
+ size 1996490968
model-00017-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b9144ac774abae9cc5450e224e866062f1392bd2faad13aa55b538719944d3f
3
+ size 1963019136
model-00018-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8de9dac93b5c66174274808c63e0ae3d51ffb606fdf2457ede84b8ba29902ca1
3
+ size 1963019144
model-00019-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bb05dc007d6969177e60c4fd33ad1cd02f71e077c35d15a857370c0277e0d890
3
+ size 1996490952
model-00020-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5f37d0bc5cec6025ce69ddfbd23fc16ddd36640c4ace500b7cbc9ae9c23f81c6
3
+ size 1963019120
model-00021-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:544fe19be4150da02fcc49550cbb5a4d386f82e11bedcb5f65a6f9133d64c467
3
+ size 1963019144
model-00022-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01bfc20ffa36ad266aa1ca78cb7379c44224b505377003cb2bcfde6f59c7ed29
3
+ size 1996490968
model-00023-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:614a7ee884e44aba91640b7b68132c9caa857a120967498b50a12a4dcfbe1eca
3
+ size 1963019136
model-00024-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6a3cf16c95e08236f6b3c98c76ba30c272e9e49722bb5d887c8a66e20f98152
3
+ size 1963019144
model-00025-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f297431a9d0bb5e917083925dc94ef187f9afe3756ff38f974915000a4653bd
3
+ size 1996490968
model-00026-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2a8bf8b3c6340346456ecddfc7bb2da901d288b7ef7d37785492112ab54a1794
3
+ size 1963019136
model-00027-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b514d0dd5b529c7a0935f329849325a777f80a996de78d05cf22aa908ecac40f
3
+ size 1963019144
model-00028-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:15e990bfa5cbee19b1662255e327dba07401fb6d40eda7becdbc461369f4cc4a
3
+ size 1996490968
model-00029-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10b8528c01c62efc211c92f2fa0c6b74d6f7e07eb513c7761eaf622d04fe5104
3
+ size 1963019136
model-00030-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ddc8e0d12eed467fac67cb46686cb86ad7cad616636da33ebb07dc7b0662025a
3
+ size 1963019144
model-00031-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4a16245f01f0063bbd8d83861a17d528683713980f6d53e23b526592810e6cc
3
+ size 1996490968
model-00032-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a67b74d38b01015e19a08c4f6f4509fc0a6e1a2a42bbf713c11214b9b191837b
3
+ size 1963019136
model-00033-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e80b0fc461b8b565e9bf17a999f3774093652664577f8778961460e72ab4e1d
3
+ size 1963019144
model-00034-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1344a4a5c2001b28652e2439d87895c5ee4c5f56ce85eb84f9128878bcefd43
3
+ size 1996490968
model-00035-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cf5802a67abaa8fd542e82ae5aa2c4b0372bf1bf578426a7e2ec6779edaec097
3
+ size 1963019120
model-00036-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ad0a0d79b4814573aa8bf488bb4503ece58286eb0d9279b9284116c03ad17e7b
3
+ size 1963019128
model-00037-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17485077e4d9f977f57a4583acbce7af7e904613723e42c0c6d54727a263450b
3
+ size 1996573240
model-00038-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:341031042703d3cf354826802dc391b853506cff930a0ca89dcb17355c295abf
3
+ size 1962936856
model-00039-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:06c59049ec1758dfe3209a9645a976ac9983382e726b4d60b1234c54bf27ff0d
3
+ size 1963019128
model-00040-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5f9f668b566a1268928f974bc25e4f6e1ef50e01a59a4995b749a3d0782ab352
3
+ size 1963019128
model-00041-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee984384e94a34d806d60dc30c3c4eb3cf77f5312ca66d7bc8957a80ba6b2465
3
+ size 1996490952
model-00042-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3db541dda4e22ab986e21e7b2740a02134ad1f249bb965257c717f917a9d94be
3
+ size 1963019112
model-00043-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a3a98b30e184b39afd2a4e5809d2fce86c8198de05ce28494d1e545f07b9ec92
3
+ size 1963019128
model-00044-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:96c7e791982920dd8f5a87b8a0964453fe813d9171202e77924811a886f15b54
3
+ size 1996490952
model-00045-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f28160c27502bf9bff1052c65145ef13f7881ee1a8770b7d5f3133fc8b8c1851
3
+ size 1963019112
model-00046-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aed47bc2c07cf82e520209a2542ff6d36c0322cd9f49fc15207a6c03e61d833d
3
+ size 1963019128
model-00047-of-00048.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8723825d90421bef905e01a70065a813ade69932248b77dae871df9816dcd50
3
+ size 1996490952