Hugging Face's logo

saucam
/
aqua-qwen-0.1-110B

Text Generation
Transformers
Safetensors
English
qwen2
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
saucam commited on
Commit
c15e6a8
•
1 Parent(s): cb81044

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +48 -0
  2. added_tokens.json +5 -0
  3. config.json +27 -0
  4. mergekit_config.yml +18 -0
  5. merges.txt +0 -0
  6. model-00001-of-00049.safetensors +3 -0
  7. model-00002-of-00049.safetensors +3 -0
  8. model-00003-of-00049.safetensors +3 -0
  9. model-00004-of-00049.safetensors +3 -0
  10. model-00005-of-00049.safetensors +3 -0
  11. model-00006-of-00049.safetensors +3 -0
  12. model-00007-of-00049.safetensors +3 -0
  13. model-00008-of-00049.safetensors +3 -0
  14. model-00009-of-00049.safetensors +3 -0
  15. model-00010-of-00049.safetensors +3 -0
  16. model-00011-of-00049.safetensors +3 -0
  17. model-00012-of-00049.safetensors +3 -0
  18. model-00013-of-00049.safetensors +3 -0
  19. model-00014-of-00049.safetensors +3 -0
  20. model-00015-of-00049.safetensors +3 -0
  21. model-00016-of-00049.safetensors +3 -0
  22. model-00017-of-00049.safetensors +3 -0
  23. model-00018-of-00049.safetensors +3 -0
  24. model-00019-of-00049.safetensors +3 -0
  25. model-00020-of-00049.safetensors +3 -0
  26. model-00021-of-00049.safetensors +3 -0
  27. model-00022-of-00049.safetensors +3 -0
  28. model-00023-of-00049.safetensors +3 -0
  29. model-00024-of-00049.safetensors +3 -0
  30. model-00025-of-00049.safetensors +3 -0
  31. model-00026-of-00049.safetensors +3 -0
  32. model-00027-of-00049.safetensors +3 -0
  33. model-00028-of-00049.safetensors +3 -0
  34. model-00029-of-00049.safetensors +3 -0
  35. model-00030-of-00049.safetensors +3 -0
  36. model-00031-of-00049.safetensors +3 -0
  37. model-00032-of-00049.safetensors +3 -0
  38. model-00033-of-00049.safetensors +3 -0
  39. model-00034-of-00049.safetensors +3 -0
  40. model-00035-of-00049.safetensors +3 -0
  41. model-00036-of-00049.safetensors +3 -0
  42. model-00037-of-00049.safetensors +3 -0
  43. model-00038-of-00049.safetensors +3 -0
  44. model-00039-of-00049.safetensors +3 -0
  45. model-00040-of-00049.safetensors +3 -0
  46. model-00041-of-00049.safetensors +3 -0
  47. model-00042-of-00049.safetensors +3 -0
  48. model-00043-of-00049.safetensors +3 -0
  49. model-00044-of-00049.safetensors +3 -0
  50. model-00045-of-00049.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - cognitivecomputations/dolphin-2.9.1-qwen-110b
4
+ - Qwen/Qwen1.5-110B-Chat
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+
10
+ ---
11
+ # merge
12
+
13
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
+
15
+ ## Merge Details
16
+ ### Merge Method
17
+
18
+ This model was merged using the linear [DARE](https://arxiv.org/abs/2311.03099) merge method using [cognitivecomputations/dolphin-2.9.1-qwen-110b](https://huggingface.co/cognitivecomputations/dolphin-2.9.1-qwen-110b) as a base.
19
+
20
+ ### Models Merged
21
+
22
+ The following models were included in the merge:
23
+ * [Qwen/Qwen1.5-110B-Chat](https://huggingface.co/Qwen/Qwen1.5-110B-Chat)
24
+
25
+ ### Configuration
26
+
27
+ The following YAML configuration was used to produce this model:
28
+
29
+ ```yaml
30
+ name: aqua-qwen-0.1-110B
31
+ base_model:
32
+ model:
33
+ path: cognitivecomputations/dolphin-2.9.1-qwen-110b
34
+ dtype: bfloat16
35
+ merge_method: dare_linear
36
+ parameters:
37
+ normalize: 1.0
38
+ slices:
39
+ - sources:
40
+ - model: cognitivecomputations/dolphin-2.9.1-qwen-110b
41
+ layer_range: [0, 80]
42
+ parameters:
43
+ weight: 0.6
44
+ - model: Qwen/Qwen1.5-110B-Chat
45
+ layer_range: [0, 80]
46
+ parameters:
47
+ weight: 0.4
48
+ ```
added_tokens.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "<|endoftext|>": 151643,
3
+ "<|im_end|>": 151645,
4
+ "<|im_start|>": 151644
5
+ }
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "cognitivecomputations/dolphin-2.9.1-qwen-110b",
3
+ "architectures": [
4
+ "Qwen2ForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "eos_token_id": 151645,
8
+ "hidden_act": "silu",
9
+ "hidden_size": 8192,
10
+ "initializer_range": 0.02,
11
+ "intermediate_size": 49152,
12
+ "max_position_embeddings": 32768,
13
+ "max_window_layers": 28,
14
+ "model_type": "qwen2",
15
+ "num_attention_heads": 64,
16
+ "num_hidden_layers": 80,
17
+ "num_key_value_heads": 8,
18
+ "rms_norm_eps": 1e-06,
19
+ "rope_theta": 1000000.0,
20
+ "sliding_window": 32768,
21
+ "tie_word_embeddings": false,
22
+ "torch_dtype": "bfloat16",
23
+ "transformers_version": "4.40.2",
24
+ "use_cache": false,
25
+ "use_sliding_window": false,
26
+ "vocab_size": 152064
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: aqua-qwen-0.1-110B
2
+ base_model:
3
+ model:
4
+ path: cognitivecomputations/dolphin-2.9.1-qwen-110b
5
+ dtype: bfloat16
6
+ merge_method: dare_linear
7
+ parameters:
8
+ normalize: 1.0
9
+ slices:
10
+ - sources:
11
+ - model: cognitivecomputations/dolphin-2.9.1-qwen-110b
12
+ layer_range: [0, 80]
13
+ parameters:
14
+ weight: 0.6
15
+ - model: Qwen/Qwen1.5-110B-Chat
16
+ layer_range: [0, 80]
17
+ parameters:
18
+ weight: 0.4
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model-00001-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a749f0b541d1de384587f6009b69c7255d022603c36d978e1699014250fe5c9c
3
+ size 4982849880
model-00002-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77f645fa1a422e4f92ce24867e55d27d3b62c268e9755393f01f7536aba01d1d
3
+ size 4328576608
model-00003-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:244bf9e8e6d8672e801b9898c2f7c9121da1407dd71e945399b40baa51b2900d
3
+ size 4630620776
model-00004-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a87bda019e8d7a446001f36e555a8f8ede75df193ade801032b24ebf61b897c9
3
+ size 4630620800
model-00005-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4804f87c7d44b6cc914c940a40d84f3b9f0b0a63fc741cb4c7cdeffe721ee1fb
3
+ size 4328576624
model-00006-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6d05d5dd7babe3aebdd601b8a37feef70eca03a33ab2a625b499ee294426ece4
3
+ size 4630620784
model-00007-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:601a011c6315121ec23e0c56f9bd60024472918b6b8f36f0987f91aa25d4c160
3
+ size 4630620800
model-00008-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4e14eb640deb880d4316d531456a4eacb560f91f6e067971ea9fbcaafbc2d22
3
+ size 4328576624
model-00009-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f17a62e853a2571aea2d660089d2e349fc919d3b8b59e17c8730b42e4baddd3
3
+ size 4630620776
model-00010-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4d0df5d72f7efdd05ab86de84590400d99bbd9128f0165d05e0ddb662dbc7f79
3
+ size 4630620800
model-00011-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d7ace8de20c45980dcbf0357ff40637019aa72ac000549f47d8f7e2c7d6308e5
3
+ size 4328576624
model-00012-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:52f28d489ea257d5899875fe57615fb63df87cccae375840b1402eea5ce4e5da
3
+ size 4630620784
model-00013-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d35ea22915abce387b66f4e325273ab16eda4ddf3abadc890b9822ab24d6c9ff
3
+ size 4630620800
model-00014-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:29af7647bba0fa570dccb18a2619da612ff9e9faa9f3864881619ed249ef90e4
3
+ size 4328576624
model-00015-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5b2f7f9cc95603855efa826714e6d5266b538bb66578fc73a431852790c37b0f
3
+ size 4630620784
model-00016-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e7d913dd535eec02dca5a30f521ab2c110a697177725777f7168fdd955eeaea2
3
+ size 4630620784
model-00017-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:704aded26e1ce6666a1390f113334fd62aa807a8b00fdee1e79895df577bf953
3
+ size 4328576624
model-00018-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d3f70904a1ffd7f02de2ad31c0a3af3737688514bf2ca47fe17cdaae86f4a72c
3
+ size 4630620784
model-00019-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0559957256bca6471d859cabc0d42aea625cbd5f16f502869395b96c79bd3559
3
+ size 4630620800
model-00020-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a36c8af9aed00bfe17d9302d7778aeba4a494e7ac8ff964591a8791fdfe87274
3
+ size 4328576624
model-00021-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58b5910a432070feffc786d5d5fde0cd503852e351bfe2b817f914bd3ceeefc0
3
+ size 4630620784
model-00022-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:247bc43760fc2a6a15f75e02352ee4457e77138b5e6fb5dda11ddddb935911a8
3
+ size 4630620784
model-00023-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91f7f514cb8eddf1863f30f600a18826d221c452e316ccea58d8dbe8d1a4814d
3
+ size 4328576624
model-00024-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:71209ffba09a028bba8b7dd28ab6ad05cadfc71d5dcd79b2aa4aad53cad617ed
3
+ size 4630620784
model-00025-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b5932f470a7c8ef71d4c40fab61c0c76101776fb9f1b99228de5a02d89d8622
3
+ size 4630620800
model-00026-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6b72fc600b144f8f3fcde23a2dd9c29580a5b2916d586e9eff373a3ed5c83af
3
+ size 4328576624
model-00027-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4db89bb9aba976a0f96eb6176f7538907ab3af4022e1573019c3abf20b8024a
3
+ size 4630620784
model-00028-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:227157f6d0bbea02600a631604b46b7d5492a072b86416057855899d7f58def6
3
+ size 4630620792
model-00029-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aef607efbc13109cdcb0bca8eab1056b0dc4910f08d2ab8d122785fbf6c05510
3
+ size 4328576616
model-00030-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c2356c939195331ce95e616db19190106096bffb5965d8b8facb2b27c500a740
3
+ size 4630620784
model-00031-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0a6358035e64b95ff4f5bfa46b8e0f352dc89acf4f582813bc4ccdee240af4fa
3
+ size 4630620800
model-00032-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4044206a0f4bf64cbd86ac8efd18f706c80b7e8741f9f67231d971a1ddaf389
3
+ size 4328576624
model-00033-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:089954bada7bc4335c34005478405b4bb8e2c48872b33a066db0ef8f19355c23
3
+ size 4630620784
model-00034-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:85cf52555fc4f81b6ee8ed877d3cdebcf9fd11ae7a0c41327543714c4af68f33
3
+ size 4630620800
model-00035-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:12e57415cfe1733cc860a190d14770f372599d36beb2f3fa422fdbea88ea4dc8
3
+ size 4328576624
model-00036-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:56aec78f17a28d7eaf818b2d05c7d6af61d31a10b7463be66fae48a413e4902f
3
+ size 4630620776
model-00037-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:49cd90695edf30eed40d3919e4d7bce229347862d7f8860b5c3e5030bdd1cd38
3
+ size 4630620800
model-00038-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:05c93988b60aa8696aef97254a051657f543b2ae94519d2f421bbbcaf97ca50e
3
+ size 4328576624
model-00039-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f21ca9fd3524e8c4f8f97f93acc8368430d8ca1ca37ae8f9108a843e75f33fd4
3
+ size 4630620784
model-00040-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:abaf49a6e8cc6927017aa194cd6789f2590c4f72e9b7b69dd79706eb93e28f38
3
+ size 4630620800
model-00041-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:060e91cc09936f6d2e5051caafa51546fa85df74c151e2bfd6f149614a39cbc1
3
+ size 4328576624
model-00042-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f58e896779c52a65fac0acbb4181025c10a64b0422d84501b9cc93aa5a58df8
3
+ size 4630620776
model-00043-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:30f9906e8bb194d0459536a8e70b0be23b7dc781c7f424f53027bfdfbf9484f7
3
+ size 4630620800
model-00044-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1c4d3362f95c3d63c01dd4c764dd27af381ff7e89cc06c9019147ab91857f652
3
+ size 4328576624
model-00045-of-00049.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:892ce587d19a333e169441a606b2c6fcd1dc897737fc6e051e8a6ccd6ae93f4a
3
+ size 4630620784