Update README.md

README.md

@@ -5,72 +5,115 @@ base_model:
 - OpenBuddy/openbuddy-mistral2-7b-v20.3-32k
 - meta-math/MetaMath-Mistral-7B
 - HuggingFaceH4/mistral-7b-grok
+- HuggingFaceH4/mistral-7b-anthropic
 - NousResearch/Yarn-Mistral-7b-128k
 - ajibawa-2023/Code-Mistral-7B
-- HuggingFaceH4/mistral-7b-anthropic
 - SherlockAssistant/Mistral-7B-Instruct-Ukrainian
+datasets:
+- HuggingFaceH4/grok-conversation-harmless
+- HuggingFaceH4/ultrachat_200k
+- HuggingFaceH4/ultrafeedback_binarized_fixed
+- HuggingFaceH4/cai-conversation-harmless
+- meta-math/MetaMathQA
+- emozilla/yarn-train-tokenized-16k-mistral
+- snorkelai/Snorkel-Mistral-PairRM-DPO-Dataset
+- microsoft/orca-math-word-problems-200k
+- m-a-p/Code-Feedback
+- teknium/openhermes
+- lksy/ru_instruct_gpt4
+- IlyaGusev/ru_turbo_saiga
+- IlyaGusev/ru_sharegpt_cleaned
+- IlyaGusev/oasst1_ru_main_branch
 library_name: transformers
 tags:
-- mergekit
+- mistral
+- gistral
+- gistral-16b
+- multilingual
+- code
+- metamath
+- grok
+- anthropic
+- openhermes
+- instruct
 - merge
-
+language:
+- en
+- fr
+- ru
+- de
+- ja
+- ko
+- zh
+- it
+- uk
+pipeline_tag: text-generation
 ---
-# merge
-
-This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
-
-## Merge Details
-### Merge Method
-
-This model was merged using the passthrough merge method.
-
-### Models Merged
-
-The following models were included in the merge:
-* [Gaivoronsky/Mistral-7B-Saiga](https://huggingface.co/Gaivoronsky/Mistral-7B-Saiga)
-* [snorkelai/Snorkel-Mistral-PairRM-DPO](https://huggingface.co/snorkelai/Snorkel-Mistral-PairRM-DPO)
-* [OpenBuddy/openbuddy-mistral2-7b-v20.3-32k](https://huggingface.co/OpenBuddy/openbuddy-mistral2-7b-v20.3-32k)
-* [meta-math/MetaMath-Mistral-7B](https://huggingface.co/meta-math/MetaMath-Mistral-7B)
-* [HuggingFaceH4/mistral-7b-grok](https://huggingface.co/HuggingFaceH4/mistral-7b-grok)
-* [NousResearch/Yarn-Mistral-7b-128k](https://huggingface.co/NousResearch/Yarn-Mistral-7b-128k)
-* [ajibawa-2023/Code-Mistral-7B](https://huggingface.co/ajibawa-2023/Code-Mistral-7B)
-* [HuggingFaceH4/mistral-7b-anthropic](https://huggingface.co/HuggingFaceH4/mistral-7b-anthropic)
-* [SherlockAssistant/Mistral-7B-Instruct-Ukrainian](https://huggingface.co/SherlockAssistant/Mistral-7B-Instruct-Ukrainian)
-
-### Configuration
-
-The following YAML configuration was used to produce this model:
-
-```yaml
-slices:
-  - sources:
-    - model: Gaivoronsky/Mistral-7B-Saiga
-      layer_range: [0, 32]
-  - sources:
-    - model: HuggingFaceH4/mistral-7b-grok
-      layer_range: [24, 32]
-  - sources:
-    - model: HuggingFaceH4/mistral-7b-anthropic
-      layer_range: [24, 32]
-  - sources:
-    - model: NousResearch/Yarn-Mistral-7b-128k
-      layer_range: [26, 32]
-  - sources:
-    - model: snorkelai/Snorkel-Mistral-PairRM-DPO
-      layer_range: [26, 32]
-  - sources:
-    - model: OpenBuddy/openbuddy-mistral2-7b-v20.3-32k
-      layer_range: [26, 32]
-  - sources:
-    - model: meta-math/MetaMath-Mistral-7B
-      layer_range: [28, 32]
-  - sources:
-    - model: ajibawa-2023/Code-Mistral-7B
-      layer_range: [28, 32]
-  - sources:
-    - model: SherlockAssistant/Mistral-7B-Instruct-Ukrainian
-      layer_range: [30, 32]
-merge_method: passthrough
-dtype: bfloat16
 
+# Gistral 16B (Mistral from 7B to 16B)
+
+![logo](assets/logo.png)
+
+We created a model from other cool models to combine everything into one cool model.
+
+
+## Model Details
+
+### Model Description 
+
+- **Developed by:** [@ehristoforu](https://huggingface.co/ehristoforu)
+- **Model type:** Text Generation (conversational)
+- **Language(s) (NLP):** English, French, Russian, German, Japanese, Chinese, Korean, Italian, Ukrainian, Code
+- **Finetuned from model:** [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)
+
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+```py
+from transformers import AutoModelForCausalLM, AutoTokenizer
+model_id = "ehristoforu/Gistral-16B-v0.1"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
+messages = [
+    {"role": "user", "content": "What is your favourite condiment?"},
+    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
+    {"role": "user", "content": "Do you have mayonnaise recipes?"}
+]
+inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
+outputs = model.generate(inputs, max_new_tokens=20)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
+
+
+## About merge
+
+Base model: mistralai/Mistral-7B-Instruct-v0.2
+
+Merge models:
+- Gaivoronsky/Mistral-7B-Saiga
+- snorkelai/Snorkel-Mistral-PairRM-DPO
+- OpenBuddy/openbuddy-mistral2-7b-v20.3-32k
+- meta-math/MetaMath-Mistral-7B
+- HuggingFaceH4/mistral-7b-grok
+- HuggingFaceH4/mistral-7b-anthropic
+- NousResearch/Yarn-Mistral-7b-128k
+- ajibawa-2023/Code-Mistral-7B
+- SherlockAssistant/Mistral-7B-Instruct-Ukrainian
+
+Merge datasets:
+- HuggingFaceH4/grok-conversation-harmless
+- HuggingFaceH4/ultrachat_200k
+- HuggingFaceH4/ultrafeedback_binarized_fixed
+- HuggingFaceH4/cai-conversation-harmless
+- meta-math/MetaMathQA
+- emozilla/yarn-train-tokenized-16k-mistral
+- snorkelai/Snorkel-Mistral-PairRM-DPO-Dataset
+- microsoft/orca-math-word-problems-200k
+- m-a-p/Code-Feedback
+- teknium/openhermes
+- lksy/ru_instruct_gpt4
+- IlyaGusev/ru_turbo_saiga
+- IlyaGusev/ru_sharegpt_cleaned
+- IlyaGusev/oasst1_ru_main_branch