Create README.md

README.md

@@ -0,0 +1,104 @@
+---
+license: apache-2.0
+language:
+- en
+- de
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- finetune
+- sft
+- dpo
+- laser
+- augmentation
+- german
+- english
+- moe
+---
+![SauerkrautLM](https://vago-solutions.de/wp-content/uploads/2024/02/Sauerkraut_Laserchat-14b-MoE.png "SauerkrautLM-14b-MoE-LaserChat")
+## VAGO solutions SauerkrautLM-14b-MoE-LaserChat
+Introducing **SauerkrautLM-14b-MoE-LaserChat** – our Sauerkraut (2x7b) 14b MoE version of the powerful [SauerkrautLM-7b-LaserChat](https://huggingface.co/VAGOsolutions/SauerkrautLM-7b-LaserChat) and [yam-peleg/Experiment26-7B](https://huggingface.co/yam-peleg/Experiment26-7B)  !
+
+By combining the two models, we were able to significantly increase both the German and English language skills.
+In addition, the initial SauerkrautLM-7b-LaserChat also acts as an adapter for Experiment26-7B, which means it benefits from the chat capabilities of the SauerkrautLM-7b-LaserChat. 
+At the same time, the SauerkrautLM-7b-LaserChat benefits from the knowledge and creativity of Experiment26-7B.
+
+The model **SauerkrautLM-14b-MoE-LaserChat** is a **joint effort** between **VAGO solutions** and **Hyperspace.ai.** 
+Much appreciation goes to the tremendous research effort of **Fernando Fernandes Neto, David Golchinfar and Eric Hartford on their laserRMT approach.** 
+Without their independent research collaboration this model release would not have been possible. 
+
+
+# Table of Contents
+1. [Overview of all SauerkrautLM-14b-MoE-LaserChat models](#all-sauerkrautlm-14b-MoE-laserchat-models)
+2. [Model Details](#model-details)
+   - [Prompt template](#prompt-template)
+   - [Training procedure](#proceed-of-the-training)
+3. [Evaluation](#evaluation)
+5. [Disclaimer](#disclaimer)
+6. [Contact](#contact)
+7. [Collaborations](#collaborations)
+8. [Acknowledgement](#acknowledgement)
+
+
+## All SauerkrautLM-14b-MoE-LaserChat Models
+
+| Model | HF    | GPTQ  | GGUF  | AWQ  |
+|-------|-------|-------|-------|-------|
+| SauerkrautLM-14b-MoE-LaserChat  | [Link](https://huggingface.co/VAGOsolutions/SauerkrautLM-14b-MoE-LaserChat) | coming soon | coming soon | coming soon |
+
+## Model Details
+**SauerkrautLM-14b-MoE-LaserChat**
+- **Model Type:** SauerkrautLM-14b-MoE-LaserChat is a MoE Model based on [SauerkrautLM-7b-LaserChat](https://huggingface.co/VAGOsolutions/SauerkrautLM-7b-LaserChat) and [yam-peleg/Experiment26-7B](https://huggingface.co/yam-peleg/Experiment26-7B) 
+- **Language(s):** German, English
+- **License:** Apache 2.0
+- **Contact:** [VAGO solutions](https://vago-solutions.ai), [Hyperspace.computer](https://hyperspace.computer/)
+
+
+We improved the German language skills on this model further. Nevertheless, certain formulations may occur that are not entirely correct.
+
+
+### Prompt Template:
+```
+GPT4 Correct User: Hallo, wie geht es dir?<|end_of_turn|>GPT4 Correct Assistant: Hallo! Ich bin ein künstliches Intelligenzsystem und habe keine persönlichen Gefühle oder körperliche Zustände. Wie kann ich Ihnen helfen?<|end_of_turn|>GPT4 Correct User: Ich benötige nur einen kurzen Satz, den ich in das Prompt Template veröffentlichen kann.<|end_of_turn|>GPT4 Correct Assistant:
+
+
+```
+
+
+```
+GPT4 Correct User: Hello<|end_of_turn|>GPT4 Correct Assistant: Hello! How can I help you today? If you have any questions or need assistance, feel free to ask.<|end_of_turn|>GPT4 Correct User: I just need a short sentence to post in the prompt template.<|end_of_turn|>GPT4 Correct Assistant:
+
+```
+
+
+## Evaluation
+
+**Open LLM Leaderboard:**
+
+benchmarked on lm-evaluation-harness 0.4.1
+
+| Metric                | Value                     |
+|-----------------------|---------------------------|
+| Avg.                  | 71.65 |
+| ARC (25-shot)         | 68.09         |
+| HellaSwag (10-shot)   | 84.78  |
+| MMLU (5-shot)         | 63.59|
+| TruthfulQA (0-shot)   | 58.57 |
+| Winogrande (5-shot)   | 80.74  |
+| GSM8K (5-shot)        | 74.15        |
+
+
+
+## Disclaimer
+We must inform users that despite our best efforts in data cleansing, the possibility of uncensored content slipping through cannot be entirely ruled out.
+However, we cannot guarantee consistently appropriate behavior. Therefore, if you encounter any issues or come across inappropriate content, we kindly request that you inform us through the contact information provided.
+Additionally, it is essential to understand that the licensing of these models does not constitute legal advice. We are not held responsible for the actions of third parties who utilize our models.
+ 
+## Contact
+If you are interested in customized LLMs for business applications, please get in contact with us via our websites. We are also grateful for your feedback and suggestions.
+ 
+## Collaborations
+We are also keenly seeking support and investment for our startups, VAGO solutions and Hyperspace where we continuously advance the development of robust language models designed to address a diverse range of purposes and requirements. If the prospect of collaboratively navigating future challenges excites you, we warmly invite you to reach out to us at [VAGO solutions](https://vago-solutions.de/#Kontakt), [Hyperspace.computer](https://hyperspace.computer/)
+
+## Acknowledgement
+Many thanks to [yam-peleg](https://huggingface.co/yam-peleg) for providing such valuable model to the Open-Source community