SteelStorage
/

Umbra-v3-MoE-4x11b

@@ -5,7 +5,6 @@ tags:
 - frankenmoe
 - merge
 - mergekit
-- lazymergekit
 - Himitsui/Kaiju-11B
 - Sao10K/Fimbulvetr-11B-v2
 - decapoda-research/Antares-11b-v2
@@ -16,157 +15,111 @@ base_model:
 - decapoda-research/Antares-11b-v2
 - beberik/Nyxene-v3-11B
 ---
-# Umbra-v3-MoE-4x11b
-Umbra-v3-MoE-4x11b is a Mixure of Experts (MoE) made with the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
-* [Himitsui/Kaiju-11B](https://huggingface.co/Himitsui/Kaiju-11B)
-* [Sao10K/Fimbulvetr-11B-v2](https://huggingface.co/Sao10K/Fimbulvetr-11B-v2)
-* [decapoda-research/Antares-11b-v2](https://huggingface.co/decapoda-research/Antares-11b-v2)
-* [beberik/Nyxene-v3-11B](https://huggingface.co/beberik/Nyxene-v3-11B)
-## 🧩 Configuration
-```yaml
-base_model: vicgalle/CarbonBeagle-11B-truthy
-gate_mode: hidden
-dtype: bfloat16
-experts_per_token: 4
-experts:
-  - source_model: Himitsui/Kaiju-11B
-    positive_prompts:
-      - "Imagine"
-      - "Create"
-      - "Envision"
-      - "Fantasize"
-      - "Invent"
-      - "Narrate"
-      - "Plot"
-      - "Portray"
-      - "Storytell"
-      - "Visualize"
-      - "Describe"
-      - "Develop"
-      - "Forge"
-      - "Craft"
-      - "Conceptualize"
-      - "Dream"
-      - "Concoct"
-      - "Characterize"
-    negative_prompts:
-      - "Recite"
-      - "Report"
-      - "Summarize"
-      - "Enumerate"
-      - "List"
-      - "Cite"
-  - source_model: Sao10K/Fimbulvetr-11B-v2
-    positive_prompts:
-      - "Dramatize"
-      - "Embody"
-      - "Illustrate"
-      - "Perform"
-      - "Roleplay"
-      - "Simulate"
-      - "Stage"
-      - "Unfold"
-      - "Weave"
-      - "Design"
-      - "Outline"
-      - "Script"
-      - "Sketch"
-      - "Spin"
-      - "Depict"
-      - "Render"
-      - "Fashion"
-      - "Conceive"
-    negative_prompts:
-      - "Analyze"
-      - "Critique"
-      - "Dissect"
-      - "Explain"
-      - "Clarify"
-      - "Interpret"
-  - source_model: decapoda-research/Antares-11b-v2
-    positive_prompts:
-      - "Solve"
-      - "Respond"
-      - "Convey"
-      - "Disclose"
-      - "Expound"
-      - "Narrate"
-      - "Present"
-      - "Reveal"
-      - "Specify"
-      - "Uncover"
-      - "Decode"
-      - "Examine"
-      - "Report"
-      - "Survey"
-      - "Validate"
-      - "Verify"
-      - "Question"
-      - "Query"
-    negative_prompts:
-      - "Divert"
-      - "Obscure"
-      - "Overstate"
-      - "Undermine"
-      - "Misinterpret"
-      - "Skew"
-  - source_model: beberik/Nyxene-v3-11B
-    positive_prompts:
-      - "Explain"
-      - "Instruct"
-      - "Clarify"
-      - "Educate"
-      - "Guide"
-      - "Inform"
-      - "Teach"
-      - "Detail"
-      - "Elaborate"
-      - "Enlighten"
-      - "Advise"
-      - "Interpret"
-      - "Analyze"
-      - "Define"
-      - "Demonstrate"
-      - "Illustrate"
-      - "Simplify"
-      - "Summarize"
-    negative_prompts:
-      - "Speculate"
-      - "Fabricate"
-      - "Exaggerate"
-      - "Mislead"
-      - "Confuse"
-      - "Distort"
-```
-## 💻 Usage
-```python
-!pip install -qU transformers bitsandbytes accelerate
-from transformers import AutoTokenizer
-import transformers
-import torch
-model = "Steelskull/Umbra-v3-MoE-4x11b"
-tokenizer = AutoTokenizer.from_pretrained(model)
-pipeline = transformers.pipeline(
-    "text-generation",
-    model=model,
-    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
-)
-messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
-prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
-outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
-print(outputs[0]["generated_text"])
-```

 - frankenmoe
 - merge
 - mergekit
 - Himitsui/Kaiju-11B
 - Sao10K/Fimbulvetr-11B-v2
 - decapoda-research/Antares-11b-v2
 - decapoda-research/Antares-11b-v2
 - beberik/Nyxene-v3-11B
 ---
+<!DOCTYPE html>
+<style>
+    body {
+    font-family: 'Quicksand', sans-serif;
+    background: linear-gradient(135deg, #2E3440 0%, #1A202C 100%);
+    color: #D8DEE9;
+    margin: 0;
+    padding: 0;
+    font-size: 16px;
+}
+.container {
+    width: 80%;
+    max-width: 800px;
+    margin: 20px auto;
+    background-color: rgba(255, 255, 255, 0.02);
+    padding: 20px;
+    border-radius: 12px;
+    box-shadow: 0 4px 10px rgba(0, 0, 0, 0.2);
+    backdrop-filter: blur(10px);
+    border: 1px solid rgba(255, 255, 255, 0.1);
+}
+.header h1 {
+    font-size: 28px;
+    color: #ECEFF4;
+    margin: 0 0 20px 0;
+    text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.3);
+}
+.info img {
+    width: 100%;
+    border-radius: 10px;
+    margin-bottom: 15px;
+}
+a {
+    color: #88C0D0;
+    text-decoration: none;
+}
+a:hover {
+    color: #A3BE8C;
+}
+a::before {
+    content: '';
+    position: absolute;
+    width: 100%;
+    height: 2px;
+    bottom: -5px;
+    left: 0;
+    background-color: #A3BE8C;
+    visibility: hidden;
+    transform: scaleX(0);
+    transition: all 0.3s ease-in-out;
+}
+a:hover::before {
+    visibility: visible;
+    transform: scaleX(1);
+}
+.button {
+    display: inline-block;
+    background-color: #5E81AC;
+    color: #E5E9F0;
+    padding: 10px 20px;
+    border-radius: 5px;
+    cursor: pointer;
+    text-decoration: none;
+}
+.button:hover {
+    background-color: #81A1C1;
+}
+</style>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Umbra-v3-MoE-4x11b Data Card</title>
+    <link href="https://fonts.googleapis.com/css2?family=Quicksand:wght@400;500;600&display=swap" rel="stylesheet">
+</head>
+<body>
+<div class="container">
+    <div class="header">
+        <h1>Umbra-v3-MoE-4x11b</h1>
+    </div>
+    <div class="info">
+        <img src="https://cdn-uploads.huggingface.co/umbra-v3-moe-visual.webp">
+        <p><strong>Creator:</strong> <a href="https://huggingface.co/Steelskull" target="_blank">SteelSkull</a></p>
+        <p><strong>About Umbra-v3-MoE-4x11b:</strong> A Mixture of Experts model designed for general assistance with a special knack for storytelling and RP/ERP, built using LazyMergekit.</p>
+        <p>Integrates models from notable sources for enhanced performance in diverse tasks.</p>
+        <p><strong>Source Models:</strong></p>
+        <ul>
+            <li><a href="https://huggingface.co/Himitsui/Kaiju-11B">Himitsui/Kaiju-11B</a></li>
+            <li><a href="https://huggingface.co/Sao10K/Fimbulvetr-11B-v2">Sao10K/Fimbulvetr-11B-v2</a></li>
+            <li><a href="https://huggingface.co/decapoda-research/Antares-11b-v2">decapoda-research/Antares-11b-v2</a></li>
+            <li><a href="https://huggingface.co/beberik/Nyxene-v3-11B">beberik/Nyxene-v3-11B</a></li>
+        </ul>
+        <p><strong>Configuration Highlights:</strong> Features a carefully curated mix of positive and negative prompts tailored to leverage the unique strengths of each model, enhancing its general AI capabilities while maintaining prowess in creative storytelling.</p>
+        <p><strong>Usage Instructions:</strong> Provided in the form of Python code snippets, making it easy for developers to integrate Umbra-v3 into their projects for advanced text generation tasks.</p>
+    </div>
+</div>
+</body>
+</html>