Update README.md

README.md

@@ -1,26 +1,18 @@
 ---
 license: apache-2.0
-pipeline_tag: text-generation
 ---
 
-<p align="center" style="font-size:34px;"><b>Buddhi-128K-Chat</b></p>
+<p align="center" style="font-size:34px;"><b>Buddhi 7B</b></p>
 
-# Buddhi-128K-Chat (7B) vLLM Inference: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/11_8W8FpKK-856QdRVJLyzbu9g-DMxNfg?usp=sharing)
+# Buddhi-7B vLLM Inference: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/11_8W8FpKK-856QdRVJLyzbu9g-DMxNfg?usp=sharing)
 
-# Read release article: [🔗 Introducing Buddhi: Open-Source Chat Model with a 128K Context Window 🔗 ](https://medium.aiplanet.com/introducing-buddhi-open-source-chat-model-with-a-128k-context-window-06a1848121d0)
+# Model Description
 
-![4.png](https://cdn-uploads.huggingface.co/production/uploads/630f3058236215d0b7078806/VUY0c4xOGpH9jTNmf6XNU.png)
+<!-- Provide a quick summary of what the model is/does. -->
 
-## Model Description
-
-Buddhi-128k-Chat is a general-purpose first chat model with 128K context length window. It is meticulously fine-tuned on the Mistral 7B Instruct, and optimised to handle an extended context length of up to 128,000 tokens using the innovative YaRN (Yet another Rope Extension) Technique. This enhancement allows Buddhi to maintain a deeper understanding of context in long documents or conversations, making it particularly adept at tasks requiring extensive context retention, such as comprehensive document summarization, detailed narrative generation, and intricate question-answering.
+Buddhi is a general-purpose chat model, meticulously fine-tuned on the Mistral 7B Instruct, and optimised to handle an extended context length of up to 128,000 tokens using the innovative YaRN [(Yet another Rope Extension)](https://arxiv.org/abs/2309.00071) Technique. This enhancement allows Buddhi to maintain a deeper understanding of context in long documents or conversations, making it particularly adept at tasks requiring extensive context retention, such as comprehensive document summarization, detailed narrative generation, and intricate question-answering. 
 
 ## Architecture
-The Buddhi-128K-Chat model is fine-tuned on the Mistral-7B Instruct base model. We selected the Mistral 7B Instruct v0.2 as the parent model due to its superior reasoning capabilities. The architecture of the Mistral-7B model includes features like Grouped-Query Attention and Byte-fallback BPE tokenizer. Originally, this model has 32,768 maximum position embeddings. To increase the context size to 128K, we needed to modify the positional embeddings, which is where YaRN comes into play.
-
-In our approach, we utilized the NTK-aware technique, which recommends alternative interpolation techniques for positional interpolation. One experimentation involved Dynamic-YARN, suggesting the dynamic value of the 's' scale factor. This is because during inference, the sequence length changes by 1 after every word prediction. By integrating these position embeddings with the Mistral-7B Instruct base model, we achieved the 128K model. 
-
-Additionally, we fine-tuned the model on our dataset to contribute one of the very few 128K chat-based models available in the open-source community with greater reasoning capabilities than all of it. 
 
 ### Hardware requirements:
 > For 128k Context Length
@@ -41,17 +33,13 @@ Please check out [Flash Attention 2](https://github.com/Dao-AILab/flash-attentio
 
 **Implementation**:
 
-> Note: The actual hardware requirements to run the model is roughly around 70GB VRAM. For experimentation, we are limiting the context length to 75K instead of 128K. This make it suitable for testing the model in 30-35 GB VRAM
-
 ```python
 from vllm import LLM, SamplingParams
 
 llm = LLM(
-    model='aiplanet/buddhi-128k-chat-7b',
-    trust_remote_code=True,
-    dtype = 'bfloat16',
-    gpu_memory_utilization=1,
-    max_model_len= 75000 
+  model='aiplanet/Buddhi-128K-Chat',
+  gpu_memory_utilization=0.99,
+  max_model_len=131072
 )
 
 prompts = [
@@ -72,12 +60,8 @@ for output in outputs:
     generated_text = output.outputs[0].text
     print(generated_text)
     print("\n\n")
-
-# we have also attached a colab notebook, that contains: 2 more experimentations: Long Essay and Entire Book
 ```
 
-For Output, do check out the colab notebook: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/11_8W8FpKK-856QdRVJLyzbu9g-DMxNfg?usp=sharing)
-
 ### Transformers - Basic Implementation
 
 ```python
@@ -130,8 +114,7 @@ Why don't scientists trust atoms?
 Because they make up everything.
 ```
 
-
-## Prompt Template for Buddi-128-Chat
+## Prompt Template for Panda Coder 13B
 
 In order to leverage instruction fine-tuning, your prompt should be surrounded by [INST] and [/INST] tokens. The very first instruction should begin with a begin of sentence id. The next instructions should not. The assistant generation will be ended by the end-of-sentence token id.
 
@@ -141,39 +124,18 @@ In order to leverage instruction fine-tuning, your prompt should be surrounded b
 "[INST] Do you have mayonnaise recipes? [/INST]"
 
 ```
+## 🔗 Key Features:
 
-# Benchmarks
-
-### Long Context Benchmark
+ 🎯 Precision and Efficiency: The model is tailored for accuracy, ensuring your code is not just functional but also efficient.
 
-<strong>LongICLBench Banking77</strong>
-<div>
-  
-| Model                                   | 1R/2k | 2R/4K | 3R/7K | 4R/9K | 5R/14K |
-|-----------------------------------------|-------|-------|-------|-------|--------|
-| aiplanet/buddhi-128k-chat-7b            | 47.8  | 60.8  | 57.8  | 62.4  |  57.2  |
-| NousResearch/Yarn-Mistral-7b-128k       | 31.6  | 68.6  | 68    | 47    |  65.6  |
-| CallComply/zephyr-7b-beta-128k          | 40.2  | 41.2  | 33.6  | 03    |    0   |
-| Eric111/Yarn-Mistral-7b-128k-DPO        | 28.6  | 62.8  | 58    | 41.6  |  59.8  |
+ ✨ Unleash Creativity: Whether you're a novice or an expert coder, Panda-Coder is here to support your coding journey, offering creative solutions to your programming challenges.
 
-</div>
+ 📚 Evol Instruct Code: It's built on the robust Evol Instruct Code 80k-v1 dataset, guaranteeing top-notch code generation.
 
-<strong>Short Context Benchmark</strong>
-<div>
-  
-| Model                             | # Params | Average | ARC (25-shot) | HellaSwag (10-shot) | Winogrande (5-shot) | TruthfulOA (0-shot) | MMLU (5-shot) |
-|-----------------------------------|----------|---------|---------------|---------------------|---------------------|---------------------|---------------|
-| aiplanet/buddhi-128k-chat-7b      | 7B       | 64.42   | 60.84         | 84                  | 77.27               | 65.72               | 60.42         |
-| migtissera/Tess-XS-vl-3-yarn-128K | 7B       | 62.66   | 61.09         | 82.95               | 74.43               | 50.13               | 62.15         |
-| migtissera/Tess-XS-v1-3-yarn-128K | 7B       | 62.49   | 61.6          | 82.96               | 74.74               | 50.2                | 62.1          |
-| Eric111/Yarn-Mistral-7b-128k-DPO  | 7B       | 60.15   | 60.84         | 82.99               | 78.3                | 43.55               | 63.09         |
-| NousResearch/Yam-Mistral-7b-128k  | 7B       | 59.42   | 59.64         | 82.5                | 76.95               | 41.78               | 63.02         |
-| CallComply/openchat-3.5-0106-128k | 7B       | 59.38   | 64.25         | 77.31               | 77.66               | 46.5                | 57.58         |
-| CallComply/zephyr-7b-beta-128k    | 7B       | 54.45   | 58.28         | 81                  | 74.74               | 46.1                | 53.57         |
+ 📢 What's Next?: We believe in continuous improvement and are excited to announce that in our next release, Panda-Coder will be enhanced with a custom dataset. This dataset will not only expand the language support but also include hardware programming languages like MATLAB, Embedded C, and Verilog. 🧰💡
 
-</div>
 
-## Get in Touch
+ ## Get in Touch
 
  You can schedule a 1:1 meeting with our DevRel & Community Team to get started with AI Planet Open Source LLMs and GenAI Stack. Schedule the call here: [https://calendly.com/jaintarun](https://calendly.com/jaintarun)
 
@@ -191,8 +153,8 @@ In order to leverage instruction fine-tuning, your prompt should be surrounded b
  ### Citation
 
  ```
- @misc {Chaitanya890, lucifertrj ,
-	author       = { Chaitanya Singhal, Tarun Jain },
+ @misc {Chaitanya890,
+	author       = { {Chaitanya Singhal} },
 	title        = { Buddhi-128k-Chat by AI Planet},
 	year         = 2024,
 	url          = { https://huggingface.co/aiplanet//Buddhi-128K-Chat },

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "aiplanet/buddhi-128k-chat-7b",
+  "_name_or_path": "aiplanet/Buddhi-128K-Chat",
   "architectures": [
     "MistralForCausalLM"
   ],
@@ -25,7 +25,7 @@
     "factor": 4.0,
     "finetuned": true,
     "original_max_position_embeddings": 32768,
-    "type": "yarn"
+    "type": "dynamic-yarn"
   },
   "rope_theta": 1000000.0,
   "sliding_window": null,