Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,12 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+code-millenials-34b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+code-millenials-34b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+code-millenials-34b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+code-millenials-34b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+code-millenials-34b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+code-millenials-34b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+code-millenials-34b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+code-millenials-34b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+code-millenials-34b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,103 @@
+---
+license: llama2
+library_name: transformers
+tags:
+- code
+model-index:
+- name: Code Millenials
+  results:
+  - task:
+      type: text-generation
+    dataset:
+      type: openai_humaneval
+      name: HumanEval
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 0.8048
+      verified: false
+---
+
+
+# Bud Code Millenials 34B
+
+Welcome to our Code Model repository! Our model is specifically fine-tuned for code generation tasks. Bud Millenial Code Gen open-source models are currently the State of the Art (SOTA) for code generation, beating all the existing models of all sizes. We have achieved a HumanEval value of 80.48 @ Pass 1, beating proprietary models like Gemini Ultra, Claude, GPT-3.5 etc. by a large margin, and on par with GPT-4 (HumanEval ~ 82. Ref. WizardCoder). Our proprietary model (Bud Code Jr) beats GPT-4 as well with a HumanEval value of 88.2 & a context size of 168K, we will be releasing an API for Researchers, Enterprises, and potential Partners by January 2024 end. If interested, please reach out to jithinvg@bud.studio
+### News 🔥🔥🔥
+
+- [2024/01/09] We released **Code Millenials 3B** , which achieves the **56.09 pass@1** on the [HumanEval Benchmarks](https://github.com/openai/human-eval).
+- [2024/01/09] We released **Code Millenials 1B** , which achieves the **51.82 pass@1** on the [HumanEval Benchmarks](https://github.com/openai/human-eval).
+- [2024/01/03] We released **Code Millenials 34B** , which achieves the **80.48 pass@1** on the [HumanEval Benchmarks](https://github.com/openai/human-eval).
+- [2024/01/02] We released **Code Millenials 13B** , which achieves the **76.21 pass@1** on the [HumanEval Benchmarks](https://github.com/openai/human-eval).
+
+
+### HumanEval
+
+<p align="center" width="100%">
+<a ><img src="https://raw.githubusercontent.com/BudEcosystem/code-millenials/main/assets/result.png" alt="CodeMillenials" style="width: 100%; min-width: 300px; display: block; margin: auto;"></a>
+</p>
+
+For the millenial models, the eval script in the github repo is used for the above result.
+
+Note: The humaneval values of other models are taken from the official repos of [WizardCoder](https://github.com/nlpxucan/WizardLM), [DeepseekCoder](https://github.com/deepseek-ai/deepseek-coder), [Gemini](https://deepmind.google/technologies/gemini/#capabilities) etc. 
+
+
+### Models
+
+|   Model | Checkpoint  | HumanEval (+) | MBPP (+) |
+|---------|-------------|---------------|----------|
+|Code Millenials 34B | <a href="https://huggingface.co/budecosystem/code-millenials-34b" target="_blank">HF Link</a> | 80.48 (75) | 74.68 (62.9) |
+|Code Millenials 13B | <a href="https://huggingface.co/budecosystem/code-millenials-13b" target="_blank">HF Link</a> | 76.21 (69.5) | 70.17 (57.6) |
+|Code Millenials 3B | <a href="https://huggingface.co/budecosystem/code-millenials-3b" target="_blank">HF Link</a> | 56.09 (52.43) | 55.13 (47.11) |
+|Code Millenials 1B | <a href="https://huggingface.co/budecosystem/code-millenials-1b" target="_blank">HF Link</a> | 51.82 (48.17) | 53.13 (44.61) |
+
+
+
+
+### 🚀 Quick Start
+
+Inference code  using the pre-trained model from the Hugging Face model hub
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("budecosystem/code-millenials-34b")
+model = AutoModelForCausalLM.from_pretrained("budecosystem/code-millenials-34b")
+
+template = """A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
+
+### Instruction: {instruction}
+
+### Response:"""
+
+instruction = <Your code instruction here>
+
+prompt = template.format(instruction=instruction)
+
+inputs = tokenizer(prompt, return_tensors="pt")
+sample = model.generate(**inputs, max_length=128)
+print(tokenizer.decode(sample[0]))
+
+```
+
+
+## Training details
+
+The model is trained of 16 A100 80GB for approximately 50hrs. 
+
+| Hyperparameters              | Value  |
+| :----------------------------| :-----: |
+| per_device_train_batch_size  | 16      |
+| gradient_accumulation_steps  | 1      |
+| epoch | 3 |
+| steps | 2157 |
+| learning_rate                | 2e-5   |
+| lr schedular type | cosine |
+| warmup ratio | 0.1 |
+| optimizer                    | adamw  |
+| fp16                         | True   |
+| GPU                          | 16 A100 80GB |
+
+### Important Note
+
+- **Bias, Risks, and Limitations:** Model may sometimes make errors, produce misleading contents, or struggle to manage tasks that are not related to coding.

code-millenials-34b-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6334921c5c0cb87b7c2be929d09cd9b65411bd0732776cd6e18ab25a6588af84
+size 17772181504

code-millenials-34b-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5199eba1b8469d5463e9a55793df84a27e6d7369e800a72c63f806e216f15f9
+size 16306796544

code-millenials-34b-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91420a5eecdfd129566fdd61184333f763a7ce25bf27f93ba1f24c6c0a28f000
+size 14606006272

code-millenials-34b-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6894744877f3e78a2f3cbf127e89f3ef0b26d4689b7c5d387c13045a6f66943
+size 20220626944

code-millenials-34b-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5b01825757116f22cfde53e2967ffb9e93acecb6d935d08fa7a032afcdf1ed6
+size 19192236032

code-millenials-34b-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8186fb4e84c91f28e04e316e72e21a92854143a5bfffd30ed8c6b6faed11fd9f
+size 23839590400

code-millenials-34b-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3c1a29810569c0bc38bc6475cb54ac915d0721e5ed949bee7397d56de8d3f400
+size 23237969920

code-millenials-34b-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcf896cec77ab82ca6ebf2f5cbe44418e3ec1830b607aa14544411f975486ef6
+size 27684739072

code-millenials-34b-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb01fd4068fb1e296ccedc0c90acdda36fe4a25dc18b096f1901493e72b07101
+size 35857168384

huggingface-metadata.txt

@@ -0,0 +1,19 @@
+url: https://huggingface.co/budecosystem/code-millenials-34b
+branch: main
+download date: 2024-01-23 11:57:32
+sha256sum:
+    d4680f6688122f232cd5f2769334676dafa24b7f261f3804d0c9be6ab6a4adb1 model-00001-of-00014.safetensors
+    1cc13e942a15e613e2e1d8a2ee1db1fe794e2d4f503bf32e84058fe4af001d42 model-00002-of-00014.safetensors
+    aa7bc658af1ba337a0d62efb8c645bc8b10280af8fca6ab6e3aec18eecdad01e model-00003-of-00014.safetensors
+    c862b756654cf664cd3483adeb0e03b96faa6bf16ce7f8ef9f3deb419d410e6b model-00004-of-00014.safetensors
+    b5ee437f3f73ecac402bf6c9731679f7c9e69bf87046fee3ac2b4fa3d077a1bd model-00005-of-00014.safetensors
+    26a432540d5227afc9775b54f4defe4cd6901c2401f53293113f32dcfecb2d70 model-00006-of-00014.safetensors
+    5c02ce91d74d0e06b35af56dcaf75321835f16a4390b1ed0a7ad6f36a481fa6a model-00007-of-00014.safetensors
+    0b217184ddd796820899986fdad4b940e3bb44ec57bf6cf7ad96ab055831e54a model-00008-of-00014.safetensors
+    f7389e0f61bf14ac0f927db6e50ccb939fa40fbabce8881a47481fcb737f707b model-00009-of-00014.safetensors
+    5ce25353bd93d6f91b27268076b226955523719382528384410f9001751eff58 model-00010-of-00014.safetensors
+    476791d7521e8c665ed000549c334f18b63651f113820e472e13d5b6e689eca5 model-00011-of-00014.safetensors
+    78d68c3e502aebf8f2490b8b537f7b75ef2c99bc3c49438e19f8ed31f53824b0 model-00012-of-00014.safetensors
+    92cdaa59d370a9e98defec990ccc023ffac1dc1add303bdb2768e4ead58e42f0 model-00013-of-00014.safetensors
+    1a86efb7d1b6aeb00552879caa7e8cbee24bfa5b73d7860f889cc1f6b88a69b6 model-00014-of-00014.safetensors
+    9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347 tokenizer.model