| Quantization made by Richard Erkhov. | |
| [Github](https://github.com/RichardErkhov) | |
| [Discord](https://discord.gg/pvy7H8DZMG) | |
| [Request more models](https://github.com/RichardErkhov/quant_request) | |
| Llama-160M-Chat-v1 - GGUF | |
| - Model creator: https://huggingface.co/Felladrin/ | |
| - Original model: https://huggingface.co/Felladrin/Llama-160M-Chat-v1/ | |
| | Name | Quant method | Size | | |
| | ---- | ---- | ---- | | |
| | [Llama-160M-Chat-v1.Q2_K.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q2_K.gguf) | Q2_K | 0.07GB | | |
| | [Llama-160M-Chat-v1.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.IQ3_XS.gguf) | IQ3_XS | 0.07GB | | |
| | [Llama-160M-Chat-v1.IQ3_S.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.IQ3_S.gguf) | IQ3_S | 0.07GB | | |
| | [Llama-160M-Chat-v1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q3_K_S.gguf) | Q3_K_S | 0.07GB | | |
| | [Llama-160M-Chat-v1.IQ3_M.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.IQ3_M.gguf) | IQ3_M | 0.08GB | | |
| | [Llama-160M-Chat-v1.Q3_K.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q3_K.gguf) | Q3_K | 0.08GB | | |
| | [Llama-160M-Chat-v1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q3_K_M.gguf) | Q3_K_M | 0.08GB | | |
| | [Llama-160M-Chat-v1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q3_K_L.gguf) | Q3_K_L | 0.08GB | | |
| | [Llama-160M-Chat-v1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.IQ4_XS.gguf) | IQ4_XS | 0.09GB | | |
| | [Llama-160M-Chat-v1.Q4_0.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q4_0.gguf) | Q4_0 | 0.09GB | | |
| | [Llama-160M-Chat-v1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.IQ4_NL.gguf) | IQ4_NL | 0.09GB | | |
| | [Llama-160M-Chat-v1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q4_K_S.gguf) | Q4_K_S | 0.09GB | | |
| | [Llama-160M-Chat-v1.Q4_K.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q4_K.gguf) | Q4_K | 0.1GB | | |
| | [Llama-160M-Chat-v1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q4_K_M.gguf) | Q4_K_M | 0.1GB | | |
| | [Llama-160M-Chat-v1.Q4_1.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q4_1.gguf) | Q4_1 | 0.1GB | | |
| | [Llama-160M-Chat-v1.Q5_0.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q5_0.gguf) | Q5_0 | 0.11GB | | |
| | [Llama-160M-Chat-v1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q5_K_S.gguf) | Q5_K_S | 0.11GB | | |
| | [Llama-160M-Chat-v1.Q5_K.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q5_K.gguf) | Q5_K | 0.11GB | | |
| | [Llama-160M-Chat-v1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q5_K_M.gguf) | Q5_K_M | 0.11GB | | |
| | [Llama-160M-Chat-v1.Q5_1.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q5_1.gguf) | Q5_1 | 0.12GB | | |
| | [Llama-160M-Chat-v1.Q6_K.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q6_K.gguf) | Q6_K | 0.12GB | | |
| | [Llama-160M-Chat-v1.Q8_0.gguf](https://huggingface.co/RichardErkhov/Felladrin_-_Llama-160M-Chat-v1-gguf/blob/main/Llama-160M-Chat-v1.Q8_0.gguf) | Q8_0 | 0.16GB | | |
| Original model description: | |
| --- | |
| language: | |
| - en | |
| license: apache-2.0 | |
| tags: | |
| - text-generation | |
| base_model: JackFram/llama-160m | |
| datasets: | |
| - ehartford/wizard_vicuna_70k_unfiltered | |
| - totally-not-an-llm/EverythingLM-data-V3 | |
| - Open-Orca/SlimOrca-Dedup | |
| - databricks/databricks-dolly-15k | |
| - THUDM/webglm-qa | |
| widget: | |
| - messages: | |
| - role: system | |
| content: You are a helpful assistant, who answers with empathy. | |
| - role: user | |
| content: Got a question for you! | |
| - role: assistant | |
| content: "Sure! What's it?" | |
| - role: user | |
| content: Why do you love cats so much!? 🐈 | |
| - messages: | |
| - role: system | |
| content: "You are a helpful assistant who answers user's questions with empathy." | |
| - role: user | |
| content: Who is Mona Lisa? | |
| - messages: | |
| - role: system | |
| content: You are a helpful assistant who provides concise responses. | |
| - role: user | |
| content: Heya! | |
| - role: assistant | |
| content: Hi! How may I help you today? | |
| - role: user | |
| content: I need to build a simple website. Where should I start learning about web development? | |
| - messages: | |
| - role: user | |
| content: Invited some friends to come home today. Give me some ideas for games to play with them! | |
| - messages: | |
| - role: system | |
| content: "You are a helpful assistant who answers user's questions with details and curiosity." | |
| - role: user | |
| content: What are some potential applications for quantum computing? | |
| - messages: | |
| - role: system | |
| content: You are a helpful assistant who gives creative responses. | |
| - role: user | |
| content: Write the specs of a game about mages in a fantasy world. | |
| - messages: | |
| - role: system | |
| content: "You are a helpful assistant who answers user's questions with details." | |
| - role: user | |
| content: Tell me about the pros and cons of social media. | |
| - messages: | |
| - role: system | |
| content: "You are a helpful assistant who answers user's questions with confidence." | |
| - role: user | |
| content: What is a dog? | |
| - role: assistant | |
| content: 'A dog is a four-legged, domesticated animal that is a member of the class Mammalia, | |
| which includes all mammals. Dogs are known for their loyalty, playfulness, and | |
| ability to be trained for various tasks. They are also used for hunting, herding, | |
| and as service animals.' | |
| - role: user | |
| content: What is the color of an apple? | |
| inference: | |
| parameters: | |
| max_new_tokens: 250 | |
| penalty_alpha: 0.5 | |
| top_k: 4 | |
| repetition_penalty: 1.01 | |
| model-index: | |
| - name: Llama-160M-Chat-v1 | |
| results: | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: AI2 Reasoning Challenge (25-Shot) | |
| type: ai2_arc | |
| config: ARC-Challenge | |
| split: test | |
| args: | |
| num_few_shot: 25 | |
| metrics: | |
| - type: acc_norm | |
| value: 24.74 | |
| name: normalized accuracy | |
| source: | |
| url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: HellaSwag (10-Shot) | |
| type: hellaswag | |
| split: validation | |
| args: | |
| num_few_shot: 10 | |
| metrics: | |
| - type: acc_norm | |
| value: 35.29 | |
| name: normalized accuracy | |
| source: | |
| url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: MMLU (5-Shot) | |
| type: cais/mmlu | |
| config: all | |
| split: test | |
| args: | |
| num_few_shot: 5 | |
| metrics: | |
| - type: acc | |
| value: 26.13 | |
| name: accuracy | |
| source: | |
| url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: TruthfulQA (0-shot) | |
| type: truthful_qa | |
| config: multiple_choice | |
| split: validation | |
| args: | |
| num_few_shot: 0 | |
| metrics: | |
| - type: mc2 | |
| value: 44.16 | |
| source: | |
| url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: Winogrande (5-shot) | |
| type: winogrande | |
| config: winogrande_xl | |
| split: validation | |
| args: | |
| num_few_shot: 5 | |
| metrics: | |
| - type: acc | |
| value: 51.3 | |
| name: accuracy | |
| source: | |
| url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: GSM8k (5-shot) | |
| type: gsm8k | |
| config: main | |
| split: test | |
| args: | |
| num_few_shot: 5 | |
| metrics: | |
| - type: acc | |
| value: 0.0 | |
| name: accuracy | |
| source: | |
| url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 | |
| name: Open LLM Leaderboard | |
| --- | |
| # A Llama Chat Model of 160M Parameters | |
| - Base model: [JackFram/llama-160m](https://huggingface.co/JackFram/llama-160m) | |
| - Datasets: | |
| - [ehartford/wizard_vicuna_70k_unfiltered](https://huggingface.co/datasets/ehartford/wizard_vicuna_70k_unfiltered) | |
| - [totally-not-an-llm/EverythingLM-data-V3](https://huggingface.co/datasets/totally-not-an-llm/EverythingLM-data-V3) | |
| - [Open-Orca/SlimOrca-Dedup](https://huggingface.co/datasets/Open-Orca/SlimOrca-Dedup) | |
| - [databricks/databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) | |
| - [THUDM/webglm-qa](https://huggingface.co/datasets/THUDM/webglm-qa) | |
| - Availability in other ML formats: | |
| - GGUF: [Felladrin/gguf-Llama-160M-Chat-v1](https://huggingface.co/Felladrin/gguf-Llama-160M-Chat-v1) | |
| - ONNX: [Felladrin/onnx-Llama-160M-Chat-v1](https://huggingface.co/Felladrin/onnx-Llama-160M-Chat-v1) | |
| - MLC: [Felladrin/mlc-q4f16-Llama-160M-Chat-v1](https://huggingface.co/Felladrin/mlc-q4f16-Llama-160M-Chat-v1) | |
| - MLX: [mlx-community/Llama-160M-Chat-v1-4bit-mlx](https://huggingface.co/mlx-community/Llama-160M-Chat-v1-4bit-mlx) | |
| ## Recommended Prompt Format | |
| ``` | |
| <|im_start|>system | |
| {system_message}<|im_end|> | |
| <|im_start|>user | |
| {user_message}<|im_end|> | |
| <|im_start|>assistant | |
| ``` | |
| ## Recommended Inference Parameters | |
| ```yml | |
| penalty_alpha: 0.5 | |
| top_k: 4 | |
| repetition_penalty: 1.01 | |
| ``` | |
| ## Usage Example | |
| ```python | |
| from transformers import pipeline | |
| generate = pipeline("text-generation", "Felladrin/Llama-160M-Chat-v1") | |
| messages = [ | |
| { | |
| "role": "system", | |
| "content": "You are a helpful assistant who answers user's questions with details and curiosity.", | |
| }, | |
| { | |
| "role": "user", | |
| "content": "What are some potential applications for quantum computing?", | |
| }, | |
| ] | |
| prompt = generate.tokenizer.apply_chat_template( | |
| messages, tokenize=False, add_generation_prompt=True | |
| ) | |
| output = generate( | |
| prompt, | |
| max_new_tokens=1024, | |
| penalty_alpha=0.5, | |
| top_k=4, | |
| repetition_penalty=1.01, | |
| ) | |
| print(output[0]["generated_text"]) | |
| ``` | |
| ## [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) | |
| Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Felladrin__Llama-160M-Chat-v1) | |
| | Metric |Value| | |
| |---------------------------------|----:| | |
| |Avg. |30.27| | |
| |AI2 Reasoning Challenge (25-Shot)|24.74| | |
| |HellaSwag (10-Shot) |35.29| | |
| |MMLU (5-Shot) |26.13| | |
| |TruthfulQA (0-shot) |44.16| | |
| |Winogrande (5-shot) |51.30| | |
| |GSM8k (5-shot) | 0.00| | |