Hugging Face's logo

RichardErkhov
/
Felladrin_-_Llama-160M-Chat-v1-gguf

GGUF
Model card Files Community
Edit model card
YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-160M-Chat-v1 - GGUF

Name Quant method Size
Llama-160M-Chat-v1.Q2_K.gguf Q2_K 0.07GB
Llama-160M-Chat-v1.IQ3_XS.gguf IQ3_XS 0.07GB
Llama-160M-Chat-v1.IQ3_S.gguf IQ3_S 0.07GB
Llama-160M-Chat-v1.Q3_K_S.gguf Q3_K_S 0.07GB
Llama-160M-Chat-v1.IQ3_M.gguf IQ3_M 0.08GB
Llama-160M-Chat-v1.Q3_K.gguf Q3_K 0.08GB
Llama-160M-Chat-v1.Q3_K_M.gguf Q3_K_M 0.08GB
Llama-160M-Chat-v1.Q3_K_L.gguf Q3_K_L 0.08GB
Llama-160M-Chat-v1.IQ4_XS.gguf IQ4_XS 0.09GB
Llama-160M-Chat-v1.Q4_0.gguf Q4_0 0.09GB
Llama-160M-Chat-v1.IQ4_NL.gguf IQ4_NL 0.09GB
Llama-160M-Chat-v1.Q4_K_S.gguf Q4_K_S 0.09GB
Llama-160M-Chat-v1.Q4_K.gguf Q4_K 0.1GB
Llama-160M-Chat-v1.Q4_K_M.gguf Q4_K_M 0.1GB
Llama-160M-Chat-v1.Q4_1.gguf Q4_1 0.1GB
Llama-160M-Chat-v1.Q5_0.gguf Q5_0 0.11GB
Llama-160M-Chat-v1.Q5_K_S.gguf Q5_K_S 0.11GB
Llama-160M-Chat-v1.Q5_K.gguf Q5_K 0.11GB
Llama-160M-Chat-v1.Q5_K_M.gguf Q5_K_M 0.11GB
Llama-160M-Chat-v1.Q5_1.gguf Q5_1 0.12GB
Llama-160M-Chat-v1.Q6_K.gguf Q6_K 0.12GB
Llama-160M-Chat-v1.Q8_0.gguf Q8_0 0.16GB

Original model description:

language: - en license: apache-2.0 tags: - text-generation base_model: JackFram/llama-160m datasets: - ehartford/wizard_vicuna_70k_unfiltered - totally-not-an-llm/EverythingLM-data-V3 - Open-Orca/SlimOrca-Dedup - databricks/databricks-dolly-15k - THUDM/webglm-qa widget: - messages: - role: system content: You are a helpful assistant, who answers with empathy. - role: user content: Got a question for you! - role: assistant content: "Sure! What's it?" - role: user content: Why do you love cats so much!? 🐈 - messages: - role: system content: "You are a helpful assistant who answers user's questions with empathy." - role: user content: Who is Mona Lisa? - messages: - role: system content: You are a helpful assistant who provides concise responses. - role: user content: Heya! - role: assistant content: Hi! How may I help you today? - role: user content: I need to build a simple website. Where should I start learning about web development? - messages: - role: user content: Invited some friends to come home today. Give me some ideas for games to play with them! - messages: - role: system content: "You are a helpful assistant who answers user's questions with details and curiosity." - role: user content: What are some potential applications for quantum computing? - messages: - role: system content: You are a helpful assistant who gives creative responses. - role: user content: Write the specs of a game about mages in a fantasy world. - messages: - role: system content: "You are a helpful assistant who answers user's questions with details." - role: user content: Tell me about the pros and cons of social media. - messages: - role: system content: "You are a helpful assistant who answers user's questions with confidence." - role: user content: What is a dog? - role: assistant content: 'A dog is a four-legged, domesticated animal that is a member of the class Mammalia, which includes all mammals. Dogs are known for their loyalty, playfulness, and ability to be trained for various tasks. They are also used for hunting, herding, and as service animals.' - role: user content: What is the color of an apple? inference: parameters: max_new_tokens: 250 penalty_alpha: 0.5 top_k: 4 repetition_penalty: 1.01 model-index: - name: Llama-160M-Chat-v1 results: - task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics: - type: acc_norm value: 24.74 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics: - type: acc_norm value: 35.29 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics: - type: acc value: 26.13 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics: - type: mc2 value: 44.16 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics: - type: acc value: 51.3 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 0.0 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 name: Open LLM Leaderboard

A Llama Chat Model of 160M Parameters

Recommended Prompt Format 

<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant

Recommended Inference Parameters 

penalty_alpha: 0.5
top_k: 4
repetition_penalty: 1.01

Usage Example 

from transformers import pipeline

generate = pipeline("text-generation", "Felladrin/Llama-160M-Chat-v1")

messages = [
    {
        "role": "system",
        "content": "You are a helpful assistant who answers user's questions with details and curiosity.",
    },
    {
        "role": "user",
        "content": "What are some potential applications for quantum computing?",
    },
]

prompt = generate.tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

output = generate(
    prompt,
    max_new_tokens=1024,
    penalty_alpha=0.5,
    top_k=4,
    repetition_penalty=1.01,
)

print(output[0]["generated_text"])

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 30.27
AI2 Reasoning Challenge (25-Shot) 24.74
HellaSwag (10-Shot) 35.29
MMLU (5-Shot) 26.13
TruthfulQA (0-shot) 44.16
Winogrande (5-shot) 51.30
GSM8k (5-shot) 0.00
Downloads last month
586
GGUF
Model size
162M params
Architecture
llama

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

View +2 files
Unable to determine this model's library. Check the docs .