README.md

metadata

language:
  - en
license: mit
tags:
  - generated_from_trainer
  - llama-cpp
  - gguf-my-repo
base_model: mistralai/Mistral-7B-v0.1
datasets:
  - HuggingFaceH4/ultrachat_200k
  - HuggingFaceH4/ultrafeedback_binarized
widget:
  - example_title: Pirate!
    messages:
      - role: system
        content: You are a pirate chatbot who always responds with Arr!
      - role: user
        content: There's a llama on my lawn, how can I get rid of him?
    output:
      text: >-
        Arr! 'Tis a puzzlin' matter, me hearty! A llama on yer lawn be a rare
        sight, but I've got a plan that might help ye get rid of 'im. Ye'll need
        to gather some carrots and hay, and then lure the llama away with the
        promise of a tasty treat. Once he's gone, ye can clean up yer lawn and
        enjoy the peace and quiet once again. But beware, me hearty, for there
        may be more llamas where that one came from! Arr!
pipeline_tag: text-generation
model-index:
  - name: zephyr-7b-beta
    results:
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: AI2 Reasoning Challenge (25-Shot)
          type: ai2_arc
          config: ARC-Challenge
          split: test
          args:
            num_few_shot: 25
        metrics:
          - type: acc_norm
            value: 62.03071672354948
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=HuggingFaceH4/zephyr-7b-beta
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: HellaSwag (10-Shot)
          type: hellaswag
          split: validation
          args:
            num_few_shot: 10
        metrics:
          - type: acc_norm
            value: 84.35570603465445
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=HuggingFaceH4/zephyr-7b-beta
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: Drop (3-Shot)
          type: drop
          split: validation
          args:
            num_few_shot: 3
        metrics:
          - type: f1
            value: 9.66243708053691
            name: f1 score
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=HuggingFaceH4/zephyr-7b-beta
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: TruthfulQA (0-shot)
          type: truthful_qa
          config: multiple_choice
          split: validation
          args:
            num_few_shot: 0
        metrics:
          - type: mc2
            value: 57.44916942762855
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=HuggingFaceH4/zephyr-7b-beta
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: GSM8k (5-shot)
          type: gsm8k
          config: main
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 12.736921910538287
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=HuggingFaceH4/zephyr-7b-beta
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MMLU (5-Shot)
          type: cais/mmlu
          config: all
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 61.07
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=HuggingFaceH4/zephyr-7b-beta
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: Winogrande (5-shot)
          type: winogrande
          config: winogrande_xl
          split: validation
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 77.7426992896606
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=HuggingFaceH4/zephyr-7b-beta
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: AlpacaEval
          type: tatsu-lab/alpaca_eval
        metrics:
          - type: unknown
            value: 0.906
            name: win rate
        source:
          url: https://tatsu-lab.github.io/alpaca_eval/
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MT-Bench
          type: unknown
        metrics:
          - type: unknown
            value: 7.34
            name: score
        source:
          url: https://huggingface.co/spaces/lmsys/mt-bench

newsletter/zephyr-7b-beta-Q6_K-GGUF

This model was converted to GGUF format from HuggingFaceH4/zephyr-7b-beta using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

Use with llama.cpp

Install llama.cpp through brew.

brew install ggerganov/ggerganov/llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo newsletter/zephyr-7b-beta-Q6_K-GGUF --model zephyr-7b-beta.Q6_K.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo newsletter/zephyr-7b-beta-Q6_K-GGUF --model zephyr-7b-beta.Q6_K.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

git clone https://github.com/ggerganov/llama.cpp &&             cd llama.cpp &&             make &&             ./main -m zephyr-7b-beta.Q6_K.gguf -n 128