README.md

metadata

license: llama2
base_model: migtissera/Tess-70B-v1.6
metrics:
  - accuracy
inference: false
model-index:
  - name: Covasna-0.1
    results:
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: AI2 Reasoning Challenge (25-Shot)
          type: ai2_arc
          config: ARC-Challenge
          split: test
          args:
            num_few_shot: 25
        metrics:
          - type: acc_norm
            value: 48.81
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Mihaiii/Covasna-0.1
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: HellaSwag (10-Shot)
          type: hellaswag
          split: validation
          args:
            num_few_shot: 10
        metrics:
          - type: acc_norm
            value: 70.07
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Mihaiii/Covasna-0.1
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MMLU (5-Shot)
          type: cais/mmlu
          config: all
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 61.9
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Mihaiii/Covasna-0.1
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: TruthfulQA (0-shot)
          type: truthful_qa
          config: multiple_choice
          split: validation
          args:
            num_few_shot: 0
        metrics:
          - type: mc2
            value: 52.64
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Mihaiii/Covasna-0.1
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: Winogrande (5-shot)
          type: winogrande
          config: winogrande_xl
          split: validation
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 70.8
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Mihaiii/Covasna-0.1
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: GSM8k (5-shot)
          type: gsm8k
          config: main
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 0.99
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Mihaiii/Covasna-0.1
          name: Open LLM Leaderboard

This is a BF16 and pruned version of migtissera/Tess-70B-v1.6 .

migtissera/Tess-70B-v1.6 has 69 billion params and Covasna-0.1 has 41.6 billion (~60.3% param size)

Steps to replicate:

Use laserQlora.ipynb from cognitivecomputations/laserRMT to determine which layers should be eliminated.

Adapt the script for migtissera/Tess-70B-v1.6 by replacing model_name = "mistralai/Mistral-7B-v0.1" with model_name = "migtissera/Tess-70B-v1.6" and layer_numbers = list(range(31, -1, -1)) with layer_numbers = list(range(79, -1, -1)), 79 being the last recurrent layer index Tess-70B-v1.6 has.

Then look for the layer indexes where self_attn.v_proj snr is Infinity and eliminate those layers using mergekit.

Here is the mergekit config:

slices:
  - sources:
    - model: "migtissera/Tess-70B-v1.6"
      layer_range: [0, 7]
  - sources:
    - model: "migtissera/Tess-70B-v1.6"
      layer_range: [8, 9]
  - sources:
    - model: "migtissera/Tess-70B-v1.6"
      layer_range: [12, 29]
  - sources:
    - model: "migtissera/Tess-70B-v1.6"
      layer_range: [31, 32]
  - sources:
    - model: "migtissera/Tess-70B-v1.6"
      layer_range: [33, 45]
  - sources:
    - model: "migtissera/Tess-70B-v1.6"
      layer_range: [50, 52]
  - sources:
    - model: "migtissera/Tess-70B-v1.6"
      layer_range: [60, 61]
  - sources:
    - model: "migtissera/Tess-70B-v1.6"
      layer_range: [67, 68]
  - sources:
    - model: "migtissera/Tess-70B-v1.6"
      layer_range: [74, 80]
merge_method: passthrough
dtype: bfloat16

GGUF: Covasna-0.1-GGUF

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	50.87
AI2 Reasoning Challenge (25-Shot)	48.81
HellaSwag (10-Shot)	70.07
MMLU (5-Shot)	61.90
TruthfulQA (0-shot)	52.64
Winogrande (5-shot)	70.80
GSM8k (5-shot)	0.99