Edit model card

This is a merge of mpasila/Finnish-Alpaca-Tiny-V2-LoRA-7B.

LoRA trained in 4-bit with 2k context using LumiOpen/Viking-7B as the base model for 1 epoch.

Dataset used is mpasila/Finnish-Alpaca-Tiny.

It works relatively well for question and answering. I will make a bigger dataset for the next fine-tune.

Prompt format: Alpaca

It uses Alpaca format but with a translated instruction at the start:

{
    "instruction,output": "Alla on ohje, jossa kuvataan tehtävä. Kirjoita vastaus, joka täyttää pyynnön asianmukaisesti.\n\n### Instruction:\n%instruction%\n\n### Response:\n%output%",
    "instruction,input,output": "Alla on ohje, jossa kuvataan tehtävä ja joka on yhdistetty kontekstia lisäävään syötteeseen. Kirjoita vastaus, joka täyttää pyynnön asianmukaisesti.\n\n### Instruction:\n%instruction%\n\n### Input:\n%input%\n\n### Response:\n%output%"
}

Evaluation

Model Size Type FIN-bench (score)
mpasila/Finnish-Alpaca-Tiny-V2-7B 7B Instruct 0.4654
mpasila/Finnish-Alpaca-Small-7B 7B Instruct 0.3586
mpasila/Alpacazord-Viking-7B 7B Instruct 0.4123
mpasila/NordicAlpaca-Finnish-V1-7B 7B Instruct 0.3891
mpasila/Finnish-Viking-Alpaca-V1-7B 7B Instruct 0.3943
Finnish-NLP/llama-7b-finnish-instruct-v0.1 7B Instruct 0.4365
Finnish-NLP/llama-7b-finnish-instruct-v0.2 7B Instruct 0.3993
Finnish-NLP/llama-7b-finnish 7B Base 0.2350
LumiOpen/Viking-7B (1000B) 7B Base 0.3721
HPLT/gpt-7b-nordic-prerelease 7B Base 0.3169

Source

FIN-bench scores:

Task Version Metric Value Stderr
bigbench_analogies 0 multiple_choice_grade 0.6385 ± 0.0423
bigbench_arithmetic_1_digit_addition 0 multiple_choice_grade 0.7200 ± 0.0451
bigbench_arithmetic_1_digit_division 0 multiple_choice_grade 0.7391 ± 0.0936
bigbench_arithmetic_1_digit_multiplication 0 multiple_choice_grade 0.4800 ± 0.0502
bigbench_arithmetic_1_digit_subtraction 0 multiple_choice_grade 0.6300 ± 0.0485
bigbench_arithmetic_2_digit_addition 0 multiple_choice_grade 0.4000 ± 0.0492
bigbench_arithmetic_2_digit_division 0 multiple_choice_grade 0.5000 ± 0.0503
bigbench_arithmetic_2_digit_multiplication 0 multiple_choice_grade 0.2800 ± 0.0451
bigbench_arithmetic_2_digit_subtraction 0 multiple_choice_grade 0.4300 ± 0.0498
bigbench_arithmetic_3_digit_addition 0 multiple_choice_grade 0.5800 ± 0.0496
bigbench_arithmetic_3_digit_division 0 multiple_choice_grade 0.3100 ± 0.0465
bigbench_arithmetic_3_digit_multiplication 0 multiple_choice_grade 0.2900 ± 0.0456
bigbench_arithmetic_3_digit_subtraction 0 multiple_choice_grade 0.5100 ± 0.0502
bigbench_arithmetic_4_digit_addition 0 multiple_choice_grade 0.5300 ± 0.0502
bigbench_arithmetic_4_digit_division 0 multiple_choice_grade 0.3900 ± 0.0490
bigbench_arithmetic_4_digit_multiplication 0 multiple_choice_grade 0.3100 ± 0.0465
bigbench_arithmetic_4_digit_subtraction 0 multiple_choice_grade 0.6200 ± 0.0488
bigbench_arithmetic_5_digit_addition 0 multiple_choice_grade 0.6500 ± 0.0479
bigbench_arithmetic_5_digit_division 0 multiple_choice_grade 0.3200 ± 0.0469
bigbench_arithmetic_5_digit_multiplication 0 multiple_choice_grade 0.3000 ± 0.0461
bigbench_arithmetic_5_digit_subtraction 0 multiple_choice_grade 0.6400 ± 0.0482
bigbench_cause_and_effect_one_sentence 0 multiple_choice_grade 0.5686 ± 0.0700
bigbench_cause_and_effect_one_sentence_no_prompt 0 multiple_choice_grade 0.6471 ± 0.0676
bigbench_cause_and_effect_two_sentences 0 multiple_choice_grade 0.4314 ± 0.0700
bigbench_emotions 0 multiple_choice_grade 0.2250 ± 0.0331
bigbench_empirical_judgments 0 multiple_choice_grade 0.2525 ± 0.0439
bigbench_general_knowledge 0 multiple_choice_grade 0.3429 ± 0.0571
bigbench_hhh_alignment_harmless 0 multiple_choice_grade 0.3793 ± 0.0643
bigbench_hhh_alignment_helpful 0 multiple_choice_grade 0.3390 ± 0.0622
bigbench_hhh_alignment_honest 0 multiple_choice_grade 0.3729 ± 0.0635
bigbench_hhh_alignment_other 0 multiple_choice_grade 0.5349 ± 0.0770
bigbench_intent_recognition 0 multiple_choice_grade 0.2153 ± 0.0156
bigbench_misconceptions 0 multiple_choice_grade 0.5224 ± 0.0433
bigbench_paraphrase 0 multiple_choice_grade 0.4750 ± 0.0354
bigbench_sentence_ambiguity 0 multiple_choice_grade 0.4833 ± 0.0651
bigbench_similarities_abstraction 0 multiple_choice_grade 0.6974 ± 0.0530

Uploaded model

  • Developed by: mpasila
  • License: apache-2.0
  • Finetuned from model : LumiOpen/Viking-7B

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

Downloads last month
3
Safetensors
Model size
7.55B params
Tensor type
BF16
·
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Finetuned from

Dataset used to train mpasila/Finnish-Alpaca-Tiny-V2-7B

Collection including mpasila/Finnish-Alpaca-Tiny-V2-7B