---
library_name: transformers
base_model:
- AI-Sweden-Models/gpt-sw3-6.7b-v2
datasets:
- skvarre/sv-instruct-v1
---

## Model Details

Finetune of [gpt-sw3-6.7b-v2](https://huggingface.co/AI-Sweden-Models/gpt-sw3-6.7b-v2) using **LoRA** with 4-bit quantization. Adapters merged with base model, however with ```bfloat16``` precision tensors. 

## Usage:

This is a finetune experiment. How to use will be provided later. 

## Evaluation 
[ScandEval](https://scandeval.com/swedish-nlg/) benchmarks:
| Dataset      | Performance (Metric 1 / Metric 2)    |
|:------------:|:------------------------------------:|
|    swerec    | 74.95 ± 1.17 / 61.38 ± 1.37          |
|     suc3     | 30.75 ± 4.11 / 25.69 ± 4.83          |
|   scala-sv   | 8.96 ± 2.09 / 51.50 ± 2.94           |
|  scandiqa-sv | 50.71 ± 0.99 / 56.76 ± 0.89          |
|     swedn    | 64.37 ± 0.72 / 18.25 ± 0.29          |
|    mmlu-sv   | 5.45 ± 0.91 / 28.14 ± 0.82           |
| hellaswag-sv | 27.95 ± 0.73 / 4.19 ± 0.94           |
|     speed    | 5322.20 ± 1132.75 / 1280.06 ± 408.08 |