---
license: apache-2.0
base_model: openlm-research/open_llama_3b_v2
tags:
- generated_from_trainer
- finetune
- oasst
- basque
- euskara
- euskera
- llama
- openllama
- axolotl
- chat
model-index:
- name: data/openllama-3b-basque
  results: []
datasets:
- xezpeleta/oasst1_eu_threads
language:
- eu
---

# data/openllama-3b-basque

This model is a fine-tuned version of [openlm-research/open_llama_3b_v2](https://huggingface.co/openlm-research/open_llama_3b_v2) on the [xezpeleta/oasst1_eu_threads](https://huggingface.co/datasets/xezpeleta/oasst1_eu_threads) dataset.
It achieves the following results on the evaluation set:
- Loss: 2.1151

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-06
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 20
- num_epochs: 4

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 3.4011        | 0.0   | 1     | 3.5208          |
| 2.4173        | 0.25  | 917   | 2.6480          |
| 2.4849        | 0.5   | 1834  | 2.4465          |
| 2.2639        | 0.75  | 2751  | 2.3302          |
| 2.4403        | 1.0   | 3668  | 2.2548          |
| 2.1016        | 1.23  | 4585  | 2.2111          |
| 2.0178        | 1.48  | 5502  | 2.1805          |
| 2.041         | 1.73  | 6419  | 2.1476          |
| 2.0026        | 1.98  | 7336  | 2.1347          |
| 1.7586        | 2.22  | 8253  | 2.1187          |
| 1.8564        | 2.47  | 9170  | 2.1191          |
| 1.766         | 2.72  | 10087 | 2.1147          |
| 2.0431        | 2.97  | 11004 | 2.1157          |
| 2.3418        | 3.2   | 11921 | 2.1147          |
| 1.9713        | 3.45  | 12838 | 2.1151          |
| 1.9072        | 3.7   | 13755 | 2.1151          |


### Framework versions

- Transformers 4.37.0.dev0
- Pytorch 2.0.1+cu118
- Datasets 2.16.1
- Tokenizers 0.15.0