xezpeleta commited on
Commit
543981a
1 Parent(s): 76aa437

Update dataset name

Browse files
Files changed (1) hide show
  1. README.md +2 -79
README.md CHANGED
@@ -16,91 +16,14 @@ model-index:
16
  - name: data/openllama-3b-basque
17
  results: []
18
  datasets:
19
- - xezpeleta/oasst-eu-llama2-chat
20
  language:
21
  - eu
22
  ---
23
 
24
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
25
- should probably proofread and complete it, then remove this comment. -->
26
-
27
- [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
28
- <details><summary>See axolotl config</summary>
29
-
30
- axolotl version: `0.3.0`
31
- ```yaml
32
- base_model: openlm-research/open_llama_3b_v2
33
- model_type: LlamaForCausalLM
34
- tokenizer_type: LlamaTokenizer
35
- load_in_8bit: false
36
- load_in_4bit: false
37
- strict: false
38
- hf_use_auth_token: true
39
- push_dataset_to_hub:
40
- datasets:
41
- - path: xezpeleta/oasst-eu-llama2-chat
42
- type: completion
43
- conversation: llama2
44
- dataset_prepared_path:
45
- val_set_size: 0.02
46
- adapter:
47
- lora_model_dir:
48
- sequence_len: 1024
49
- sample_packing: true
50
- lora_r:
51
- lora_alpha:
52
- lora_dropout:
53
- lora_target_modules:
54
- lora_target_linear:
55
- lora_fan_in_fan_out:
56
- wandb_project: openllama-3b-basque
57
- wandb_entity:
58
- wandb_watch:
59
- wandb_name: openllama-3b-basque-v1
60
- wandb_log_model:
61
- output_dir: /data/openllama-3b-basque
62
- gradient_accumulation_steps: 1
63
- micro_batch_size: 1
64
- num_epochs: 4
65
- optimizer: adamw_bnb_8bit
66
- torchdistx_path:
67
- lr_scheduler: cosine
68
- learning_rate: 0.000003
69
- train_on_inputs: false
70
- group_by_length: false
71
- float16: true
72
- bf16: false
73
- fp16: false
74
- tf32: false
75
- gradient_checkpointing: true
76
- early_stopping_patience:
77
- resume_from_checkpoint:
78
- local_rank:
79
- logging_steps: 1
80
- xformers_attention:
81
- flash_attention: true
82
- gptq_groupsize:
83
- gptq_model_v1:
84
- warmup_steps: 20
85
- evals_per_epoch: 4
86
- saves_per_epoch: 1
87
- debug:
88
- deepspeed:
89
- weight_decay: 0.1
90
- fsdp:
91
- fsdp_config:
92
- special_tokens:
93
- bos_token: "<s>"
94
- eos_token: "</s>"
95
- unk_token: "<unk>"
96
-
97
- ```
98
-
99
- </details><br>
100
-
101
  # data/openllama-3b-basque
102
 
103
- This model is a fine-tuned version of [openlm-research/open_llama_3b_v2](https://huggingface.co/openlm-research/open_llama_3b_v2) on the None dataset.
104
  It achieves the following results on the evaluation set:
105
  - Loss: 2.1151
106
 
 
16
  - name: data/openllama-3b-basque
17
  results: []
18
  datasets:
19
+ - xezpeleta/oasst1_eu_threads
20
  language:
21
  - eu
22
  ---
23
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
  # data/openllama-3b-basque
25
 
26
+ This model is a fine-tuned version of [openlm-research/open_llama_3b_v2](https://huggingface.co/openlm-research/open_llama_3b_v2) on the [xezpeleta/oasst1_eu_threads](https://huggingface.co/datasets/xezpeleta/oasst1_eu_threads) dataset.
27
  It achieves the following results on the evaluation set:
28
  - Loss: 2.1151
29