richardcsuwandi commited on
Commit
118fa06
·
verified ·
1 Parent(s): d167ea8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -13
README.md CHANGED
@@ -13,19 +13,7 @@ This model is a fine-tuned adaptation of [Llama-2-7b-chat-hf](https://huggingfac
13
 
14
  ## Training
15
 
16
- The model was fine-tuned on a dataset translated into Javanese using the [NLLB](https://ai.meta.com/research/no-language-left-behind/) model. This dataset includes texts from both [OASST1](https://huggingface.co/datasets/OpenAssistant/oasst1) and [OASST2](https://huggingface.co/datasets/OpenAssistant/oasst2), covering a wide range of conversational scenarios. The training process employed [TRL](https://github.com/huggingface/trl) and [PEFT](https://github.com/huggingface/peft) to facilitate efficient and rapid fine-tuning.
17
-
18
- The following `bitsandbytes` quantization settings were applied during training:
19
- - quant_method: bitsandbytes
20
- - load_in_8bit: False
21
- - load_in_4bit: True
22
- - llm_int8_threshold: 6.0
23
- - llm_int8_skip_modules: None
24
- - llm_int8_enable_fp32_cpu_offload: False
25
- - llm_int8_has_fp16_weight: False
26
- - bnb_4bit_quant_type: fp4
27
- - bnb_4bit_use_double_quant: False
28
- - bnb_4bit_compute_dtype: float32
29
 
30
  ## Usage
31
 
 
13
 
14
  ## Training
15
 
16
+ The model was fine-tuned on a dataset translated into Javanese using the [NLLB](https://ai.meta.com/research/no-language-left-behind/) model. This dataset includes texts from both [OASST1](https://huggingface.co/datasets/OpenAssistant/oasst1) and [OASST2](https://huggingface.co/datasets/OpenAssistant/oasst2), covering a wide range of conversational scenarios. The training process employed [PEFT](https://github.com/huggingface/peft) and [TRL](https://github.com/huggingface/trl) to facilitate efficient and rapid fine-tuning.
 
 
 
 
 
 
 
 
 
 
 
 
17
 
18
  ## Usage
19