Training procedure

The following bitsandbytes quantization config was used during training:

  • quant_method: gptq
  • bits: 4
  • tokenizer: None
  • dataset: None
  • group_size: 128
  • damp_percent: 0.01
  • desc_act: False
  • sym: True
  • true_sequential: True
  • use_cuda_fp16: True
  • model_seqlen: None
  • block_name_to_quantize: None
  • module_name_preceding_first_block: None
  • batch_size: 1
  • pad_token_id: None
  • disable_exllama: True
  • max_input_length: None

The following bitsandbytes quantization config was used during training:

  • quant_method: gptq
  • bits: 4
  • tokenizer: None
  • dataset: None
  • group_size: 128
  • damp_percent: 0.01
  • desc_act: False
  • sym: True
  • true_sequential: True
  • use_cuda_fp16: True
  • model_seqlen: None
  • block_name_to_quantize: None
  • module_name_preceding_first_block: None
  • batch_size: 1
  • pad_token_id: None
  • disable_exllama: True
  • max_input_length: None

The following bitsandbytes quantization config was used during training:

  • quant_method: bitsandbytes
  • load_in_8bit: False
  • load_in_4bit: True
  • llm_int8_threshold: 6.0
  • llm_int8_skip_modules: None
  • llm_int8_enable_fp32_cpu_offload: False
  • llm_int8_has_fp16_weight: False
  • bnb_4bit_quant_type: fp4
  • bnb_4bit_use_double_quant: False
  • bnb_4bit_compute_dtype: float16

The following bitsandbytes quantization config was used during training:

  • quant_method: bitsandbytes
  • load_in_8bit: False
  • load_in_4bit: True
  • llm_int8_threshold: 6.0
  • llm_int8_skip_modules: None
  • llm_int8_enable_fp32_cpu_offload: False
  • llm_int8_has_fp16_weight: False
  • bnb_4bit_quant_type: fp4
  • bnb_4bit_use_double_quant: False
  • bnb_4bit_compute_dtype: float16

The following bitsandbytes quantization config was used during training:

  • quant_method: bitsandbytes
  • load_in_8bit: False
  • load_in_4bit: True
  • llm_int8_threshold: 6.0
  • llm_int8_skip_modules: None
  • llm_int8_enable_fp32_cpu_offload: False
  • llm_int8_has_fp16_weight: False
  • bnb_4bit_quant_type: fp4
  • bnb_4bit_use_double_quant: False
  • bnb_4bit_compute_dtype: float16

The following bitsandbytes quantization config was used during training:

  • quant_method: gptq
  • bits: 4
  • tokenizer: None
  • dataset: None
  • group_size: 128
  • damp_percent: 0.01
  • desc_act: False
  • sym: True
  • true_sequential: True
  • use_cuda_fp16: False
  • model_seqlen: None
  • block_name_to_quantize: None
  • module_name_preceding_first_block: None
  • batch_size: 1
  • pad_token_id: None
  • disable_exllama: True
  • max_input_length: None

The following bitsandbytes quantization config was used during training:

  • quant_method: gptq
  • bits: 4
  • tokenizer: None
  • dataset: None
  • group_size: 128
  • damp_percent: 0.01
  • desc_act: False
  • sym: True
  • true_sequential: True
  • use_cuda_fp16: False
  • model_seqlen: None
  • block_name_to_quantize: None
  • module_name_preceding_first_block: None
  • batch_size: 1
  • pad_token_id: None
  • disable_exllama: True
  • max_input_length: None

Framework versions

  • PEFT 0.5.0

  • PEFT 0.5.0

  • PEFT 0.5.0

  • PEFT 0.5.0

  • PEFT 0.5.0

  • PEFT 0.5.0

  • PEFT 0.5.0

Downloads last month
2
Inference API
Unable to determine this model’s pipeline type. Check the docs .