Uploaded model

  • Developed by: mariana-coelho-9
  • License: apache-2.0
  • Finetuned from model : unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

Dataset Information

Dataset used: mariana-coelho-9/icd-10

  • Training set size: 28261
  • Validation set size: 3141

Memory and Time Statistics

GPU = Tesla T4.

  • Max memory = 14.748 GB.
  • Peak reserved memory = 7.717 GB.
  • Peak reserved memory for training = 0.0 GB.
  • Peak reserved memory % of max memory = 52.326 %.
  • Peak reserved memory for training % of max memory = 0.0 %.
  • 2033.5873 seconds used for training.
  • 33.89 minutes used for training.

Hyperparameters

Hyperparameters:

  • 'r': 16,
  • 'target_modules': ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj']
  • 'lora_alpha': 16
  • 'lora_dropout': 0
  • 'bias': 'none'
  • 'use_gradient_checkpointing': 'unsloth'
  • 'random_state': 3407
  • 'use_rslora': False
  • 'loftq_config': None
  • 'per_device_train_batch_size': 2
  • 'gradient_accumulation_steps': 4
  • 'warmup_steps': 5, 'max_steps': 60
  • 'learning_rate': 0.0002
  • 'fp16': True
  • 'bf16': False
  • 'logging_steps': 10
  • 'optim': 'adamw_8bit'
  • 'weight_decay': 0.01
  • 'lr_scheduler_type': 'linear'
  • 'seed': 3407
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no pipeline_tag.

Model tree for mariana-coelho-9/lora-adapter-Llama-3.1-8B-Instruct-bnb-4bit-icd-10

Dataset used to train mariana-coelho-9/lora-adapter-Llama-3.1-8B-Instruct-bnb-4bit-icd-10