Is fine-tuning of already quanted GPT real?
Does anyone know if it's possible to fine-tune GPTQ 4bit quantized model?
I tried to do it via transformers.Trainer, but got malformed tensors with Infs or Nans...
Got
RuntimeError: probability tensor contains either inf, nan or element < 0
tried with this script:
train_path = 'train_dataset.txt'
train_dataset = TextDataset(tokenizer=tokenizer,file_path=train_path,block_size=64)
data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer,
mlm=False)
from transformers import Trainer, TrainingArguments
training_args = TrainingArguments(
output_dir="./finetuned2", # The output directory
overwrite_output_dir=True, # Overwrite the content of the output dir
num_train_epochs=0.01, # number of training epochs
per_device_train_batch_size=1, # batch size for training
auto_find_batch_size=True,
per_device_eval_batch_size=1, # batch size for evaluation
warmup_steps=10, # number of warmup steps for learning rate scheduler
gradient_accumulation_steps=1, # to make "virtual" batch size larger
)
trainer = Trainer(
model=model,
args=training_args,
data_collator=data_collator,
train_dataset=train_dataset,
optimizers = (torch.optim.AdamW(model.parameters(),lr=1e-5), None)
)
trainer.train()
Hello! training with lora is possible, look at the peft documentation - https://github.com/huggingface/peft