Choice of QLora parameter value

by nlux - opened Apr 15

nlux

Apr 15

Hi, thank you for your great work. Is there a reason you choose 16 for r and alpha in QLora? According to https://lightning.ai/pages/community/lora-insights/ r=256, alpha=512 gave the best results.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment