ntropydev/ntropy-labeler-llama-lora-13b

This repo contains a low-rank adapter for LLaMA-13b finetuned on Ntropy proprietary dataset (consumer financial transactions).

This version of the weights was trained with the following hyperparameters:

• Base Model: decapoda-research/llama-13b-hf
• Epochs: 10 (load from best epoch)
• Batch size: 16
• Cutoff length: 1024
• Learning rate: 3e-4
• Lora r: 16
• Lora target modules: q_proj, k_proj, v_proj, o_proj

Instructions for running the adapter can be found at https://github.com/ntropy-network/enrichment_models/blob/main/notebooks/llama.ipynb