Landmark Attention LLaMA 33B
This model has been trained using the PEFT LoRA technique with the Landmark Attention method over 200 steps. Model will likely be trained further and updated later on.
Usage
Requires trust_remote_code to be set to True. In oobabooga, you can simply add the --trust_remote_code flag.
You will also need to disable the Add the bos_token to the beginning of prompts option in the settings.
PEFT Checkpoint
You can probably merge the checkpoint with any other LLaMA-based model (provided they're 33B, of course). This repo contains the merged weights, but you can grab the adapter here.
Training Code
You can find the training code here.
- Downloads last month
- 21
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.