README.md

metadata

license: apache-2.0
datasets:
  - togethercomputer/RedPajama-Data-1T-Sample
language:
  - en

Landmark Attention LLaMA 33B

This model has been trained using the PEFT LoRA method using the Landmark Attention method over 200 steps. Model will likely be trained further and updated later on.

Usage

Unlikely to be usable with the popular frontends (e.g. KoboldAI and Oobabooga) due to the lack of support for landmark tokens.