metadata
language:
- en
pipeline_tag: text-generation
library_name: ExLlamaV2
tags:
- llama
- llama-3
license: other
license_name: llama3
license_link: https://huggingface.co/meta-llama/Meta-Llama-3-8B/blob/main/LICENSE
base_model: R136a1/BeyondInfinity-4x7B
Exl2 quants for L3-8B-Niitama-v1
Automatically quantized using the auto quant script from hf-scripts
Niitama is a model created by Sao10k. There's currently no card for this model other than the fact it was a horde model.
BPW:
How to download:
oobabooga's downloader
use something like download-model.py to download with python requests.
Install requirements:
pip install requests tqdm
Example for downloading 8bpw:
python download-model.py Anthonyg5005/L3-8B-Niitama-v1-exl2-exl2:8.0bpw
huggingface-cli
You may also use huggingface-cli
To install it, install python hf-hub
pip install huggingface-hub
Example for 8bpw:
huggingface-cli download Anthonyg5005/L3-8B-Niitama-v1-exl2 --local-dir L3-8B-Niitama-v1-exl2-8bpw --revision 8.0bpw
Git LFS (not recommended)
I would recommend the http downloaders over using git, they can resume downloads if failed and are much easier to work with.
Make sure to have git and git LFS installed.
Example for 8bpw download with git:
Have LFS file skip disabled
# windows
set GIT_LFS_SKIP_SMUDGE=0
# linux
export GIT_LFS_SKIP_SMUDGE=0
Clone repo branch
git clone https://huggingface.co/Anthonyg5005/L3-8B-Niitama-v1-exl2 -b 8.0bpw