--- quantized_by: bartowski --- # Exllama v2 Quantizations of Lelantos-7B at 8.0 bits per weight Using turboderp's ExLlamaV2 v0.0.11 for quantization. Conversion was done using the default calibration dataset. Original model: https://huggingface.co/SanjiWatsuki/Lelantos-7B ## Download instructions With git: ```shell git clone --single-branch --branch 8.0 https://huggingface.co/bartowski/Lelantos-7B-exl2 ``` With huggingface hub (credit to TheBloke for instructions): ```shell pip3 install huggingface-hub ``` To download from a different branch, add the `--revision` parameter: ```shell mkdir Lelantos-7B-exl2 huggingface-cli download bartowski/Lelantos-7B-exl2 --revision 8_0 --local-dir Lelantos-7B-exl2 --local-dir-use-symlinks False ```