Quantization made by Richard Erkhov.
temp-1-distilled-code-llama - EXL2
- Model creator: https://huggingface.co/anudaw/
- Original model: https://huggingface.co/anudaw/temp-1-distilled-code-llama/
Available sizes
| Branch | Bits | Description | | ----- | ---- | ------- | ------ | ------ | ------ | ------ | ------------ | | 8_0 | 8.0 | Maximum quality that ExLlamaV2 can produce, near unquantized performance. | | 6_5 | 6.5 | Very similar to 8.0, good tradeoff of size vs performance, recommended. | | 5_0 | 5.0 | Slightly lower quality vs 6.5, but usable on 8GB cards. | | 4_25 | 4.25 | GPTQ equivalent bits per weight, slightly higher quality. | | 3_5 | 3.5 | Lower quality, only use if you have to. |
Download instructions
With git:
git clone --single-branch --branch 6_5 https://huggingface.co/anudaw_-_temp-1-distilled-code-llama-exl2 temp-1-distilled-code-llama-6_5
With huggingface hub:
pip3 install huggingface-hub
To download a specific branch, use the --revision parameter. For example, to download the 6.5 bpw branch:
Linux:
huggingface-cli download anudaw_-_temp-1-distilled-code-llama-exl2 --revision 6_5 --local-dir temp-1-distilled-code-llama-6_5 --local-dir-use-symlinks False
Windows (which apparently doesn't like _ in folders sometimes?):
huggingface-cli download anudaw_-_temp-1-distilled-code-llama-exl2 --revision 6_5 --local-dir temp-1-distilled-code-llama-6.5 --local-dir-use-symlinks False
Original model description:
license: apache-2.0 base_model: anudaw/temp-1-distilled-code-llama tags: - trl - sft - generated_from_trainer model-index: - name: temp-1-distilled-code-llama results: []
temp-1-distilled-code-llama
This model is a fine-tuned version of anudaw/temp-1-distilled-code-llama on an unknown dataset.
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 32
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- lr_scheduler_warmup_ratio: 0.03
- num_epochs: 5
Framework versions
- Transformers 4.40.1
- Pytorch 2.3.0+cu121
- Datasets 2.19.0
- Tokenizers 0.19.1