|
--- |
|
language: en |
|
--- |
|
|
|
# dummy-CodeLlama-7b-hf |
|
|
|
This is a dummy version of the model based on [`codellama/CodeLlama-7b-hf`](https://huggingface.co/codellama/CodeLlama-7b-hf). |
|
|
|
## 🧩 Dummy |
|
|
|
`dummy-CodeLlama-7b-hf` has a size of 888.04 MB instead of the original 12852.88 MB (compression factor of 14.47) but keeps the base model's functionality. |
|
|
|
The purpose of this dummy version is to be used for **debugging**, so you don't have to download the entire original model. Do not use it for inference. |
|
|
|
## 💻 Usage |
|
|
|
```python |
|
# pip install transformers accelerate |
|
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
import torch |
|
|
|
model = "dummy-CodeLlama-7b-hf" |
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model) |
|
model = AutoModelForCausalLM.from_pretrained( |
|
model, |
|
low_cpu_mem_usage=True, |
|
return_dict=True, |
|
torch_dtype=torch.float16, |
|
device_map={"": 0}, |
|
) |
|
``` |