---
license: mit
---

A Llama3 based model fine-tuned for cybersecurity domain. 

The model was finetuned over https://huggingface.co/datasets/unibuc-cs/CyberGuardianDataset.
While the Llama3 original perplexity on the dataset started ~21.9, we managed to fine-tune to ~7.8 on this dataset, still preserving the general language abilities.

You can load the model and see its LoRA config as below:
```
from peft import get_peft_model, AutoPeftModelForCausalLM, PeftModel, PeftConfig
from transformers import AutoModel
config = PeftConfig.from_pretrained("unibuc-cs/CyberGuardian")
model = AutoModel.from_pretrained(config.base_model_name_or_path)
```


Refer to our github page for details: https://github.com/unibuc-cs/CyberGuardian