license: mit
datasets:
- trendmicro-ailab/Primus-FineWeb
language:
- en
base_model:
- trendmicro-ailab/Llama-Primus-Base
library_name: transformers
Model Card for 8Bit GGUF version of TrendMicro-Llama-Primus-Base-8bit-gguf
This model is a 8bit Quantized GGUF model of trendmicro-ailab/Llama-Primus-Base For original model and documentation visit
https://huggingface.co/trendmicro-ailab/Llama-Primus-Base
Primus: A Pioneering Collection of Open-Source Datasets for Cybersecurity LLM Training
TL;DR: Llama-Primus-Base is a foundation model based on Llama-3.1-8B-Instruct, continually pre-trained on Primus-Seed (0.2B) and Primus-FineWeb (2.57B). Primus-Seed is a high-quality, manually curated cybersecurity text dataset, while Primus-FineWeb consists of cybersecurity texts filtered from FineWeb, a refined version of Common Crawl. By pretraining on such a large-scale cybersecurity corpus, it achieves a 🚀15.88% improvement in aggregated scores across multiple cybersecurity benchmarks, demonstrating the effectiveness of cybersecurity-specific pretraining.
🔥 For more details, please refer to the paper: [📄Paper].
License
This model is based on the MIT license, but you must also comply with the Llama 3.1 Community License Agreement.