MISHANM/deepseek-ai-DeepSeek-R1-BF16.gguf

The deepseek-ai/DeepSeek-R1 model has been converted into a GGUF format for optimal use with the llama.cpp framework, ensuring efficient performance. Tested on an AMD EPYC™ 9755 CPUs, the model demonstrates robust capabilities in handling complex natural language processing tasks with speed and accuracy. This CPU-focused optimization offers a cost-effective solution for users who prefer or require CPU-based processing, eliminating the need for specialized GPU hardware. Ideal for various applications, from research to production, this version provides a reliable and versatile tool for achieving high performance in natural language processing tasks.

Model Details

1. Language: English
2. Tasks: Text generation
3. Base Model: deepseek-ai/DeepSeek-R1

Building and Running the Model

To build and run the model using llama.cpp, follow these steps:

Model

Steps to Download the Model:

1. Go to the "Files and Versions" section.
2. Click on the model.
3. Copy the download link.
4. Create a directory (e.g., for Linux: mkdir DeepSeek-R1).
5. Navigate to that directory (cd DeepSeek-R1).
6. Download the model parts: DeepSeek-R1-BF16.gguf.part_01, DeepSeek-R1-BF16.gguf.part_02, DeepSeek-R1-BF16.gguf.part_03, DeepSeek-R1-BF16.gguf.part_04, DeepSeek-R1-BF16.gguf.part_05, DeepSeek-R1-BF16.gguf.part_06, DeepSeek-R1-BF16.gguf.part_07, DeepSeek-R1-BF16.gguf.part_08, DeepSeek-R1-BF16.gguf.part_09, DeepSeek-R1-BF16.gguf.part_10, DeepSeek-R1-BF16.gguf.part_11, DeepSeek-R1-BF16.gguf.part_12, DeepSeek-R1-BF16.gguf.part_13, DeepSeek-R1-BF16.gguf.part_14, DeepSeek-R1-BF16.gguf.part_15, DeepSeek-R1-BF16.gguf.part_16, DeepSeek-R1-BF16.gguf.part_17, DeepSeek-R1-BF16.gguf.part_18, DeepSeek-R1-BF16.gguf.part_19,DeepSeek-R1-BF16.gguf.part_20, DeepSeek-R1-BF16.gguf.part_21, DeepSeek-R1-BF16.gguf.part_22, DeepSeek-R1-BF16.gguf.part_23, DeepSeek-R1-BF16.gguf.part_24, DeepSeek-R1-BF16.gguf.part_25, DeepSeek-R1-BF16.gguf.part_26, DeepSeek-R1-BF16.gguf.part_27, DeepSeek-R1-BF16.gguf.part_28, DeepSeek-R1-BF16.gguf.part_29, DeepSeek-R1-BF16.gguf.part_30, DeepSeek-R1-BF16.gguf.part_31, DeepSeek-R1-BF16.gguf.part_32 (e.g., using wget with the copied link).

After downloading the model parts, use the following command to combine them into a complete model:

cat DeepSeek-R1-BF16.gguf.part_01 DeepSeek-R1-BF16.gguf.part_02 DeepSeek-R1-BF16.gguf.part_03 DeepSeek-R1-BF16.gguf.part_04 DeepSeek-R1-BF16.gguf.part_05 DeepSeek-R1-BF16.gguf.part_06 DeepSeek-R1-BF16.gguf.part_07 DeepSeek-R1-BF16.gguf.part_08 DeepSeek-R1-BF16.gguf.part_09 DeepSeek-R1-BF16.gguf.part_10 DeepSeek-R1-BF16.gguf.part_11 DeepSeek-R1-BF16.gguf.part_12 DeepSeek-R1-BF16.gguf.part_13 DeepSeek-R1-BF16.gguf.part_14 DeepSeek-R1-BF16.gguf.part_15 DeepSeek-R1-BF16.gguf.part_16 DeepSeek-R1-BF16.gguf.part_17 DeepSeek-R1-BF16.gguf.part_18 DeepSeek-R1-BF16.gguf.part_19 DeepSeek-R1-BF16.gguf.part_20 DeepSeek-R1-BF16.gguf.part_21 DeepSeek-R1-BF16.gguf.part_22 DeepSeek-R1-BF16.gguf.part_23 DeepSeek-R1-BF16.gguf.part_24 DeepSeek-R1-BF16.gguf.part_25 DeepSeek-R1-BF16.gguf.part_26 DeepSeek-R1-BF16.gguf.part_27 DeepSeek-R1-BF16.gguf.part_28 DeepSeek-R1-BF16.gguf.part_29 DeepSeek-R1-BF16.gguf.part_30 DeepSeek-R1-BF16.gguf.part_31 DeepSeek-R1-BF16.gguf.part_32  > deepseek-ai-DeepSeek-R1-BF16.gguf

Build llama.cpp Locally

git clone https://github.com/ggerganov/llama.cpp  
cd llama.cpp  
cmake -B build  
cmake --build build --config Release  

Run the Model

Navigate to the build directory and run the model with a prompt:

cd llama.cpp/build/bin   

Inference with llama.cpp

./llama-cli -m /path/to/deepseek-ai-DeepSeek-R1-BF16.gguf/ -p "Your prompt here" -n 500 --ctx-size 8192 --temp 0.6 --seed 3407

Citation Information

@misc{MISHANM/deepseek-ai-DeepSeek-R1-BF16.gguf,
  author = {Mishan Maurya},
  title = {Introducing deepseek-ai-DeepSeek-R1-BF16.gguf Model},
  year = {2025},
  publisher = {Hugging Face},
  journal = {Hugging Face repository},
  
}