Instructions to use moonshotai/Kimi-K3.1-Pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use moonshotai/Kimi-K3.1-Pro with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="moonshotai/Kimi-K3.1-Pro", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("moonshotai/Kimi-K3.1-Pro", trust_remote_code=True, device_map="auto") - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use moonshotai/Kimi-K3.1-Pro with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "moonshotai/Kimi-K3.1-Pro" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "moonshotai/Kimi-K3.1-Pro", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/moonshotai/Kimi-K3.1-Pro
- SGLang
How to use moonshotai/Kimi-K3.1-Pro with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "moonshotai/Kimi-K3.1-Pro" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "moonshotai/Kimi-K3.1-Pro", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "moonshotai/Kimi-K3.1-Pro" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "moonshotai/Kimi-K3.1-Pro", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use moonshotai/Kimi-K3.1-Pro with Docker Model Runner:
docker model run hf.co/moonshotai/Kimi-K3.1-Pro
Please soften the hard-trained persona
Because of the hard-trained persona that it's Kimi by MoonshotAI, etc., it has poor behavioral instruction following. For example, I wanted it to be a balanced researcher with web search. While my prompt works for other models (DSV4.1, GLM-5.3), K3.1 continuously falls back to its default persona and essentially ignores my system prompt. I also don't like how it will always explicitly call itself Kimi.
up
oh no. Kimi is Kimi .... but yes... it would be nice for the model to be flexible in persona. but for my application, i Let her be Kimi
Could this be a measure designed to punish those who rudely took Moonshot AI’s work, performed only minor fine-tuning, and then claimed the credit for themselves?
Perhaps it’s set up so that any significant modification (ex: persona) leads to a drop in performance.(SWE-??)
Could this be a measure designed to punish those who rudely took Moonshot AI’s work, performed only minor fine-tuning, and then claimed the credit for themselves?
Perhaps it’s set up so that any significant modification (ex: persona) leads to a drop in performance.(SWE-??)
I don’t think so. It seems like something that could be trivially abliterated.
It’s easy for @sumiremelody to say that. Well then, show me you can actually do it.
https://github.com/p-e-w/heretic already demonstrates this for models.
Simply presenting a method does not prove its effectiveness. The case of DeepSeek-V4.1Flash is a prime example of such "resilience." Besides, you didn’t actually carry out the work yourself, did you? I, for one, have already had the opportunity to test this method using Gemma-4 what about you? What, exactly, have @sumiremelody managed to accomplish?
It’s easy for @sumiremelody to say that. Well then, show me you can actually do it.
https://github.com/p-e-w/heretic already demonstrates this for models.
... It seems like something that could be trivially abliterated.
AFAICT, there are exactly zero Kimi models that have been abliterated with heretic. It certainly isn't "trivial", especially for a 2T+ model...
It’s easy for @sumiremelody to say that. Well then, show me you can actually do it.
https://github.com/p-e-w/heretic already demonstrates this for models.
... It seems like something that could be trivially abliterated.AFAICT, there are exactly zero Kimi models that have been abliterated with heretic. It certainly isn't "trivial", especially for a 2T+ model...
To be clear, I originally said,
I don’t think so. It seems like something that could be trivially abliterated.
And the model was released a day ago. It would take time for people to abliterate for something like censorship (which is the common target), if there is any interest at all given the hardware requirements exceeding consumer and most enthusiast hardware.
It could be trivial in the sense that a hypothetical team may only need a judge to determine whether it’s adhering to a non-existent Kimi identity to thus eliminate it. Again, in this context, the abliteration in question was not about censorship but about the “I am Kimi” persona.