tags:
- gguf
- llama.cpp
- unsloth
- mistral
- python
base_model:
- mistralai/Mistral-7B-Instruct-v0.3
mistral-7b-python-gguf
Conversational Python fine-tune of Mistral 7B exported to GGUF format for local inference.
- Base model: Mistral 7B
- Fine-tuning framework: Unsloth
- Format: GGUF
- Author: AntoineChatry
⚠️ Disclaimer
This is an early experimental fine-tune.
It is not production-ready, not fully aligned, and not optimized for reliability or long-form reasoning.
This project was created primarily for learning and experimentation.
Please do not expect state-of-the-art coding performance.
Model Overview
This model is a conversational fine-tune of Mistral 7B trained primarily on:
- ShareGPT-style conversations
- Python-focused discussions
- Coding Q&A format
The objective was to:
- Experiment with fine-tuning
- Build a conversational Python model
- Export to GGUF for llama.cpp compatibility
- Test local inference workflows
No RLHF or advanced alignment was applied beyond the base model.
Known Limitations
Repetition Issues
- Frequently repeats phrases like:
"Here's the code:"
- Can loop or restate similar sentences
- Overuses patterns learned from dataset formatting
Weak Long-Form Explanations
- Struggles with multi-paragraph structured reasoning
- May repeat itself when asked for detailed explanations
- Limited depth on conceptual explanations
Instruction Following
- Not fully aligned
- May ignore strict formatting constraints
- Tends to prioritize generating code over detailed explanations
Dataset Bias
- Strong ShareGPT conversational tone
- Python-heavy bias
- Some templated response structure
What Works Reasonably Well
- Short Python snippets
- Basic debugging help
- Simple function generation
- Conversational coding prompts
Best performance is observed when:
- Prompts are clear and direct
- Expected output is short
- Tasks are code-focused
Training Details
- Base: Mistral 7B
- Dataset format: ShareGPT-style conversational dataset (Python-oriented)
- Fine-tuned using Unsloth notebooks
- Converted to GGUF for llama.cpp compatibility
- Quantized version included (Q4_K_M)
No additional safety tuning or post-training optimization was applied.
Example Usage
This model was finetuned and converted to GGUF format using Unsloth.
llama.cpp
For text-only LLMs:
llama-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja
For multimodal models:
llama-mtmd-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja
Available Model files:
mistral-7b-instruct-v0.3.Q4_K_M.gguf
Ollama
An Ollama Modelfile is included for easy deployment.
Example:
ollama create mistral-python -f Modelfile
ollama run mistral-python
Why This Model Is Public
This model represents a learning milestone.
Sharing imperfect models helps:
- Document fine-tuning progress
- Enable experimentation
- Collect feedback
- Iterate toward better versions
This is not a finished product.
Unsloth
This model was trained 2x faster using Unsloth.
https://github.com/unslothai/unsloth
License
Please refer to the original Mistral 7B license from Mistral AI.