Bio-Medical-Llama-3.2-1B-MLX-4bit

This is a 4-bit quantized MLX version of ContactDoctor/Bio-Medical-Llama-3-2-1B-CoT-012025 for efficient inference on Apple Silicon.

Model Details

• Base Model: Bio-Medical-Llama-3-2-1B-CoT-012025
• Format: MLX (Apple Silicon optimized)
• Quantization: 4-bit (~4.5 bits per weight)
• Size: ~685 MB
• Parameters: 1B
• Use Case: Medical/Healthcare Q&A on iOS devices

Usage

from mlx_lm import load, generate

model, tokenizer = load("oscardeng/Bio-Medical-Llama-3.2-1B-MLX-4bit")

prompt = "What are the common symptoms of type 2 diabetes?"
messages = [{"role": "user", "content": prompt}]
formatted = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

response = generate(model, tokenizer, prompt=formatted, max_tokens=256)
print(response)

Disclaimer

This model is for informational purposes only and should not be used as a substitute for professional medical advice, diagnosis, or treatment. Always consult with a qualified healthcare provider.

License

This model inherits the Llama 3.2 Community License from the base model.