anoopbhargavakm87/Mistral-7B-Instruct-v0.3-4bit-Compressed

4-bit Quantized Model

This repository contains a high-efficiency 4-bit compressed version of the model. It is optimized using bitsandbytes (NF4 precision) to run on smaller graphics cards or consumer hardware.

Hardware Setup Used

Processed and generated on enterprise-grade hardware utilizing NVIDIA Multi-Instance GPU (MIG) architecture and high-capacity server systems.

Quick Start Code

To use this compressed version in your own Jupyter notebook, run:

from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("YOUR_USERNAME/YOUR_REPO_NAME", device_map="auto")