How to use tintwotin/Marlin-2B-SDNQ-int8 with Transformers:
# Load model directly from transformers import AutoProcessor, AutoModelForCausalLM processor = AutoProcessor.from_pretrained("tintwotin/Marlin-2B-SDNQ-int8", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("tintwotin/Marlin-2B-SDNQ-int8", trust_remote_code=True)
The community tab is the place to discuss and collaborate with the HF community!
