How to use google/gemma-4-26B-A4B-it-assistant with Transformers:
# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-26B-A4B-it-assistant") model = AutoModelForCausalLM.from_pretrained("google/gemma-4-26B-A4B-it-assistant", device_map="auto")
Hi thereI seeking for AWQ version of gemma 31b or 26b assistant(MTP), but i only can find none assistant versions AWQ ,what is issue about create AWQ version of this models ?
· Sign up or log in to comment
