Hugging Face's logo
Michael Goin's picture
TFLOPS icon 82.6 TFLOPS
72 15 20

Michael Goin PRO

mgoin
shubhrapandit's profile picture nicoboss's profile picture heli-stand's profile picture
·

AI & ML interests

LLM inference optimization, compression, quantization, pruning, distillation

Recent Activity

updated a model 12 days ago
nm-testing/pixtral-12b-FP8-dynamic-all
updated a model 15 days ago
mistralai/Pixtral-Large-Instruct-2411
New activity 15 days ago
mistralai/Pixtral-Large-Instruct-2411:Add config_format and load_format to vLLM args
View all activity

Organizations

Neural Magic's profile picture garage-bAInd's profile picture Blog-explorers's profile picture Revel Labs's profile picture ZeroGPU Explorers's profile picture NM Testing's profile picture MLX Community's profile picture Social Post Explorers's profile picture

mgoin's activity

New activity in mistralai/Pixtral-Large-Instruct-2411 15 days ago

Add config_format and load_format to vLLM args
#5 opened 15 days ago by mgoin

Update config.json to use null for sliding_window
#4 opened 15 days ago by mgoin
New activity in mgoin/nemotron-3-8b-chat-4k-sft-hf 20 days ago

Adding `safetensors` variant of this model
#1 opened 21 days ago by SFconvertbot
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16 29 days ago

Is this the standard GPTQ quantization?

1
#5 opened about 1 month ago by molereddy
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 29 days ago

Model weights are not loaded

4
#3 opened 3 months ago by MarvelousMouse
New activity in neuralmagic/pixtral-12b-FP8-dynamic about 1 month ago

Update model card
#1 opened about 1 month ago by nm-research
New activity in nm-testing/llava-1.5-7b-hf-FP8-dynamic about 1 month ago

Add chat_template to tokenizer_config.json
#1 opened about 1 month ago by nm-research
New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 about 1 month ago

7900xtx torch._scaled_mm is only supported on CUDA devices with compute capability >= 9.0 or 8.9, or ROCm MI300+

1
#3 opened about 1 month ago by aaaaaaaaaasdf
New activity in mistral-community/pixtral-12b about 2 months ago

Why is the Pixtral activation function "gelu" when the reference code uses "silu"?

2
#10 opened about 2 months ago by mgoin

Update tokenizer_config.json with chat_template

2
#11 opened about 2 months ago by mgoin
New activity in neuralmagic/Llama-3.2-90B-Vision-Instruct-FP8-dynamic about 2 months ago

Any chance your team is working on a 4-bit Llama-3.2-90B-Vision-Instruct-quantized.w4a16 version?

1
#1 opened 2 months ago by mrhendrey
New activity in neuralmagic/Llama-3.2-11B-Vision-Instruct-FP8-dynamic 2 months ago

Oom with 24g vram

3
#1 opened 2 months ago by Klopez
New activity in neuralmagic/Phi-3.5-mini-instruct-FP8-KV 2 months ago

latest vllm docker (v0.6.2) fail to load

2
#1 opened 2 months ago by choronz333
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w4a16 3 months ago

Issue with loading model

1
#1 opened 3 months ago by xSumukhax
New activity in neuralmagic/DeepSeek-Coder-V2-Instruct-FP8 3 months ago

Can it run on A100/A800 with VLLM?

3
#1 opened 4 months ago by Parkerlambert123
New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16 4 months ago

weights does not exist when trying to deploy in sagemaker endpoint

1
#1 opened 4 months ago by LorenzoCevolaniAXA
New activity in meta-llama/Llama-3.1-405B-Instruct 4 months ago

8-kv-heads

4
#17 opened 4 months ago by ArthurZ
New activity in meta-llama/Llama-3.1-405B 4 months ago

8-kv-heads

3
#21 opened 4 months ago by ArthurZ
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16 4 months ago

run with vllm

8
#4 opened 4 months ago by kuliev-vitaly
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8 4 months ago

Not able to run Model using VLLM

1
#3 opened 4 months ago by Pchaudhary