Hugging Face's logo
Michael Goin's picture
TFLOPS icon 82.6 TFLOPS
46 12 19

Michael Goin

mgoin
wendlerc's profile picture malikzeeshan's profile picture imjliao's profile picture
·

AI & ML interests

LLM inference optimization, compression, quantization, pruning, distillation

Organizations

Neural Magic's profile picture garage-bAInd's profile picture Blog-explorers's profile picture Revel Labs's profile picture ZeroGPU Explorers's profile picture NM Testing's profile picture MLX Community's profile picture Social Post Explorers's profile picture

mgoin's activity

New activity in neuralmagic/Mistral-7B-Instruct-v0.3-FP8 1 day ago

Compression script limits context length to 4098?

1
#1 opened 1 day ago by Kayvane
New activity in nvidia/Minitron-4B-Base 2 days ago

Where is Minitron-4B-Instruct?

1
#2 opened 2 days ago by mgoin
New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8 2 days ago

Is this compatible with the KV_Cache_dtype being FP8?

2
#1 opened 2 days ago by nickandbro
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8 2 days ago

Are these models limited to H100s?

7
#2 opened 2 days ago by RonanMcGovern
New activity in nvidia/Minitron-8B-Base 3 days ago

Replace kv_channels with head_dim
#1 opened 3 days ago by mgoin
New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 6 days ago

Error serving model

3
#2 opened 6 days ago by EvGUT
New activity in neuralmagic/Phi-3-mini-128k-instruct-quantized.w8a8 6 days ago

Model doesnt load in vllm version 0.5.0 : ValueError: `rope_scaling`'s type field must be one of ['su', 'yarn'], got longrope

1
#1 opened 7 days ago by sujjosep
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8-KV 24 days ago

How to load this model?

1
#1 opened 25 days ago by Frz614
New activity in neuralmagic/Meta-Llama-3-70B-Instruct-FP8 about 1 month ago

How to run Meta-Llama-3-70B-Instruct-FP8 using several devices?

5
#3 opened about 1 month ago by Fertel

Update model.safetensors.index.json
#2 opened about 1 month ago by mgoin
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 about 1 month ago

Update model.safetensors.index.json
#4 opened about 1 month ago by mgoin

`model.safetensors.index.json` still has the legacy name`act_scale` for activation scales

1
#3 opened about 1 month ago by Alchan
New activity in nm-testing/SparseLlama-3-8B-pruned_50.2of4-FP8 about 1 month ago

Update README.md
#1 opened about 1 month ago by alexmarques
New activity in neuralmagic/SparseLlama-3-8B-pruned_50.2of4 about 1 month ago

Update README.md
#1 opened about 1 month ago by alexmarques
New activity in neuralmagic/Qwen2-72B-Instruct-FP8 about 2 months ago

Update README.md
#1 opened about 2 months ago by abhinavnmagic
New activity in neuralmagic/Mixtral-8x7B-Instruct-v0.1-FP8 about 2 months ago

Update README.md
#1 opened about 2 months ago by abhinavnmagic
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 about 2 months ago

Update README.md
#2 opened about 2 months ago by abhinavnmagic
New activity in neuralmagic/Meta-Llama-3-70B-Instruct-FP8 about 2 months ago

Create README.md
#1 opened about 2 months ago by abhinavnmagic
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 about 2 months ago

Fails to run with nm-vllm

1
#1 opened 3 months ago by clintonruairi
New activity in mgoin/ultrachat_2k 2 months ago

Librarian Bot: Add language metadata for dataset
#2 opened 2 months ago by librarian-bot
New activity in neuralmagic/Mistral-7B-Instruct-v0.3-GPTQ-4bit 2 months ago

Inference GPU Ram requirement >60GB

1
#1 opened 2 months ago by Ksgk-fy
New activity in mgoin/Meta-Llama-3-70B-Instruct-Marlin 2 months ago

What conversion process are you using?

2
#2 opened 2 months ago by matt-psaltis-devbricks
New activity in mgoin/Meta-Llama-3-70B-Instruct-Marlin 3 months ago

What is Marlin?

2
#1 opened 3 months ago by Samvanity
New activity in mgoin/Meta-Llama-3-8B-Instruct-Marlin 3 months ago

Inference Issues

7
#1 opened 3 months ago by qeternity
New activity in neuralmagic/Llama-2-7b-dolphin-open_platypus-pruned_50 4 months ago

Update README.md
#2 opened 4 months ago by shubhrapandit
New activity in neuralmagic/Llama-2-7b-dolphin-open_platypus-pruned_70-quantized-deepsparse 4 months ago

Update README.md
#1 opened 4 months ago by shubhrapandit
New activity in neuralmagic/Llama-2-7b-dolphin-open_platypus-pruned_50-quantized-deepsparse 4 months ago

Update README.md
#1 opened 4 months ago by shubhrapandit
New activity in neuralmagic/Llama-2-7b-dolphin-open_platypus-pruned_70 4 months ago

Update README.md
#1 opened 4 months ago by shubhrapandit
New activity in neuralmagic/Llama-2-7b-dolphin-open_platypus-pruned_50 4 months ago

Update README.md
#1 opened 4 months ago by shubhrapandit
New activity in neuralmagic/Llama-2-7b-evolcodealpaca 4 months ago

Update README.md
#1 opened 4 months ago by abhinavnmagic
New activity in neuralmagic/Llama-2-7b-evol-code-alpaca-pruned_70-quantized-deepsparse 4 months ago

Update README.md
#1 opened 4 months ago by abhinavnmagic
New activity in neuralmagic/Llama-2-7b-evol-code-alpaca-pruned_50-quantized-deepsparse 4 months ago

Update README.md
#1 opened 4 months ago by abhinavnmagic
New activity in neuralmagic/Llama-2-7b-evol-code-alpaca-pruned_70 4 months ago

Update README.md
#1 opened 4 months ago by abhinavnmagic
New activity in neuralmagic/Llama-2-7b-evol-code-alpaca-pruned_50 4 months ago

Update README.md
#1 opened 4 months ago by abhinavnmagic
New activity in neuralmagic/Llama-2-7b-ultrachat200k-pruned_70-quantized-deepsparse 4 months ago

Update README.md
#1 opened 4 months ago by alexmarques
New activity in neuralmagic/Llama-2-7b-ultrachat200k-pruned_70 4 months ago

Update README.md
#1 opened 4 months ago by alexmarques
New activity in neuralmagic/Llama-2-7b-ultrachat200k-pruned_50 4 months ago

Update README.md
#1 opened 4 months ago by alexmarques
New activity in neuralmagic/Llama-2-7b-pruned70-retrained 4 months ago

Update README.md
#1 opened 4 months ago by alexmarques
New activity in neuralmagic/Llama-2-7b-pruned50-retrained 4 months ago

Update README.md
#1 opened 4 months ago by alexmarques
New activity in neuralmagic/CLIP-ViT-B-32-256x256-DataComp-s34B-b86K-quant-ds 7 months ago

Update README.md
#4 opened 7 months ago by chrisxx

Update README with model author names and speedup numbers.
#3 opened 7 months ago by jen

Update README.md
#1 opened 7 months ago by wendlerc
New activity in neuralmagic/mpt-7b-gsm8k-pruned60-pt 8 months ago

Adding `safetensors` variant of this model
#1 opened 8 months ago by mgoin
New activity in reciprocate/llama2-7b-gsm8k 10 months ago

Create README.md
#2 opened 10 months ago by mgoin