Follow up - tools calling + dtype kv cache
#4
by Jonah16 - opened
Hi !
All my setup is ready to run this quantization, I just need the right tools, minimax-m2 tool is not compatible. Does anyone find tools that allow minimax M3 to run agentic tool ?
Also, I have to use kv-cache dtype at bf16, it would be incredible to be able to set fp8 !!
Thank you for this work <3
you'd want to do - kv-cache-dtype fp8 in your launch config. check out the docker image and launch config example here https://huggingface.co/lukealonso/MiniMax-M3-NVFP4/discussions/1 point your agent at that, and i'tll pick it up!