microsoft

/

Phi-3-medium-128k-instruct-onnx-cuda

like 21

It might be challenging to run phi-3-media on mobile phones considering the mode size. FYI, even with 4 bit quantization and optimizations, the size of the model is more than 8G. We would commend phi-3-mini for mobile deployment. here is an example: https://github.com/microsoft/onnxruntime-inference-examples/tree/main/mobile/examples/phi-3/android