This is an invalid model
FATAL EXCEPTION: main
Process: ai.onnxruntime.genai.demo, PID: 10906
java.lang.RuntimeException: Unable to start activity ComponentInfo{ai.onnxruntime.genai.demo/ai.onnxruntime.genai.demo.MainActivity}: java.lang.RuntimeException: ai.onnxruntime.genai.demo.GenAIException: Load model from /data/user/0/ai.onnxruntime.genai.demo/files/phi3-mini-4k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx failed:This is an invalid model. Type Error: Type 'tensor(float)' of input parameter (/model/layers.0/attn/qkv_proj/MatMul/output_0) of operator (GroupQueryAttention) in node (/model/layers.0/attn/GroupQueryAttention) is invalid.
Can you upgrade your ONNX Runtime GenAI version and try again? The package version you have installed does not appear to have support for GroupQueryAttention on CPU. This is fixed in the latest package release.
Thanks @kvaishnavi this issue is resolved now