microsoft

/

Phi-3-mini-4k-instruct-onnx

like 102

I'm following the tutorial here: https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/phi-3-tutorial.md

Trying to run \cpu_and_mobile\cpu-int4-rtn-block-32-acc-level-4 on my PC and I see the following error after supplying an input:

2024-04-23 15:14:06.4954767 [E:onnxruntime:onnxruntime-genai, sequential_executor.cc:516 onnxruntime::ExecuteKernel] Non-zero status code returned while running Add node. Name:'/model/attn_mask_reformat/Add' Status Message: D:\a_work\1\s\onnxruntime\core/providers/cpu/math/element_wise_ops.h:540 onnxruntime::BroadcastIterator::Init axis == 1 || axis == largest was false. Attempting to broadcast an axis by a dimension other than 1. 7 by 8

Traceback (most recent call last):
File "c:\Phi-3-mini-4k-instruct-onnx\model-qa.py", line 82, in
main(args)
File "c:\Phi-3-mini-4k-instruct-onnx\model-qa.py", line 47, in main
generator.compute_logits()
onnxruntime_genai.onnxruntime_genai.OrtException: Non-zero status code returned while running Add node. Name:'/model/attn_mask_reformat/Add' Status Message: D:\a_work\1\s\onnxruntime\core/providers/cpu/math/element_wise_ops.h:540 onnxruntime::BroadcastIterator::Init axis == 1 || axis == largest was false. Attempting to broadcast an axis by a dimension other than 1. 7 by 8

Thoughts on what could be wrong?