File size: 536 Bytes
1b438fb |
1 2 3 4 5 6 7 8 9 10 |
To run inference with this model, please follow below steps -
1. Install Nvidia Graphics driver R565 or higher.
2. Install python 3.10.11.
3. Setup GenAI DirectML via
a. Install numpy - pip install numpy==2.1.0
b. Install GenAI - pip install onnxruntime-genai-directml==0.5.0
4. Download inference script - curl https://github.com/microsoft/onnxruntime-genai/blob/rel-0.5.0/examples/python/phi3-qa.py -o phi3-qa.py
5. Run inference - python phi3-qa.py -m <model-downloaded-path>
a. Enter prompt - "What is GenAI?" |