YOLO-Master-EsMoE
This repository provides the Axera NPU deployment of YOLO-Master-EsMoE, a COCO object detection model built on a customized Ultralytics pipeline. The model replaces part of the YOLO backbone/head with mixture-of-experts (MoE) modules (ES_MOE). During ONNX export the MoE routing is converted to a dense fallback (all experts participate), producing a static computation graph that can be compiled by Pulsar2 into axmodel for Axera’s NPU-based AX650 Series. Three model sizes are provided: YOLO-Master-EsMoE-S, YOLO-Master-EsMoE-M and YOLO-Master-EsMoE-N, with an input size of 640×640 and 80 COCO classes.
References links:
For those who are interested in model conversion, you can try to export axmodel through
Support Platform
Performance
| Model | Input Shape | Latency (ms) | CMM Usage (MB) |
|---|---|---|---|
| YOLO-Master-EsMoE-M.axmodel | 1 x 640 x 640 x 3 | 23.446 | 70.40 |
| YOLO-Master-EsMoE-S.axmodel | 1 x 640 x 640 x 3 | 9.453 | 54.81 |
| YOLO-Master-EsMoE-N.axmodel | 1 x 640 x 640 x 3 | 4.406 | 46.68 |
Models
Download all files from this repository to the device
root@ax650 ~/root/yolo-master-esmoe # tree -L 3
.
|-- README.md
|-- requirement.txt
|-- install
| `-- ax_yolo_master_esmoe
|-- models
| `-- ax650
| |-- config.json
| |-- YOLO-Master-EsMoE-M
| | |-- 1_host_post_YOLO-Master-EsMoE-M.onnx
| | `-- YOLO-Master-EsMoE-M.axmodel
| |-- YOLO-Master-EsMoE-N
| | |-- 1_host_post_YOLO-Master-EsMoE-N.onnx
| | `-- YOLO-Master-EsMoE-N.axmodel
| `-- YOLO-Master-EsMoE-S
| |-- 1_host_post_YOLO-Master-EsMoE-S.onnx
| `-- YOLO-Master-EsMoE-S.axmodel
|-- web
| |-- app.py
| |-- detector.py
| `-- README.md
|-- infer
| |-- infer_axmodel.py
| `-- infer_onnx.py
`-- tools
|-- convert_opset.py
|-- eval_map.py
|-- export_onnx.py
`-- split_onnx.py
python env requirement
pip install -r requirement.txt
Inference with the web ui
root@ax650 ~/root/yolo-master-esmoe # python web/app.py
* Running on local URL: http://0.0.0.0:7860
* To create a public link, set `share=True` in `launch()`.
Use the device IP address and port 7860 to access the WebApp, for example http://192.168.1.100:7860. In the page, select the device ax650 and the model YOLO-Master-EsMoE-M, upload an image and click Detect.
Input image
Result：
Inference with YOLO-Master-EsMoE-S and YOLO-Master-EsMoE-N
The demo for YOLO-Master-EsMoE-S and YOLO-Master-EsMoE-N is the same as YOLO-Master-EsMoE-M, just select the corresponding model in the WebApp.
Inference with the executable
An executable is provided under install/ for inference directly on the target board without Python. Run it on AX650:
ax_yolo_master_esmoe -m <model.axmodel> -i <image.jpg>
Example:
ax_yolo_master_esmoe -m models/ax650/YOLO-Master-EsMoE-M/YOLO-Master-EsMoE-M.axmodel -i host.jpg --repeat 10
--------------------------------------
model file : models/ax650/YOLO-Master-EsMoE-M/YOLO-Master-EsMoE-M.axmodel
image file : host.jpg
img_h, img_w : 640 640
--------------------------------------
Engine creating handle is done.
Engine creating context is done.
Engine get io info is done.
input size: 1
name: images [UINT8] [RGB]
1 x 640 x 640 x 3
output size: 4
name: /model.25/Concat_output_0 [FLOAT32]
1 x 64 x 8400
name: /model.25/Reshape_3_output_0 [FLOAT32]
1 x 80 x 6400
name: /model.25/Reshape_4_output_0 [FLOAT32]
1 x 80 x 1600
name: /model.25/Reshape_5_output_0 [FLOAT32]
1 x 80 x 400
Engine alloc io is done.
Engine push input is done.
--------------------------------------
post process cost time:12.02 ms
--------------------------------------
Repeat 10 times, avg time 23.42 ms, max_time 23.47 ms, min_time 23.38 ms
--------------------------------------
detection num: 4
2: 92%, [ 0, 151, 188, 410], car
2: 89%, [ 17, 162, 487, 475], car
16: 62%, [ 197, 118, 364, 339], dog
2: 62%, [ 592, 174, 639, 302], car
--------------------------------------
Inference scripts
Besides the Gradio WebApp, two inference scripts are provided.
Run the FP32 ONNX baseline on the development machine:
python infer/infer_onnx.py --model <model.onnx> --image <image.jpg>
Run the axmodel inference on the target board:
python infer/infer_axmodel.py --axmodel models/ax650/YOLO-Master-EsMoE-M/YOLO-Master-EsMoE-M.axmodel --post-onnx models/ax650/YOLO-Master-EsMoE-M/1_host_post_YOLO-Master-EsMoE-M.onnx --image football.jpg