HRNet model trained on Cityscapes

HRNet trained on Cityscapes dataset at resolution 512x1024 for semantic segmentation on images. It was introduced in the paper Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation by Yuhui Yuan et al. The code version we use from this repository.

We develop a modified version that could be supported by AMD Ryzen AI.

Model description

HRNet is an advanced algorithm used for image segmentation. It is based on deep learning techniques and is capable of providing accurate semantic segmentation in images.

Intended uses & limitations

You can use the raw model for semantic segmentation. See the model hub to look for all available HRNet models.

How to use

Installation

Follow Ryzen AI Installation to prepare the environment for Ryzen AI. Run the following script to install pre-requisites for this model.

pip install -r requirements.txt

Data Preparation (optional: for accuracy evaluation)

Download the Cityscapes dataset, which includes images and annotations. Download gtFine_trainvaltest.zip (241MB) and leftImg8bit_trainvaltest.zip (11GB).
Organise the dataset directory as follows:

  ./data/cityscapes/
    gtFine
    leftImg8bit
    train.lst
    val.lst
    test.lst

Test & Evaluation

Run inference on a single image

python hrnet_quantized_onnx_inference.py -m HighResolutionNet_int.onnx -idir PATH_TO_IMAGES(like .\data\cityscapes\leftImg8bit\val\frankfurt) --ipu --provider_config Path\To\vaip_config.json
#return segmentaion logits and can visualize the result.

Note: vaip_config.json is located at the setup package of Ryzen AI (refer to Installation)

Test accuracy of the quantized model on Cityscapes.

python hrnet_quantized_onnx_eval.py -m .\HighResolutionNet_int.onnx -r .\data\cityscapes -l .\val.lst --ipu --provider_config .\vaip_config.json

Performance

Model	miou
HRNet_int8_onnx_model (512x1024)	72.31%

@article{YuanCW19,
  title={Object-Contextual Representations for Semantic Segmentation},
  author={Yuhui Yuan and Xilin Chen and Jingdong Wang},
  booktitle={ECCV},
  year={2020}
}

amd
/

HRNet