---
license: mit
language:
- en
datasets:
- LIVE
- CSIQ
- TID2013
- KADID-10K
metrics:
- pearsonr
- spearmanr
---
# Non-local Modeling for Image Quality Assessment
## Table of Contents
(i) **Image Preprocessing**: The input image is pre-processed. ๐ Check [this file](https://github.com/SuperBruceJia/NLNet-IQA/blob/main/lib/image_process.py#L17).
(ii) **Graph Neural Network โ Non-Local Modeling Method**: A two-stage GNN approach is presented for the non-local feature extraction and long-range dependency construction among different regions. The first stage aggregates local features inside superpixels. The following stage learns the non-local features and long-range dependencies among the graph nodes. It then integrates short- and long-range information based on an attention mechanism. The means and standard deviations of the non-local features are obtained from the graph feature signals. ๐ Check [this file](https://github.com/SuperBruceJia/NLNet-IQA/blob/main/model/network.py#L62).
(iii) **Pre-trained VGGNet-16 โ Local Modeling Method**: Local feature means and standard deviations are derived from the pre-trained VGGNet-16 considering the hierarchical degradation process of the HVS. ๐ Check [this file](https://github.com/SuperBruceJia/NLNet-IQA/blob/main/model/network.py#L37).
(iv) **Feature Mean & Std Fusion and Quality Prediction**: The means and standard deviations of the local and non-local features are fused to deliver a robust and comprehensive representation for quality assessment. ๐ Check [this file](https://github.com/SuperBruceJia/NLNet-IQA/blob/main/model/network.py). Besides, the distortion type identification loss $L_t$ , quality prediction loss $L_q$ , and quality ranking loss $L_r$ are utilized for training the NLNet. ๐ Check [this file](https://github.com/SuperBruceJia/NLNet-IQA/blob/main/model/solver.py#L171). During inference, the final quality of the image is the averaged quality of all the non-overlapping patches. ๐ Check [this file](https://github.com/SuperBruceJia/NLNet-IQA/blob/main/lib/image_process.py#L17).
### Poster Presentation
## Structure of the Code
At the root of the project, you will see:
```text
โโโ main.py
โโโ model
โย ย โโโ layers.py
โย ย โโโ network.py
โย ย โโโ solver.py
โโโ superpixel
โ โโโ slic.py
โโโ lib
โย ย โโโ image_process.py
โย ย โโโ make_index.py
โย ย โโโ utils.py
โโโ data_process
โย ย โโโ get_data.py
โย ย โโโ load_data.py
โโโ benchmark
โย ย โโโ CSIQ_datainfo.m
โย ย โโโ CSIQfullinfo.mat
โย ย โโโ KADID-10K.mat
โย ย โโโ LIVEfullinfo.mat
โย ย โโโ TID2013fullinfo.mat
โย ย โโโ database.py
โย ย โโโ datainfo_maker.m
โโโ save_model
โย โโโ README.md
โโโ test_images
โ โโโ cr7.jpg
โโโ real_testing.py
```
## Citation
If you find our work useful in your research, please consider citing it in your publications.
We provide a BibTeX entry below.
```bibtex
@inproceedings{Jia2022NLNet,
title = {No-reference Image Quality Assessment via Non-local Dependency Modeling},
author = {Jia, Shuyue and Chen, Baoliang and Li, Dingquan and Wang, Shiqi},
booktitle = {2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)},
year = {Sept. 2022},
volume = {},
number = {},
pages = {01-06},
doi = {10.1109/MMSP55362.2022.9950035}
}
@article{Jia2022NLNetThesis,
title = {No-reference Image Quality Assessment via Non-local Modeling},
author = {Jia, Shuyue},
journal = {CityU Scholars},
year = {May 2023},
publisher = {City University of Hong Kong},
url = {https://scholars.cityu.edu.hk/en/theses/noreference-image-quality-assessment-via-nonlocal-modeling(2d1e72fb-2405-43df-aac9-4838b6da1875).html}
}
```
## Contact
If you have any questions, please drop me an email at shuyuej@ieee.org.
## Acknowledgement
The authors would like to thank Dr. Xuhao Jiang, Dr. Diqi Chen, and Dr. Jupo Ma for helpful discussions and invaluable inspiration. A special appreciation should be shown to Dr. Dingquan Li because this code is built upon his [(Wa)DIQaM-FR/NR](https://github.com/lidq92/WaDIQaM) re-implementation.