Can Signal-to-Noise Ratio Perform as a Baseline Indicator for Medical Image Quality Assessment

Natural image quality assessment (NIQA) wins increasing attention, while NIQA models are rarely used in the medical community. A couple of studies employ the NIQA methodologies for medical image quality assessment (MIQA), but building the benchmark data sets necessitates considerable time and professional skills. In particular, the characteristics of synthesized distortions are different from those of clinical distortions, which make the results not so convincing. In clinic, signal-to-noise ratio (SNR) is widely used, which is defined as the quotient of the mean signal intensity measured in a tissue region of interest (ROI) and the standard deviation of the signal intensity in an air region outside the imaged object, and both regions are outlined by specialists. We take advantage of the knowledge that SNR is routinely used and concern whether SNR measure can perform as a baseline metric for the development of MIQA algorithms. To address the issue, the inter-observer reliability of SNR measure is investigated regarding to different tissue ROIs [white matter (WM); cerebral spinal fluid (CSF)] in magnetic resonance (MR) images. A total of 192 <inline-formula> <tex-math notation="LaTeX">${\text{T}}_{2}^{*}$ </tex-math></inline-formula>, 88 <inline-formula> <tex-math notation="LaTeX">${\text{T}}_{1}$ </tex-math></inline-formula>, 76 <inline-formula> <tex-math notation="LaTeX">${\text{T}}_{2}$ </tex-math></inline-formula> and 55 contrast-enhanced <inline-formula> <tex-math notation="LaTeX">${\text{T}}_{1}$ </tex-math></inline-formula> (<inline-formula> <tex-math notation="LaTeX">${\text{T}}_{1}\text{C}$ </tex-math></inline-formula>) weighted images are analyzed. Statistical analysis indicates that SNR values show consistency between different observers to the same ROI in each modality (Wilcoxon rank sum test, <inline-formula> <tex-math notation="LaTeX">$p_{w}\ge 0.11$ </tex-math></inline-formula>; and paired sample <inline-formula> <tex-math notation="LaTeX">$t$ </tex-math></inline-formula>-test, <inline-formula> <tex-math notation="LaTeX">$p_{p}\ge 0.28$ </tex-math></inline-formula>). Moreover, whether off-the-shelf NIQA models can predict MR image quality is considered by using SNR values as the reference scores. Four NIQA models (BIQI, BLIINDS-II, BRISQUE, and NIQE) are evaluated, and the correlation between SNR values and NIQA results is evaluated. Pearson correlation coefficient (<inline-formula> <tex-math notation="LaTeX">$r_{p}$ </tex-math></inline-formula>) shows that WM-based SNR values correlates well with BIQI, BLIINDS-II and BRISQUE in <inline-formula> <tex-math notation="LaTeX">${\text{T}}_{2}^{*}$ </tex-math></inline-formula> images (<inline-formula> <tex-math notation="LaTeX">$r_{p}\geq 0.77$ </tex-math></inline-formula>), BRISQUE and NIQE in <inline-formula> <tex-math notation="LaTeX">${\text{T}}_{1}$ </tex-math></inline-formula> images (<inline-formula> <tex-math notation="LaTeX">$r_{p}\geq 0.75$ </tex-math></inline-formula>), BLIINDS-II in <inline-formula> <tex-math notation="LaTeX">${\text{T}}_{2}$ </tex-math></inline-formula> images (<inline-formula> <tex-math notation="LaTeX">$r_{p}\geq 0.67$ </tex-math></inline-formula>), and BRISQUE and NIQE in <inline-formula> <tex-math notation="LaTeX">${\text{T}}_{1}\text{C}$ </tex-math></inline-formula> images (<inline-formula> <tex-math notation="LaTeX">$r_{p}\geq 0.58$ </tex-math></inline-formula>), while CSF-based SNR values correlates well with BLIINDS-II in <inline-formula> <tex-math notation="LaTeX">${\text{T}}_{2}^{*}$ </tex-math></inline-formula> images (<inline-formula> <tex-math notation="LaTeX">$r_{p}\geq 0.64$ </tex-math></inline-formula>) and <inline-formula> <tex-math notation="LaTeX">${\text{T}}_{2}$ </tex-math></inline-formula> images (<inline-formula> <tex-math notation="LaTeX">$r_{p}\geq 0.60$ </tex-math></inline-formula>), and all <inline-formula> <tex-math notation="LaTeX">$p_{p}<10^{-4}$ </tex-math></inline-formula>. The prediction performance analysis further proves the result from the correlation analysis. Conclusively, SNR measure is reliable to different observations and can perform as a baseline indicator for the development of MIQA algorithms. In general, BRISQUE and BLIINDS-II are full of potential to be conditionally used as objective MIQA models toward human brain MR images. This paper presents the first attempt of using SNR measure to bridge the gap between NIQA and MIQA, and large-scale experiments should be further conducted to confirm the conclusion in this paper.