# Image Quality Assessment for Gibbs Ringing Reduction
## 3 Metrics  
Many previous studies that propose a novel reconstruction algorithm or post-processing method evaluate the outcomes qualitatively. Objective image quality metrics are desirable to facilitate automation of this assessment in order to facilitate the development of machine learning techniques and to eliminate inter- and intra-observer variation. In [17], we surveyed the literature on Gibbs ringing for MRI and found 12 metrics (e.g., PSNR, RMSE, entropy) that had been used in the past for assessing the quality of ringing reduction methods. We tested these twelve metrics to assess if they could pass a simple test: given a test image with ringing and a Gaussian filter of adjustable variance \(\sigma\), could they identify which value of \(\sigma\) resulted in the best outcome. Large values of \(\sigma\) will blur the image less, but also will reduce ringing less. Small values of \(\sigma\) will suppress the ringing more, but will introduce more blurring. 'Best', in this case, means that the ringing is suppressed and the blurring from the filter is not excessive. A suitable metric must exhibit a global minimum or maximum in this situation. We were forced to conclude that none of the 12 metrics were suitable for this problem. We proposed a new full reference metric for evaluating Gibbs ringing suppression results called \(\ell_{\epsilon}^{0}\). We did not compare the inflection point of the \(\ell_{\epsilon}^{0}\) curve with human perception of the image quality. We investigate the effect of varying \(\epsilon\) in Section 5. SSIM is known to be correlated with human perception. In this paper, we make the comparison between \(\ell_{e}^{0}\) and SSIM. We also test SSIM as a potential full reference metric for ringing reduction. The definition of structural similarity index of two images, \(x\) and \(y\), can be found in Equation (12) in [23]. The structural similarity ranges from 0 to 1. When the two images are identical, the value of SSIM is equal to one. The SSIM evaluates luminance, contrast and structure at the same time. SSIM is a commonly used metric for various fields. In our problem, we define \(y\) as the reconstructed image and \(x\) as the targeted ground truth. SSIM was used as a metric for Gibbs ringing measurement mostly in machine learning papers [12; 14; 15]. We evaluate SSIM for quantifying Gibbs ringing suppression in Section 3.1. Because SSIM is known to be correlated with human perception, we then use it as a benchmark when we investigate the effect of varying \(\epsilon\) in Section 5. It has been suggested that \"For image quality assessment, it is useful to apply the SSIM index locally rather than globally\" [23], which is one reason we investigate regions of interest in Section 4. However, the papers that utilized SSIM for evaluating Gibbs suppression all used it globally on the entire reconstructed image rather than on the region of interest with ringing artefacts. Therefore, we investigate MS-SSIM as a potential metric for evaluating Gibbs ringing suppression. MS-SSIM was first proposed in [24]. The definition of MS-SSIM between image \(x\) and \(y\) can be found in Equation (7) in [24]. MS-SSIM is more flexible than SSIM because it has the ability to incorporate the variations of image resolution and viewing conditions. In this paper, the number of scales for MS-SSIM is five. The weights for each scale are 0.0545, 0.2442, 0.4026, 0.2442 and 0.0545. The weights follow a Gaussian distribution because human visual sensitivity peaks at middle frequencies and decreases in both directions. MAE, also known as the \(\ell^{1}\) loss, is a loss function used in linear regression models and is a useful measure widely used in model evaluations. MAE can be defined as  
\[\text{MAE}=\frac{1}{N}\sum_{n=1}^{N}|y-x| \tag{3}\]  
As we noted, \(\ell_{e}^{0}\) works by counting the pixels with error above a certain threshold. Huber loss [25] is a loss function used in robust regression commonly used in statistics and machine learning.