# Image Quality Assessment to Emulate Experts' Perception in Lumbar MRI Using Machine Learning
## 3 Results  
Typical images of \"bad\" and \"good\" quality as defined by the experts are shown in Figure 2. The examples aim to emphasize that, even if the difference between a clearly good and a clearly bad image is easy to assess, the subtleties of the in-between range are more difficult to distinguish. Figure 3 presents the distribution of the image quality of our three data sets evaluated by the experts: each of the data sets presents an equilibrated quantity of \"good\" and \"bad\" quality images. Axial images are globally better evaluated than sagittal images by all experts. Figure 2: Examples of images from (left column) sagittal \(T_{1}\), (center column) sagittal \(T_{2}\), (right column) axial \(T_{1}\), with (first row) high quality according to the experts (weighted Mean Opinion Scores (wMOS) between 4.3 and 4.7), (second row) average quality (wMOS between 3.1 and 3.2) and (third row) low quality (wMOS between 2.7 and 2.8). The examples shown here were not manipulated by us to modify their quality. Agreement between neuro-radiologists about the image quality is rated from fair to substantial in the different types of data sets, as detailed in Table 3. Better agreement in quality image evaluation is found in Sagittal \(T_{2}\) data sets, with Kappa values of 0.56 \(\pm\) 0.08 over all cases, while the worst agreement is found in the axial \(T_{1}\) data set, with Kappa values of 0.38 \(\pm\) 0.22. NR\({}_{1}\) and NR\({}_{2}\) present the closest agreement in image quality evaluation, even though Kappa values reach only 0.63 \(\pm\) 0.03, underlining that excellent experts' agreement is not easy to achieve. Agreement between NR\({}_{2}\) and NR\({}_{3}\) is even lower, only 0.35 \(\pm\) 0.12. Figure 4 shows correlation coefficients between MOS and each feature used in the objective quality assessment. Most correlation coefficients calculated in the case of Axial \(T_{1}\) images are close to 0, showing that no clear linear relation between MOS and each feature separately exists that could by itself identify \"good\" or \"bad\" quality images. A very similar trend is observed in the case of Sagittal \(T_{2}\) images. Considering Sagittal \(T_{1}\) images, a few correlation coefficients rise to a magnitude of 0.6, uniformity measured in the vertebra or spectral flatness, but this correlation is not strong enough to explain the classification obtained by subjective assessment. \begin{table}
\begin{tabular}{l c c c c} \hline  & **NR\({}_{1}\) vs. NR\({}_{2}\)** & **NR\({}_{1}\) vs. NR\({}_{3}\)** & **NR\({}_{2}\) vs. NR\({}_{3}\)** & \\ \hline Sagittal \(T_{1}\) & 0.66 & 0.39 & 0.34 & 0.46 \(\pm\) 0.17 \\ Sagittal \(T_{2}\) & 0.60 & 0.62 & 0.47 & 0.56 \(\pm\) 0.08 \\ Axial \(T_{1}\) & 0.63 & 0.29 & 0.23 & 0.38 \(\pm\) 0.22 \\ \hline  & 0.63 \(\pm\) 0.03 & 0.43 \(\pm\) 0.17 & 0.35 \(\pm\) 0.12 & \\ \hline \end{tabular}
\end{table} 
Table 3: Kappa index of agreement between neuro-radiologists (NR). The column on the right and the bottom line indicate mean \(\pm\) standard deviation. We considered as “substantial” agreement kappa indices between 0.61 and 0.8, as “moderate” agreement values between 0.41 and 0.60 and as “fair” agreement values between 0.21 and 0.40. Figure 3: Average weighted Mean Opinion Scores (wMOS) histogram distributions in our three data types, obtained from the experts’ evaluation. From left to right: sagittal \(T_{1}\)-, sagittal \(T_{2}\)- and axial \(T_{1}\)-weighted images. Classification performance metrics obtained for the five ML models are described in Table 4. In general, the SVM showed superior performance in most cases, followed by LDA and MLP. Furthermore, it can be observed that QDA showed the lowest performance. The SVM achieved an accuracy of over 73% in all cases, it reached 77% in Sagittal \(T_{2}\) as well as a recall of 89% for this same image. On the other hand, the reported AUC reflects a good discriminatory precision by showing values higher than 70%.