# On-the-Fly Machine Learning for Improving Image Resolution in Tomography
## 4 Results
### Comparison with Other Network Structures  
In this section, we compare the results of the MS-D network to the widely used U-net approach described in [30] on both the simulated and experimental data. The data acquisition and reconstruction procedures remain unchanged, and the U-net is trained on the low- and high-resolution data for two days or 1000 epochs, whichever comes first. As before, the metrics on the simulated data are computed based on the original phantom and the output of the network, and the metrics on the experimental data are computed based on the high-resolution target and the output of the network. **Neural network implementation** The U-net network is implemented in PyTorch, and is based on a widely available open source implementation. We have provided our code as an open source package [42]. This implementation of the U-net architecture is almost identical to that described in [30]: the images are down-sampled four times using \(2\times 2\) max-pooling, the \"up-convolutions\" have trainable parameters, and the convolutions have \(3\times 3\) kernels. Like [43], this implementation uses batch normalization before each ReLU. Moreover, the smallest image layers are 512 channels instead of 1024 channels, and zero-padding is used instead of reflection-padding. An adaptation was made to allow the network to process images whose width or height was not divisible by 16: these images were padded to the right dimensions using reflection padding. All U-net networks are trained from scratch in each experiment. **Metrics and evaluation** The quantitative results are given in Table 6, and the network outputs are visually compared in Figure 10. On the whole, the output of the U-net is visually similar to the MS-D network, and apart from a few negative outliers, the metrics of U-net are similar to those of the MS-D network. The lower SSIM metrics of the U-net could be explained by the fact that, on the pixel-limited dataset, the U-net appears to introduce the same non-smooth texture in the foam (method A--9 slices) and in the voids (method B) that is present in the high-resolution image. Likewise, the oatmeal flakes in the U-net outputs appear to have a distinct texture, especially for method B. It appears that some care must be taken to prevent the U-net from overfitting to the training data. In Figure 11, we compare the training and test error of the U-net and MS-D networks as training progresses. The mean square error metric is computed on the central slice of the training and test set of the oatmeal dataset. For both networks, the MSE decreases on the training slice. On the test slice, however, the MSE metric of the MS-D network remains consistently low, but the MSE of the U-net increases as training progresses, indicating that the network overfits to the training data. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & & \multicolumn{2}{c}{**Method A**} & \multicolumn{2}{c}{**Method A**} & \multicolumn{2}{c}{**Method B**} \\ \cline{3-8}  & & **Slab: 9 Slices** & & **Slab: 1 Slice** & & **Slab: 1 Slice** \\ \hline \multirow{3}{*}{Pixel-limited} & & MSE & SSIM & MSE & SSIM & MSE & SSIM \\ \cline{2-8}  & U-net & 0.0057 & 0.4225 & 0.0092 & 0.9149 & 0.0111 & 0.6850 \\  & MS-D & **0.0044** & **0.9578** & 0.0089 & 0.9281 & 0.0111 & 0.8065 \\ \hline \multirow{3}{*}{Optics-limited} & U-net & 0.0159 & 0.7436 & 0.0402 & 0.1339 & 0.0178 & 0.7959 \\  & MS-D & **0.0143** & **0.8275** & 0.0154 & 0.8220 & 0.0178 & 0.8135 \\ \hline \multirow{3}{*}{Oatmeal} & U-net & 0.0030 & 0.4372 & 0.0029 & 0.4372 & 0.0037 & 0.4221 \\  & MS-D & 0.0027 & 0.4508 & 0.0026 & **0.4529** & **0.0025** & 0.4521 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison of the MSE and SSIM between MS-D and U-net for all three datasets. For each dataset and metric, the best results are shown in bold. Figure 10: Comparison of U-net and MS-D on the central slice of the upper region of interest. The high-resolution slice (left-most column) is the result of reconstruction (Equation (12)).